Adnan Tahir was fined £270 and issued three penalty points when he refused to let the pair exit his private hire car to go to a cash machine in 2018.
The child hurt his head as a result.
Mr Tahir, who has been transporting people around the city since March 2017, was brought before the licensing committee to explain himself.
He said: “The woman told me to stop because she needed to go to a cash machine. I did stop close to it but there was a car coming behind me, so I decided to move away.
“I didn’t realise that she was trying to get out of the car. I apologised to my customer and offered to take them to the hospital but the woman refused and asked me to take them horse riding instead.”
It wasn’t until two days after the incident that Mr Tahir received the complaint. Members were then given the opportunity to question the applicant about his conduct.
Councillor Alex Wilson said: “I appreciate that you held your hands up and have admitted your mistake.
“We need to ensure that our customers can get from A to B without getting injured. You had a young person in your care and what Police Scotland have told us quite distressing.”
It was also brought to the attention of the board that Mr Tahir had also failed to stop at a red light, while working as a private hire car driver, with passengers on board.
Tahir said: “This was at a pedestrian crossing. As I was driving towards the traffic lights they were green and only turned to red as I was going through them.”
Councillor Martha Wardrop said: “You have been driving for us for two-and-a-half years and there have already been two incidents.
“This is a case of dangerous driving. You have affected a child and their mother because of your behaviour and have not observed the road signs.”
Following the discussion councillors refused to renew Mr Tahir’s licence.