Numbers have dwindled since the last British troops left in 1948 as many Anglo-Indians left the country of their birth for new lives in Britain, Australia, the United States or Canada.
Some members of Baldrey's family have done the same, but when he was a child, the rollcall at scholl could have been at any school in Britain, with Wilsons, Lyons, Bentleys, Middletons, Woodmans and McNamaras.
His family lived in railwaymen's accommodation in the important railway town of Bhusawal, northeast of Mumbai, with a neat lawn and mango trees, and chickens, ducks and geese running in the yard, he recalled.
"There were 30 or 40 Anglo-Indian families at that time. We had a great time but those days are only memories, they have all gone," the 56-year-old bachelor told AFP in an interview.
"I'm the last Anglo-Indian motorman left on the Central Railway. I don't think there's anyone following in my footsteps."
Baldrey who is Christian, speaks fluent if accented English and stands a strapping six feet (1.8 metres) tall, has never visited the country of his ancestors.
For him, it is a distant concept, as far away as Mumbai's landmark main raliway station, Chhatrapati Shivaji Terminus, is from its architectural counterpart in London, St Pancras station.
Baldrey began his career as an assistant train driver in 1974, following his father, Eric, who started out as a fireman on steam engines in the city of Nagpur in 1939.
Both his grandfathers were drivers on the Indian railways, which first began criss-crossing the sub-continent in the 1850s.
His lifelong love affair with the profession began as a child when he travelled on the footplate of his father's engine as it chugged from Bhusawal to Nagpur and beyond.
"I was quite curious, you see, to see what dad's work was all about. I just thought: this is a very adventurous job," he explained with a smile, running his hand through his black hair.
His mind was made up, he said, in his mid-teens and by the time he was 22, he achieved his boyhood dream when he started work for one of the biggest regional networks on Indian Railways.
"It's very interesting, very challenging and very adventurous. Of course, there are ups and downs, sometimes we have bad times, sometimes we have good times, he said.
By "bad times," Baldrey means dealing with the brutal reality of the Mumbai rail network: the dozen or so people a day who are killed when they are hit by or fall off the city's overcrowded trains.
"We have trespassers where we mow down people. It's not our fault. We have to clear the tracks. They are the bad times you face," he said, adding: "Of course, you get seasoned to it, this is part of our job."
On any given day, Baldrey can be found plying the 80- to 100-mile (130- to 160-kilometre) routes from Chhatrapati Shivaji Terminus to Kasara on the northeast line and Khopoli, to the southeast of the city.
He then returns home to Kalyan, a northeastern suburb of India's financial capital, where he lives with his 43-year-old brother David and his family, and their 79-year-old mother Monica.
Sister Sharon, 50, lives nearby and another brother, Glenn, 54, also works on the railways, as an engineer.
In his spare time, Baldrey can be found at the local Pentecostal church or working out to maintain his sturdy physique.
His mother arrived in India in the mid-1950s, as Britain endured post-war austerity and the death throes of its once-mighty Empire, only returning to see her family nearly half a century later.
"I don't know much about Britain," said Baldrey, who also has family in west London.
"I would have loved to have gone to see my aunties and uncles, my dad's brothers. God willing, I could visit after my retirement. It might be possible."