"They wouldn't have dared to come here to the Bardo and ask for exhibits to display in the presidential palace or in the private villas. I would have refused anyway, and resigned on the spot," he said.
From the outside, the modern buildings of the Bardo are just large, white rectangular blocks.
Once inside, however, thousands of years of history unravel before you, from the ancient civilisation of the Phoenicians to the Romans.
Home to the world's largest collection of Roman mosaics, there are also statues, busts of emperors, an important collection of fabrics hailing from the Fatimid dynasty and a hoard of 1,600 gold pieces dating back to 430 AD.
Currently in a bit of a jumble, they await display in the new exhibition rooms, still empty but smelling of freshly-laid plaster and cement.
Ghalia's office is vaguely reminiscent of an Indiana Jones set: bundles of papers, books, plastic-wrapped treasures, not to mention a priceless third century Roman mosaic, "the Virgil", propped casually against a wall.
The curator took up his post in 2003. At the time it was considered a punishment for the Roman and Byzantine-era specialist who dared to oppose "the family" -- Ben Ali and his wife Leila.
"I was responsible for a particular archaeological area and I had to fight to save certain sites where they wanted to build villas by the sea in Nabeul.
"I resisted and as a result I was appointed to Bardo," he said.
Yet he has never regretted it. Even less so after unprecedented protests by the Tunisian people saw the end of their leader's 23-year rule.
The curator hopes the renovation project, which will see the museum's display space and staff double, will help restore people's thirst for culture.
"After January 14 (the fall of Ben Ali) it is absolutely vital that culture finds its place.
"We also need to create a museum of the revolution, with testimonies, films, objects," he added.
The restoration is being funded with the help of a World Bank loan of 25 million dinars (12.5 million euros, 18.6 million dollars) which will go towards research and construction, among other things.
"Displays, lighting, exhibit descriptions in Arabic, French and English, we are getting the best in the world," said Ghalia.
The Bardo, along with the Cairo Museum, are the two biggest south of the Mediterranean, according to the curator.
"The museum was already very impressive before, if a bit mixed up," said Ghalia, who for eight years has had the mammoth task of putting things in a coherent order and retracing the history of Tunisia, of the Arab-Islamic era neglected under Ben Ali's reign.
"It wasn't even about politics," says Ghalia. "It was simply that these people weren't very educated and didn't bother about our heritage.
"We had the collections, now we have the scientific knowledge about all the civilisations which have played a part in Tunisia.
The renovation project is scheduled for completion in November, when Ghalia hopes to see the number of Tunisian visitors climb from the current five to 10 percent of total visitors to closer to 30 or 40 percent.
"If we achieved that, it would be fantastic," he said.
When the new Bardo officially opens its doors, museum goers will find themselves face to face with the world's largest Roman mosaic (120 square metres, 1,292 square feet) and, of course, the famous Virgil.
"It alone symbolises Tunisia's place in Roman civilisation," explains Ghalia proudly.