"This destruction is a new war crime and an immense loss for the Syrian people and for humanity," said Irina Bokova, the head of the UN cultural watchdog UNESCO, calling for the perpetrators to be held accountable.
"Daesh (IS) is killing people and destroying sites, but cannot silence history and will ultimately fail to erase this great culture from the memory of the world," Bokova said in a statement.
Syria's antiquities chief Maamoun Abdulkarim told AFP the temple was destroyed on Sunday.
"Our worst fears are sadly being realised," Abdulkarim said.
Famed for its well-preserved Greco-Roman ruins, Palmyra was seized from government forces in May, prompting concerns IS might destroy it as it has other heritage sites in parts of Syria and Iraq under its control.
Initially most of Palmyra's best-known sites were left intact, though there were reports IS had mined them and the group reportedly destroyed a well-known statue of a lion outside the city's museum.
"Daesh placed a large quantity of explosives in the temple of Baal Shamin today and then blew it up," Abdulkarim said late Sunday.
"The cella (inner area of the temple) was destroyed and the columns around collapsed," he said.
- 'Pearl of the desert' -
The Syrian Observatory for Human Rights, a Britain-based group that monitors the country's civil war, confirmed the destruction of the temple.
But the Observatory said Baal Shamin had been destroyed a month ago -- a discrepancy which could not be immediately explained as information on Syria's civil war is often unclear.
IS captured Palmyra on May 21, sparking international concern about the fate of the heritage site described by UNESCO as of "outstanding universal value".
Baal Shamin was built in 17 AD and expanded under the reign of Roman emperor Hadrian in 130 AD.
Known as the "Pearl of the Desert", Palmyra is an oasis town about 210 kilometres (130 miles) northeast of Damascus and was a stopping point for caravans travelling on the Silk Road and between the Gulf and the Mediterranean.
Before the arrival of Christianity in the second century, Palmyra worshipped the Semitic god Bel, whose temple at Palmyra is considered the city's most significant, along with the sun god Yarhibol and lunar god Aglibol.
- Once a tourist site -
Before the Syrian conflict erupted in March 2011, more than 150,000 tourists visited Palmyra every year.
IS mined the ancient site in June before destroying the Lion Statue of Athena outside the Palmyra museum.
Most of the pieces in the museum were evacuated by antiquities staff before IS arrived, though the jihadists have blown up several historic Muslim graves.
IS's harsh version of Islam considers statues and grave markers to be idolatrous.
IS has also executed hundreds of people in the city and surrounding area, many of them government employees, and infamously used children to shoot dead 25 government soldiers in Palmyra's ancient amphitheatre.
Among those it has killed was Khaled al-Assaad, Palmyra's antiquities chief for 50 years, who was beheaded last week after refusing to leave the city following the IS takeover.
In neighbouring Iraq, the jihadist group has razed some relics of ancient Mesopotamia and looted others to sell on the black market.
Syria's war, which began with anti-regime protests, has spiralled into a multi-front conflict that has killed more than 240,000 people.
On Monday, at least 12 civilians, including five children, were killed in regime air strikes on rebel-held Eastern Ghouta outside Damascus, the Observatory said.
Nine of those killed were in the town of Douma, which regime forces have pounded with air strikes in recent days.
"There is a military escalation by the regime on Douma in response to rebel attacks on government positions in Harasta," another part of Eastern Ghouta, Abdel Rahman said.
He said regime forces also dropped barrel bombs on the Jabal Zawiya area in northwestern Idlib province, killing 14 civilians including six children.
State media said 13 people were wounded by rebel fire on Damascus on Monday.