The man, speaking in French, issues threats against Jews in France, before the boy walks around in front of the hostage and then shoots him in the forehead using a pistol.
The boy, who shouts "Allahu Akbar" ("God is greatest" in Arabic), then shoots the man four more times as he lies on the ground.
In February, Musallam's father denied in comments to AFP the IS claim that his son was an Israeli agent.
"My son is innocent; IS accused him of working for Mossad because he tried to run away," Said Musallam said, claiming that his son had travelled to Syria to join the jihadists.
Following release of the video Tuesday, Musallam said he did "not know anything".
"People are calling us; they told us IS sent a video in which Mohammed says he works for the Israeli Shin Bet. That's not true; my son doesn't work for the Shin Bet. IS did this to scare the world.
"He's dead, he's a martyr. Mohammed is just a child, like a baby, only 19 years old."
- 'Absolutely not religious' -
"He is absolutely not religious," Musallam said earlier, adding that perhaps his son had been recruited through the Internet.
Dabiq, the IS online English-language magazine, said Mohammed had been tasked by Israel with gathering information on IS bases and weapons and on Palestinians volunteering to fight for the group in Syria.
A spokesman for Israel's Shin Bet security service told AFP Mohammed had left home on October 24 "on his own initiative", implying that he had not been sent by Israel.
Official Israeli policy is not to comment on such allegations.
The Shin Bet spokesman said Mohammed travelled from Israel to Turkey and "from there crossed to Syria and joined IS to fight for the organisation".
Said Musallam said his son, an Israeli citizen, abandoned his national service in the fire department to join IS.
He said the last time they spoke by phone Mohammed was in Raqa, the self-proclaimed capital of the jihadist "caliphate" in northern Syria.
He said his son wanted to return home after having completing basic training.
He said he was very worried about his son's fate at the hands of IS, which has brutally executed many prisoners in recent months.
Asked if he could raise a ransom in exchange for his son, the father said he was a mere bus driver with no property or assets to his name.
The family lives in the Jewish settlement neighbourhood of Neve Yaakov in Israeli-annexed east Jerusalem.
Said Musallam was born in the occupied West Bank but said his four children have Israeli passports.
According to Dabiq, Mohammed was recruited in Neve Yaakov by a Jewish neighbour working for the security services and was given away by his behaviour and his refusal to obey a commander.
The execution video released Tuesday was one of several produced by IS, starting with the beheading last August of American journalist James Foley and more recently including the burning to death of a captured Jordanian pilot and the beheading of 21 Coptic Christians.
Dalia Ghanem-Yazbeck, an analyst at the Carnegie Middle East Center in Beirut, said the videos tend to coincide with jihadist military setbacks.
"With each military reverse... IS comes out with a shock video so that people will talk about them. This is a way to compensate for the military defeat with propaganda," she said.
Since early this month, Iraq forces have besieged Tikrit, a city captured by IS in a lightning sweep across the country's north last summer.
The video campaign is also seen by analysts as a recruitment vehicle for IS, which is thought to have between 25,000 and 80,000 fighters, including thousands of foreigners.