The BBC said a blog by Rippon, in which he explained why a show about Savile was dropped in December last year for editorial reasons, was "inaccurate or incomplete in some respects" and has been corrected.
Cameron said the country was "appalled" by the growing claims against Savile, who with his jangling jewellery and shiny tracksuits was one of British television's best-loved stars before his death last year aged 84.
"The nation is appalled, we're all appalled by the allegations of what Jimmy Savile did and they seem to get worse by the day," Cameron told reporters.
"The developments today are concerning because the BBC has effectively changed its story about why it dropped the Newsnight programme about Jimmy Savile."
The prime minister added that "serious questions" need to be answered by the two independent inquiries that the BBC has launched into the allegations about Savile.
British police have launched a separate criminal investigation.
The scandal has pitted the BBC against itself, with the broadcaster's investigative show Panorama due to claim on Monday that the corporation pulled the Newsnight investigation after coming under pressure from senior managers.
The BBC was allegedly reluctant to run the Newsnight investigation because it had already scheduled tributes to Savile for the Christmas period in 2011, following his death in November that year.
Rippon's departure -- just hours after the BBC denied a report in the Daily Mail newspaper saying he was going to go -- failed to stem the pressure on the corporation.
"The BBC has announced that Peter Rippon is stepping aside with immediate effect from his post," the BBC said, adding that it "regrets" the errors.
-- 'Worst crisis in 50 years' --
In the corrections posted by the BBC on Monday, the broadcaster said Rippon had incorrectly said in his blog that the corporation had no evidence that anyone from a children's home linked to the scandal knew about the abuse.
The BBC also said that while Rippon said there was no evidence that the corporation knew of Savile's activities at the time, there were in fact allegations of abuse on BBC premises.
It also said that some alleged abuse victims interviewed by Newsnight had not spoken to police, while Rippon had initially insisted that all of them had.
The row is potentially damaging for new BBC director general George Entwistle, with Monday's Panorama show reportedly set to say that his accounts of why the Newsnight programme was dropped were misleading.
Entwistle is due to testify to British lawmakers on Tuesday.
Former BBC director general Mark Thompson, who is due to start as CEO of the New York Times in November, may also be called to give evidence, the Daily Mail reported.
The Savile scandal has snowballed since a programme by the BBC's commercial rival ITV aired allegations about the entertainer by a handful of women two weeks ago.
Scotland Yard says it now believes there may be 200 possible victims. It has also said it is investigating suspects who are still alive.
Savile raised huge sums for charity in addition to his high profile on television, but lurid claims that he sexually abused underage girls have left his reputation in shreds and the BBC facing accusations of a cover-up.
Veteran BBC foreign editor John Simpson said in an interview to be aired on the Panorama programme that the Savile claims have plunged the broadcaster into "the worst crisis that I can remember in my nearly 50 years at the BBC."
But the BBC has weathered storms before.
In 2004 the BBC's then-director general and chairman both resigned after a judge-led inquiry into the death of Iraq weapons inspector David Kelly strongly criticised the BBC.
---------------------------------------------------------------------------------------------