An official announcement said he had removed the governors of Aden, Lahij and Abyan in the south, as well as Hadramut in the southeast and Hudayda in the west.
Saleh opponents massed from early morning in streets leading to a square near Sanaa University, where students and pro-democracy demonstrators have been camped for more than a week.
"The people want Ali Abdullah Saleh to leave," they chanted. "The people want to overthrow the regime."
Many protesters marched in white shrouds reading: "Either we live happily or we die as martyrs."
The opposition said half a million people turned out, although an AFP correspondent said that number was exaggerated.
"Since the sit-in began near Sanaa university, we have not witnessed such a turnout," said one of the organisers, Hashem al-Sufy.
The ruling General's People Congress gave an even wilder exaggeration of the number of people at the counter-demonstration in Tahrir Square, putting it at 1.5 million. An AFP correspondent said the reality was a fraction of that.
Several opposition figures spoke, including leading cleric Abdul Majeed al-Zendani, who is called a "global terrorist" by Washington.
Zendani said the protest movement sweeping the Arab world was "a new, effective, fast, and non-destructive means of changing regimes."
Saleh hit back, in an address at Sanaa University. "The events from Tunisia to Oman are a storm orchestrated from Tel Aviv and under Washington's supervision," he said.
"What is taking place on Yemen's streets is just a copycat attempt," he said. The protesters are "led from outside" and are in the pay of "Zionists."
Saleh lashed out at US President Barack Obama for his repeated calls for restraint by Arab regimes that had long been key allies and his support for the popular protests that ousted veteran leaders in Egypt and Tunisia.
"Every day we hear a statement by Obama... (saying) Egypt don't do this, Tunisia don't do that... What does Obama have to do with Oman, what does he have to do with Egypt? You are the US president," he said.
Washington warned Saleh to stop blaming the US and Israel.
"We don't think scapegoating will be the kind of response that the people of Yemen or the people in other countries will find adequate," White House spokesman Jay Carney said.
In London, meanwhile, the Foreign Office said it had summoned the charge d’affaires from the Yemeni embassy to express concern over reports of "deaths and casualties" in the Yemeni protests and urged the government to "listen to the legitimate grievances of the Yemeni people."
Saleh, in power since 1978, has repeatedly rejected calls for his resignation, insisting he will defend his regime "with every drop of blood."
His one concession has been to pledge not to seek re-election in 2013, which the opposition has dismissed as inadequate.
Thousands of protesters also turned out on Tuesday in the southern province of Lahij.
In Seiyun, in Hadramut province further east, 19 demonstrators were wounded, two by gunfire, when they clashed with security forces, witnesses and medics said.
Protesters had tried to tear a poster of Saleh at a construction site in the town, witnesses said. Six workers were also wounded.
In the main southern city of Aden, thousands protested in Al-Mansura district carrying black flags in mourning for those killed in clashes with police, witnesses said.
Demonstrations were also reported in Maalla and Crater districts.
The UN human rights chief warned Yemen against using violence against peaceful protesters.
"People have the legitimate right to express their grievances and demands to their government," UN High Commissioner for Human Rights Navi Pillay said in a statement issued in Geneva.
"We have seen over and over again in the past few weeks that violent responses, in breach of international law, do not make the protesters go away and only serve to exacerbate their frustration and anger."
At least 19 people, mostly in Aden, have been killed in protests and clashes across Yemen since February 16, according to an AFP tally based on reports by medics and witnesses.
Rights group Amnesty International puts the toll at 27.