Forces under the command of the ousted president's son Ahmed, who remains a powerful figure in the restive nation, laid siege to the defence ministry headquarters in Sanaa before attacking it with machineguns and rocket-propelled grenades, witnesses told AFP.
Hundreds of Republican Guardsmen blocked the roads leading to the ministry in central Sanaa, the witnesses said.
They said the forces, who had come from the main base of the Republican Guard in Sawad, south of the capital, exchanged fire with police on their way to the ministry.
The United States urged all sides to show restraint and respect reforms by the new president.
"We have been urging restraint on all sides, an immediate end to the violence and respect for President Hadi and the reforms that he is putting in place and the democratic transition," State Department spokeswoman Victoria Nuland told reporters.
The United States has been a key backer of Yemen's post-Arab Spring political transition -- which was brokered by its wealthy Gulf Arab neighbours -- and allowed Saleh to visit the United States to be treated for burns he suffered in an attack on his palace last year.
Washington regards Al-Qaeda's Yemen arm as the jihadists' biggest threat to the West and has stepped up drone attacks in recent months in support of an offensive by the Yemeni army to recapture areas lost to the militants during the upheaval that accompanied Saleh's downfall last year.
Government troops managed to repel the attack by Saleh loyalists, arresting a number of them and besieging others in buildings close to the ministry, witnesses said.
Troops guarding the ministry used armoured vehicles to chase the attackers. They were backed by military police and reinforcements from the army's Fourth Armoured Brigade.
Army reinforcements were deployed around the residence of Hadi, who replaced Saleh in the Gulf-brokered peace deal that ended 13 months of protests against the veteran leader, witnesses said.
Troops were also stationed around the central bank as employees were evacuated.
In a presidential decree last week, Hadi ordered the formation of a "presidential protection" force which will include three brigades from the Republican Guard.
The force will also include a brigade from the First Armoured Division led by General Ali Mohsen al-Ahmar who defected to the anti-Saleh opposition last year, official news agency Saba reported.
Hadi assigned other brigades from the two rival units to the central and southern regions, it added.
Under the terms of the peace agreement, Hadi is tasked with restructuring the military and security apparatus, which had been controlled by officials with close ties to Saleh and accused of corruption.
On July 31, a group of policemen occupied the interior ministry and clashed with other forces in a gunfight that left 15 people dead.
Yemen's Supreme Security Committee blamed the assault on "a group of inciters among the ranks of the police force aiming to undermine security," and the government accused those behind the attack of "seeking to spread chaos in a desperate attempt to undermine the political process in Yemen."