Their arrival followed the decision by Hungary, which has become a flashpoint in the crisis, to bus thousands of migrants stranded in Budapest for days to the Austrian border.
The packed buses, in which people were strewn across the floors, using bottles for pillows, departed all through the night.
Hungary laid on the buses after around 1,200 migrants set out on foot from Budapest for the Austrian border, 175 kilometres (110 miles) away, and after large numbers escaped from refugee camps.
Hungarian Foreign Minister Peter Szijjarto blamed Saturday the "failed migration policy of the EU and ... irresponsible statements made by some European politicians."
At least 500 more migrants embarked on similar march from Budapest towards the Austrian border Saturday, an AFP correspondent in the Hungarian capital said.
There appeared to be no more buses however to ferry them to the frontier.
- Trains to Germany -
People arriving off buses earlier walked across the frontier to the Austrian town of Nickelsdorf where authorities had set up a makeshift shelter.
"The streams (of people) keep coming," Hans Peter Doskozil, chief of police in Burgenland state, who was at the border, said Saturday morning.
Looking exhausted but happy, most boarded special buses and trains to Vienna, to take trains bound for Salzburg and from there continue to Munich, or other services running to other German cities.
German police said they expected up to 7,000 of the migrants to reach German territory Saturday.
In Vienna, the migrants, some carrying sleeping children and many wrapped in blankets, were greeted by large numbers of volunteers handing out food, drinks, sanitary products and train tickets.
"My toes hurt, a lot of blood, we walked too much. I want to go (to) Germany, but then I stop," one 26-year-old Syrian man from Homs, both his feet wrapped in thick bandages, told AFP.
One refugee held up a big sign saying "Austrians Danke schoen" ("Austrians thank you").
"After endless examples of shameful treatment by governments of refugees and migrants in Europe, it is a relief to finally see a sliver of humanity," said Amnesty International's Gauri van Gulik.
"But this is far from over, both in Hungary and in Europe as a whole. The pragmatic and humane approach finally applied here should become the rule not the exception."
- Human cost -
Hundreds of thousands of migrants have been making often perilous journeys to Europe this summer, most of them trying to reach countries in western Europe, particularly Germany.
New data from the UN refugee agency (UNHCR) Saturday showed that 366,402 migrants have landed in Europe via the treacherous sea route this year, with a further 2,800 dead or missing.
Forty-nine percent were escaping the brutal civil war and Islamic State extremists in Syria. Germany's government, which has said it will no longer deport Syrian refugees, expects to receive 800,000 asylum applications this year.
The human cost of the crisis was made horrifically clear this week by photographs of the body of a three-year-old Syrian, Aylan Kurdi, washed up on a beach in Turkey.
The UN High Commissioner for Refugees Antonio Guterres warned Friday the 28-member EU faced a "defining moment" and called for the mandatory resettlement of 200,000 refugees by EU states.
EU foreign ministers met in Luxembourg on Saturday to discuss the crisis ahead of a "State of the Union" address next week by European Commission President Jean-Claude Juncker
Juncker has proposed mandatory quotas for resettling 160,000 refugees, despite an earlier plan for 40,000 meeting stiff opposition, notably from Hungary, and attracting offers of places for only 32,000.
Germany and France back quotas but Hungary, the Czech Republic, Poland and Slovakia together rejected any such imposition in a statement on Friday.
Hungarian Prime Minister Viktor Orban during the week called the crisis a "German problem".
"Nobody wants to stay in Hungary, neither in Slovakia, nor Poland, nor Estonia. All want to go to Germany," he said.
Hungary has become a major entry point for migrants trekking up from Greece through the western Balkans, with some 50,000 entering the EU member state in August alone and several thousand more every day.
Hungary has responded by criminalising unauthorised border crossing and any damage to a razor-wire fence recently erected along the border with Serbia.
"We must not forget that those who are coming in have been brought up under a different religion and represent a profoundly different culture," Orban wrote in a German newspaper this week.
"The majority are not Christians but Muslims. That is an important question because Europe and European culture have Christian roots."