"No amount of (UN) Security Council sessions, reports or briefings could have prepared me for what I witnessed today," he said after being driven through the ruins of Gaza City's Shejaiya district and the nearby Jabaliya refugee camp.
The secretary general was speaking at a UN school in Jabaliya, where tank shells slammed into two classrooms on June 30, killing at least 14 people sheltering there.
"The shelling of the United Nations school is absolutely unacceptable. These actions must be fully and independently investigated," he said.
Relatives of the dead held up posters showing their loved ones and disabled casualties waited to see Ban.
The UN chief also called on Palestinian militant groups to cease firing rockets at Israel from the territory.
"I repeat here in Gaza the rockets fired by Hamas and other military groups must end. They have brought nothing but suffering," he said.
One classroom, now repaired, had the words "every human being has the right to life" written on its walls.
After meeting with members of a new Palestinian consensus government, Ban told reporters the devastation he had seen was worse than that caused in the previous conflict of winter 2008-2009.
"This is a much more serious destruction than what I saw in 2009," he said.
Ban, who last visited in 2012, said at a donors conference in Egypt Sunday that his trip to the enclave was "to listen directly to the people of Gaza."
He said the international pledges of reconstruction aid were "quite encouraging."
Donations include $1 billion from Qatar, $212 million from the United States and 450 million euros from the EU.
Provision of the aid will be overseen jointly by the UN and the West Bank-based Palestinian Authority, amid concerns that unchecked imports could fall into the hands of militants, including those of the Islamist movement Hamas.
Hamas and its rival Fatah, which dominates the PA, signed a unity deal in April under which the consensus government was sworn in.
Ban welcomed the rapprochement.
- 'Great opportunity' -
"This is a great opportunity to unite the West Bank and Gaza under one Palestinian leadership," he said.
Ban said the funds would go towards the "urgently needed" building of infrastructure and homes in Gaza, where nearly 2,200 Palestinians, mostly civilians, were killed and tens of thousands displaced in the 50-day war.
On the Israeli side, 73 people were killed, mainly soldiers.
At Sunday's conference in Cairo, Ban said "the root causes of the recent hostilities" were "a restrictive occupation that has lasted almost half a century, the continued denial of Palestinian rights and the lack of tangible progress in peace negotiations."
On Tuesday, he urged Israeli and Palestinian leaders to revive peace talks.
"I'm asking the leaders of both parties... to resume their talks," he said. "Otherwise it's a matter of time that the violence will continue."
Ban said a first shipment of building materials was on its way to Gaza through Israel under an agreement reached last month.
"I'm very happy to announce that the first truck carrying... construction materials is coming to Gaza today," he said.
The Israeli army said it had "transferred construction materials to the Gaza Strip in order to facilitate rehabilitation projects."
It said the supplies were "expected to include 600 tons of cement, 50 trucks of construction aggregates and 10 trucks of metal."
On a visit Monday to the West Bank city of Ramallah, Ban criticised continued Israeli settlement expansion.
"I once again strongly condemn the continued settlement activity by Israel," he said, echoing international condemnation of plans for new settler homes on occupied Palestinian territory.
The White House and European Union have criticised Israel's approval in September of 2,600 new settlement units in Israeli-annexed Arab east Jerusalem.
The issue has caused the breakdown of numerous rounds of peace talks.
------------------------------------------------------------------------------------------------------------