The balloon came down in calm weather near Carterton, a small town north of the capital Wellington and a popular area for ballooning, in New Zealand's worst aviation disaster in more than 30 years.
"It appears a fire has ignited on board, causing the hot air balloon to crash in farmland. Sadly, the pilot and 10 passengers onboard have not survived," Wellington district police commander Mike Rusbatch said.
One witness, David McKinlay, told reporters he looked up to see one side of the basket on fire and "all of a sudden there was just 10 metres of flames".
"It was like a rocket coming down; it was just unbelievable," he said.
McKinlay, who alerted the emergency services, said the balloon was about 150 metres in the air when it suddenly plummeted to the ground.
Another witness told Fairfax News he was waving to the passengers as the balloon passed overhead and appeared to hit a power cable.
"The people were enjoying a nice ride and by the looks of it they clipped a power wire," he said.
"Then I heard the screams and looked out the window and heard it coming down. They sounded like screams of joy but they weren't. It wasn't coming from a great deal of height.
"I ran down the road to see if I could help but by that stage it was too late. It was just burned out. By the time the emergency services got there, there wasn't much of a chance."
Two of the passengers are believed to have leaped from the burning basket and were found in a nearby paddock.
Jacqui O'Connor, a nurse holidaying in the area, said she raced to the scene, dodging fallen power lines to reach the victims.
"There was live wire all over the people and the paddock," she said.
It was New Zealand's worst aviation disaster since 1979 when an Air New Zealand jet crashed into Mount Erebus in Antarctica killing 257 people.
Police say they were alerted to the crash just before 7:30 am. Ballooning companies in Carterton, about 150 kilometres (95 miles) north of Wellington, recommend early morning flights.
Reporters at the scene said the crash site has been cordoned off and only emergency workers and the families of those on board the balloon were being allowed through.
The names and nationalities of those on board had yet to be released but Carterton Mayor Ron Mark said he believed they were a mix of locals and tourists.
The balloon was believed to have been owned by Ballooning New Zealand director Lance Hopping, who has more than 1,000 hours of commercial ballooning experience.
Transport Minister Gerry Brownlee said an investigation was under way to determine the cause of the crash and draw up recommendations to prevent a similar accident in future.
"We are deeply sorry to learn of this tragic accident and our hearts go out to those who are now mourning the loss of life," Brownlee said.
"Experts will be looking to learn any lessons from the investigations which have begun today which can help improve safety for others in the future."
Hot air ballooning is tightly governed in New Zealand with operators requiring special certification.
Two years ago, the civil aviation authority banned one company after "serious safety concerns" were uncovered in safety audits and spot checks.
What the stars mean:
★ Poor ★ ★ Promising ★★★ Good ★★★★ Very good ★★★★★ Exceptional