Controversial engineer: I travelled over 3,000 miles in a self-driving car

Anthony Levandowski, the controversial engineer at the heart of a lawsuit between Uber and Waymo, claims to have built an autonomous car that drove from San Francisco to New York without any human intervention.

The 3,099-mile journey started on 26 October on the Golden Gate Bridge, and finished nearly four days later on the George Washington Bridge in Manhattan.

The car, a modified Toyota Prius, used only video cameras, computers and basic digital maps to make the cross-country trip.

Levandowski told the Guardian that, although he was sitting in the driver’s seat the entire time, he did not touch the steering wheels or pedals, aside from planned stops to rest and re-fuel. “If there was nobody in the car, it would have worked,” he said.

If true, this would be the longest recorded road journey of an autonomous vehicle without a human having to take control. Elon Musk has repeatedly promised, and repeatedly delayed, one of his Tesla cars making a similar journey.

A time-lapse video of drive, released to coincide with the launch of Levandowski’s latest start-up, Pronto.AI, did not immediately reveal anything to contradict his claim. But Levandowski has little store of trust on which to draw.

In 2017, Waymo accused him of stealing self-driving secrets when he left Google to form another self-driving truck start-up, Otto, which was swiftly acquired by Uber. Levandowski pleaded the fifth hundreds of time in depositions, although Uber settled the case (and fired Levandowski) before he was called to testify in court. He has also been accused by regulators in Nevada and California of illegally testing automated vehicles there.

“I don’t really dwell on the past,” Levandowski told the Guardian during a ride in the Prius along highways near Pronto’s San Francisco offices last week. “At the end of the day, what matters is facts and reality. I’m very proud that we were able to achieve, in my mind, a pretty monumental self-driving milestone.”

Pronto.AI will not be selling Levandowski’s new technology in a self-driving vehicle, nor using it for passenger cars at all. Instead, it will form the basis of an advanced driver assistance system (ADAS) called Co-Pilot, offering lane keeping, cruise control and collision avoidance for commercial semi-trucks. Similar technology is already available for some luxury cars, notably Tesla’s Autopilot, and it requires an alert human driver to pay attention at all times.

Levandowski confirmed that he acted as a safety driver on Pronto’s coast-to-coast trip, ready to take over should the system have failed.

“Driving a truck is a really hard job, and we think Co-Pilot can make it a lot easier on drivers, and reduce fatigue, while increasing safety,” said Levandowski. Large truck crashes kill nearly 4000 people each year in the US, according to Department of Transportation statistics.

Ognen Stojanovski, a lawyer and research scholar at Stanford University who co-founded Pronto, said, “Trucking is a tight margin business. Driver retention is a huge cost, and if we can add even a little bit of safety, lower claims from less severe crashes will make a huge difference.”

The system does not use laser-ranging lidars like those that Levandowski helped to develop at Waymo, Otto and Uber. This is not because he is afraid of more lawsuits, Levandowski insists, but because he now believes that lidars are an expensive and unnecessary red herring in the quest for robotic vehicles.

The fact that completely driverless cars do not yet exist is not because lidar technology is not good enough, Levandowski said, but because the software is not good enough.

Pronto.AI’s driving technology uses only six video cameras, pointing to the front, side and rear of the vehicle, and each with a much lower resolution than those found in modern smartphones. Images from the cameras are fed to the trunk, where a computer is running two neural networks: artificial intelligence systems that can speedily process large quantities of data.

One network recognizes lane markings, signs, obstacles and other road users, and extracts information about their position and speed. The second takes that information and controls the driving, using digital signals and mechanical actuators for the throttle, brake and steering.

A seventh camera faces inwards, watching the human driver to ensure that they are keeping their eyes on the road. Should the driver look away, nod off, or pull out a cellphone, the system sounds increasingly strident alerts and could ultimately be programmed to stop the vehicle. The Guardian saw this system in operation.

Pronto will begin selling the Co-Pilot in the first half of 2019, initially as a $5,000 aftermarket installation for newer trucks. Levandowski says the company will interview and then train prospective customers so they know what the system can, and cannot, do.

Pronto’s AI-powered approach allows Co-Pilot to drive without the extremely detailed digital maps that many rival automated vehicle technologies require, Levandowski said, as well giving it the flexibility to respond intelligently to unfamiliar situations.

“There are more self-driving scenarios that we need to handle than there are atoms in the universe,” said Levandowski. This is a reference to the famously complex board game Go, at which Google’s AlphaGo AI beat the best human players in 2016.

Co-Pilot is still a long way off matching even an average human driver, however. The highway-only system has not been trained to drive on city streets, where pedestrians, cyclists, narrow roads and oncoming traffic make driving exponentially more difficult.

During the Guardian’s 48-mile test ride, it drove safely and competently, and succeeded in changing lanes several times on its own initiative. However, at one point, Levandowski took the wheel after the car failed to merge into busy traffic. Such hiccups are called disengagements in the self-driving world. Levandowski attributed the disengagement to the latest version of Pronto’s constantly evolving software.

Completing the transcontinental voyage also took multiple attempts. The first try, in late September, ended on the Bonneville Salt Flats in Utah, when the system disengaged on a banked curve in high winds. On its second go, two weeks later, Levandowski says the Co-Pilot worked perfectly for 650 miles, again as far as Utah. But it was too perfect for one Nevada Highway Patrol officer, who pulled the Prius over after noticing it driving slightly below the speed limit in an area where most drivers were speeding.

“The team tried to tell me that it wasn’t a disengagement, but I said, I can’t touch the steering wheel, brake or gas otherwise everybody’s going to look for the gotcha. So we came back to San Francisco,” recalls Levandowski.

Pronto engineers adjusted the software so that the car would be allowed to travel faster on certain roads, and tried again. On his third trip, Levandowski said that he encountered rain in Nebraska and Illinois, high winds in Wyoming, and a rolled-over semi in Pennsylvania, but eventually made it to the George Washington Bridge without a disengagement.

“If true, a truck that used only cameras to steer, brake, and accelerate for 100% of any cross-country trip is impressive,” said Bryant Walker Smith, a law professor at the University of South Carolina and member of the US Department of Transportation’s Advisory Committee on Automation in Transportation. “Making a system work with cameras alone could be a major contribution, especially if this could be applied to higher levels of driving automation.”

Missy Cummings, director of the Humans and Autonomy Laboratory at Duke University, remains deeply suspicious. “Anthony’s job is to make claims that may be at the edge of what his technology is capable of,” she said. “I have not seen evidence of amazing breakthroughs that would be a game-changer in driverless car technology, particularly if it’s only relying on cameras.”

The CEOs of two self-driving start-ups, who asked not to be identified, were also skeptical but agreed that such a trip would represent a significant advance. “The real test is how repeatable it is,” said one. The other added that Levandowski remains “radioactive” in the industry, and speculated that he would find it difficult to raise funds because of his checkered past.

Levandowski’s immediate task is less thorny: to sell a small number of prototype Co-Pilots and transfer the technology from Pronto’s Prius to commercial trucks.

“I’ve learned a lot in the last couple of years about how to do engineering, both on the technical side but also how to operate and be more responsive to people’s criticisms,” said Levandowski. “We’re not promising the moon. We want to promise things that are very concrete and that we can deliver.”

Pronto.AI is not alone in wanting to reboot trucking. Dozens of transportation start-ups are working on partially automated, driverless and even remote control semis, with Levandowski’s former employer, Waymo, already testing its own self-driving trucks in Georgia, California and Arizona.

Despite re-entering such a crowded market, Levandowski feels that his legal difficulties, at least, are now behind him. “I don’t expect letters from any lawyers,” he said. “The technology has been built from scratch and we have the logs to show every keystroke. It’s a totally new approach.”