According to marketing firm ABI, as many as 8 million driverless cars will be added to the road in 2025. Meanwhile, Research and Markets is predicting that in the U.S. alone, there will be some 20 million autonomous cars in operation by 2030.
How realistic are those numbers?
If you ask Adrian Macneil, not especially. And he should know — he’s the director of engineering at Cruise, the self-driving startup that General Motors acquired for nearly $1 billion in 2016. “I think the best way that I’ve heard this described [is], the entire industry is basically in a race to the starting line,” Macneil told VentureBeat in a phone interview. “The penetration of driving the majority of miles with autonomous miles isn’t going to happen overnight.”
Cruise is considered a pack leader in a global market that’s anticipated to hit revenue of $173.15 billion by 2023. Although it hasn’t yet launched a driverless taxi service (unlike competitors Waymo, Yandex, and Drive.ai) or sold cars to customers, it’s driven more miles than most — around 450,000 in California last year, according to a report it filed with the state’s Department of Motor Vehicles. That’s behind only Waymo, which drove 1.2 million miles. Moreover, it’s repeatedly promised to launch a commercial service this year that would feature as many as 2,600 driverless cars without steering wheels, brake pedals, and accelerators.
But it’s been a long and winding path for Cruise since its humble beginnings five years ago, to put it mildly. To get a sense of how far Cruise has come and where it’s going, we spoke with Macneil about Cruise’s ongoing efforts to train cars synthetically, why the company is targeting San Francisco as one of several potential launch cities, and how Cruise fits into the broader self-driving landscape.
Rapid growth
Cruise Automation chief technology officer Kyle Vogt — who held the role of CEO until January, when former GM president Dan Ammann took over — cofounded Cruise with Dan Kan in 2013. Vogt, an MIT computer science graduate and a founding employee of Justin.tv (which became Twitch), started a number of companies prior to Cruise, including Socialcam, a mobile social video app that was acquired by Autodesk for $60 million in 2012. (Amazon purchased Twitch in 2016 for $970 million.)
Vogt can trace his passion for robotics back to childhood. By age 14, he built a Power Wheels car that could drive using computer vision. And while an undergraduate at MIT, he competed with a team in the 2004 Defense Advanced Research Projects Agency (DARPA) Grand Challenge, a $1 million competition to develop a car that could autonomously navigate a route from Barstow, California to Primm, Nevada.
Above: GM: Fourth generation vehicle, the Cruise AV.
Roughly a year after Cruise joined Y Combinator, Vogt teamed up with Dan Kan — the younger brother of Justin.tv’s Justin Kan — and it wasn’t long before they and a small team of engineers had a prototype: the RP-1. The $10,000 direct-to-consumer aftermarket kit retrofitted the Audi A4 and S4 with highway self-driving features (much like the open source stack developed by George Hotz’s Comma.ai), with the goal of supporting additional vehicles down the line.
But at a certain point, they decided to pivot toward building a more ambitious platform that could conquer city driving. Cruise announced in January 2014 that it would abandon the RP-1 in favor of a system built on top of the Nissan Leaf, and in June 2015, it received a permit to test its tech from the California Department of Motor Vehicles.
GM acquired Cruise shortly afterward, in March 2016. Back then, Cruise had roughly 40 employees, a number that quickly ballooned to 100. Cruise had 200 as of June 2017, and it plans to hire over 2,000 new workers — double its current workforce — by 2021.
Growth hasn’t slowed in the intervening months. In May 2018, Cruise — which remains an independent division within GM — announced that SoftBank’s Vision Fund would invest $2.25 billion in the company, along with another $1.1 billion from GM itself. And in October 2018, Honda pledged $750 million, to be followed by another $2 billion in the next 12 years. Today, Cruise has an estimated valuation of $14.6 billion, and the company recently expanded to a larger office in San Francisco and committed to opening an engineering hub in Seattle.
Along the way, Cruise acquired Zippy.ai, a startup developing autonomous robots for last-mile grocery and package delivery, and more recently snatched up Strobe, a provider of “chip-scale” lidar technology. Cruise says that the latter’s hardware will enable it to reduce the cost of each sensor on its self-driving cars by 99%.
Simulating cities
Cruise runs lots of simulations across its suite of internal tools — about 200,000 hours of compute jobs each day in Google Cloud Platform — one of which is an end-to-end, three-dimensional Unreal Engine environment that Cruise employees call “The Matrix.” Macneil says it enables engineers to build any kind of situation they’re able to dream up, and to synthesize sensor inputs like camera footage and radar feeds to autonomous virtual cars.
According to Macneil, Cruise spins up 30,000 instances daily, each of which loops through a single drive’s worth of scenarios. It’s basically like having 30,000 virtual cars driving around in parallel, he explained, and it’s a bit like Waymo’s Carcraft and the browser-based framework used by Uber’s Advanced Technology Group.
Above: Dan Ammann (right), former president of General Motors, with cofounders of Cruise Automation, Kyle Vogt (center) and Dan Kan.
Image Credit: GM
“[The Matrix] is really good for understanding how the entire car behaves [and] also how it behaves in situations that we would not encounter frequently in the real world,” said Macneil. “So if we want to find out what happens, say, if a small object jumps in front of a car or something, we can create those kinds of simulations and reliably reproduce them. If every time you have a software release you deploy to the car and then go out and drive 100,000 or 1,000,000 miles, you’re going to be waiting quite a long time for feedback.”
Another testing approach Cruise employs is replay, which involves extracting real-world sensor data, playing it back against the car’s software, and comparing the performance with human-labeled ground truth data. Yet another is a planning simulation, which lets Cruise create up to hundreds of thousands of variations of a scenario by tweaking variables like the speed of oncoming cars and the space between them.
“We understand how, for example, if we take an updated version of the codebase and play back a construction zone, we can actually compare the results … We can really drill down to a really deep level and understand what our car’s behavior will be,” Macneil said. “If we take something like an unprotected left turn, which is a pretty complicated situation … we can [see how changes] affect how quickly our cars are able to identify [gaps between cars] and whether they choose to take that gap or not.”
Cruise doesn’t measure the number of simulated miles it’s driven, and that’s a conscious decision — Macneil says they prefer to place emphasis on the “quality” of miles rather than the total. “We think more about how the tests that are running hundreds of times a day [are covering a] range of scenarios,” he said. “It’s about more than just racking up a lot of miles — it’s about the exposure to different environments that you’re getting from those miles.”
But while its training data remains closely guarded, some of Cruise’s libraries and tools have begun to trickle into open source. In February, it released Worldview, a graphics stack of 2D and 3D scenes with accompanying mouse and movement controls, click interaction, and a suite of built-in commands. In the coming weeks, it will publish a full-featured visualization tool that’ll allow developers to drill into real-world and simulation data to better understand how autonomous systems — whether cars or robots — respond in certain situations.
Cruise control
In the real world, Cruise uses third-generation Chevrolet Bolt all-electric cars equipped with lidar sensors from Velodyne, as well as radar sensors, video cameras, fault-tolerant electrical and actuation systems, and computers running proprietary control algorithms engineered by Cruise. They also sport in-vehicle displays that show information about upcoming turns, merges, traffic light status, and other information, as well as brief explanations of pauses. Most are assembled in a billion-dollar Lake Orion, Michigan plant (in which GM further invested $300 million last month) that’s staffed by 1,000 people and hundreds of robots.
Cruise is testing them in Scottsdale, Arizona and the metropolitan Detroit area, with the bulk of deployment concentrated in San Francisco. It’s scaled up rapidly, growing its starting fleet of 30 driverless vehicles to about 130 by June 2017. Cruise hasn’t disclosed the exact total publicly, but the company has 180 self-driving cars registered with California’s DMV, and three years ago, documents obtained by IEEE Spectrum suggested the company planned to deploy as many as 300 test cars around the country.
Above: Cruise’s Worldview platform.
Image Credit: GM
Currently, Cruise operates an employees-only ride-hailing program in San Francisco called Cruise Anywhere that allows the lucky few who make it beyond the waitlist to use an app to get around all mapped areas of the city where its fleet operates. The Wall Street Journal reported that Cruise and GM hope to put self-driving taxis into usage tests with ride-sharing company Lyft, with the eventual goal of creating an on-demand network of driverless cars.
Building on the progress it’s made so far, Cruise earlier this year announced a partnership with DoorDash to pilot food and grocery delivery in San Francisco this year for select customers. And it’s making progress toward its fourth-generation car, which features automatic doors, rear seat airbags, and other redundant systems, and it lacks a steering wheel.
Testing and safety
Why the focus on San Francisco? Cruise argues that in densely populated cities, difficult maneuvers (like crossing into multiple lanes of oncoming traffic) happen quite often. Moreover, it points out that San Francisco offers more people, cars, and cyclists to contend with — about 17,246 people per square mile, or five times greater density than in Phoenix.
“Testing in the hardest places first means we’ll get to scale faster than starting with the easier ones,” Vogt explained in a blog post. “Based on our experience, every minute of testing in San Francisco is about as valuable as an hour of testing in the suburbs.”
For instance, Cruise’s Bolts encounter emergency vehicles almost 47 times as frequently in San Francisco as in more suburban environments like Scottsdale and Phoenix, and road construction 39 times more often, cyclists 16 times as often, and pedestrians 32 times as often. They’ve navigated in and around six-way intersections with flashing red lights in all directions and people moving pallets through the street of Chinatown, not to mention bicyclists who cut into traffic without the right of way and construction zones delineated by cones or flares.
“Just driving along in a stretch of road, whether it’s in the real world or in simulation, is not going to give you a huge amount of data,” said Macneil. “One of the reasons why we exist in San Francisco is because we encounter pedestrians, cyclists, construction zones, emergency medical, and all of these things just way more [often] … It’s critically important that we’re testing our cars and combing our real-world driving with our simulations, and with both of those looking to get a lot of coverage of what type of situations they’re encountering.”
The data seems to bear out that assertion. Last year, Cruise logged 5,205 miles between disengagements (instances when a safety driver intervened) in California, a substantial improvement over 2017’s 1,254 miles per disengagement.
Here’s how its average of 0.19 disengagements per 1,000 miles compared with others:
- Waymo: 0.09 disengagements per 1,000 miles
- Zoox: 0.50 disengagements per 1,000 miles
- Nuro: 0.97 disengagements per 1,000 miles
- Pony.ai: 0.98 disengagements per 1,000 miles
Assuming Cruise’s tech works as promised, it could be a godsend for the millions of people who risk their lives every time they step into a car. About 94% of car crashes are caused by human error, and in 2016, the top three causes of traffic fatalities were distracted driving, drunk driving, and speeding.
But will it be enough to convince a skeptical public?
Three separate studies last summer — by the Brookings Institution, think tank HNTB, and the Advocates for Highway and Auto Safety (AHAS) — found that a majority of people aren’t convinced of driverless cars’ safety. More than 60% said they were “not inclined” to ride in self-driving cars, almost 70% expressed “concerns” about sharing the road with them, and 59% expected that self-driving cars will be “no safer” than human-controlled cars.
They have their reasons. In March 2018, Uber suspended testing of its autonomous Volvo XC90 fleet after one of its cars struck and killed a pedestrian in Tempe, Arizona. Separately, Tesla’s Autopilot driver-assistance system has been blamed for a number of fender benders, including one in which a Tesla Model S collided with a parked Culver City fire truck. (Tesla temporarily stopped offering “full self-driving capability” on select new models in early October 2018.)
The Rand Corporation estimates that autonomous cars will have to rack up 11 billion miles before we’ll have reliable statistics on their safety — far more than the roughly 2 million miles the dozens of companies testing self-driving cars in California logged last year. For his part, Macneil believes we’re years away from fully autonomous cars that can drive in most cities without human intervention, and he says that even when the industry does reach that point, it’ll be the first of many iterations to come.
“When you put the rates of improvement at the macro scale and you look at the entire industry, once we get the full self-driving cars on the road that have no safety driver in them and serving passengers, that’s just the first version, right?” he said. “There’s still an endless array of different weather conditions to handle, and different speeds, different situations, long-distance driving, and driving in snow and rain.”
Competition and unexpected detours
For all of its successes so far, Cruise has had its fair share of setbacks.
It backtracked on plans to test a fleet of cars in a five-mile square section in Manhattan, and despite public assurances that its commercial driverless taxi service remains on track, it’s declined to provide timelines and launch sites.
In more disappointing news for Cruise, the firm drove less than 450,000 collective miles all of last year in California, falling far short of its projected one million miles a month. (Cruise claims that the initial target was based on “expanding [its] resources equally across all of [its] testing locations,” and says that it’s instead chosen to prioritize its resources in complex urban environments.) For the sake of comparison, Alphabet’s Waymo, which was founded about four years before Cruise, has logged more than 10 million autonomous miles to date.
In a report last year citing sources “with direct knowledge of Cruise’s technology,” The Information alleged that Cruise’s San Francisco vehicles are still repeatedly involved in accidents or near-accidents and that it’s likely a decade before they come into wide use in major cities. Anecdotally, one VentureBeat reporter experienced a close call while crossing the road in front of a Cruise test vehicle in San Francisco.
Then, there’s the competition to consider.
Cruise faces the likes of Ike and Ford, the latter of which is collaborating with Postmates to deliver items from Walmart stores in Miami-Dade County, Florida. There’s also TuSimple, a three-year-old autonomous truck company with autonomous vehicles operating in Arizona, California, and China, as well as venture-backed Swedish driverless car company Einride. Meanwhile, Paz Eshel and former Uber and Otto engineer Don Burnette recently secured $40 million for startup Kodiak Robotics. That’s not to mention Embark, which integrates its self-driving systems into Peterbilt semis (and which launched a pilot with Amazon to haul cargo), as well as Tesla, Aptiv, May Mobility, Pronto.ai, Aurora, NuTonomy, Optimus Ride, Daimler, and Baidu, to name a few others.
Vogt believes that Cruise’s advantage lies in its distributed real-world and simulated training process, which he claims will enable it to launch in multiple cities simultaneously. In a GM investor meeting last year, Vogt conceded that the cars might not match human drivers in terms of capability — at least not at first. But he said that they should quickly catch up and then surpass them.
“Building a new vehicle that has an incredible user experience, optimal operational parameters, and efficient use of space is the ultimate engineering challenge,” he wrote in a recent Medium post. “We’re always looking for ways to accelerate the deployment of self-driving technology, since it’s inherently good in many different ways … We’re going to do this right.”