Uber’s Self-Driving Cars Were Struggling Before Arizona Crash
SAN FRANCISCO — Uber’s robotic vehicle project was not living up to expectations months before a self-driving car operated by the company struck and killed a woman in Tempe, Ariz.
The cars were having trouble driving through construction zones and next to tall vehicles, like big rigs. And Uber’s human drivers had to intervene far more frequently than the drivers of competing autonomous car projects.
Waymo, formerly the self-driving car project of Google, said that in tests on roads in California last year, its cars went an average of nearly 5,600 miles before the driver had to take control from the computer to steer out of trouble. As of March, Uber was struggling to meet its target of 13 miles per “intervention” in Arizona, according to 100 pages of company documents obtained by The New York Times and two people familiar with the company’s operations in the Phoenix area but not permitted to speak publicly about it.
Yet Uber’s test drivers were being asked to do more — going on solo runs when they had worked in pairs.
And there also was pressure to live up to a goal to offer a driverless car service by the end of the year and to impress top executives. Dara Khosrowshahi, Uber’s chief executive, was expected to visit Arizona in April, and leaders of the company’s development group in the Phoenix area wanted to give him a glitch-free ride in an autonomous car. Mr. Khosrowshahi’s trip was called “Milestone 1: Confidence” in the company documents.
Tech companies like Uber, Waymo and Lyft, as well as automakers like General Motors and Toyota, have spent billions developing self-driving cars in the belief that the market for them could one day be worth trillions of dollars.
The crash, which occurred Sunday night, was a major setback for Uber, which has been trying to improve its image since Mr. Khosrowshahi replaced Travis Kalanick as the company’s chief executive in a messy transfer of power last August. In February, Uber also settled a longstanding legal fight with Waymo.
On Monday, Uber halted autonomous car tests in Arizona, Pittsburgh, San Francisco and Toronto. It is not clear when the company will revive them.
The Tempe Police Department said it was investigating the crash, and has not determined whether the car was at fault. A Volvo XC90 sport utility vehicle equipped with Uber’s sensing technology struck Elaine Herzberg, 49, while it was going 40 miles an hour in a 45-mile-an-hour zone. According to the police, the car, with one safety driver and operating in autonomous mode, did not slow down before impact.
A video shot from the vehicle’s dashboard camera showed the safety driver looking down, away from the road. It also appeared that the driver’s hands were not hovering above the steering wheel, which is what drivers are instructed to do so they can quickly retake control of the car. Ms. Herzberg, pushing a bicycle across the street, appeared in the camera right before she was hit.
“As we develop self-driving technology, safety is our primary concern every step of the way,” said Matt Kallman, an Uber spokesman. “We’re heartbroken by what happened this week, and our cars remain grounded. We continue to assist investigators in any way we can.”
Uber has been testing its self-driving cars in a regulatory vacuum in Arizona. There are few federal rules governing the testing of autonomous cars. Unlike California, where Uber had been testing since spring of 2017, Arizona state officials had taken a hands-off approach to autonomous vehicles and did not require companies to disclose how their cars were performing.
Waymo and Cruise, a self-driving car company owned by GM, reported their “intervention” numbers to California regulators. Uber’s goals in Arizona were mentioned in internal documents — Arizona does not have reporting requirements — and it has not been testing self-driving cars in California long enough to be required to report them.
Uber’s first road tests in its self-driving car effort, code-named Project Roadrunner, were actually in Pittsburgh in September 2016. The Phoenix area was added a year ago, and quickly became the company’s main testing ground, with 400 employees and more than 150 autonomous cars driving local roads because of "favorable regulatory environment, favorable weather conditions,” according to a company document.
When Mr. Khosrowshahi took over as Uber’s chief executive, he had considered shutting down the self-driving car operations, according to two other people familiar with Mr. Khosrowshahi’s thinking.
But he became convinced that it was important to Uber’s long-term prospects. His visit to Phoenix was seen by the Arizona team as a critical opportunity to demonstrate their progress, according to the people familiar with the company’s operations in the Phoenix area. They wanted to take him on a ride without human interventions to demonstrate that the cars could handle so-called edge cases, tricky road situations that are hard to predict.
“With autonomy, the edge cases kill you, so you’ve got to build out for all the edge cases,” Mr. Khosrowshahi said at a conference in November. “Which makes it a very, very difficult problem.”
By September 2017, Uber’s autonomous cars had driven one million miles in a year nationwide. Uber tallied its second million in 100 days and added its next million at an even faster clip, according to company documents.
Early on in Phoenix, there were two groups of test drivers. A smaller group “stressed” the cars by putting them in challenging situations where, without human intervention, they would have crashed.
A larger group of drivers was focused on picking up customers in the autonomous vehicles. Those drivers were expected to pay more attention to little details, often taking control to prevent a “bad experience” like hard braking, according to a company document.
Around October, Uber merged the two groups to get to a point where it could offer a truly driverless car service to customers “as quickly as possible.” The customer pickup service was mostly dropped so drivers could focus on accumulating miles and gathering data to help the system become more reliable.
Around the same time, Uber moved from two employees in every car to one. The paired employees had been splitting duties — one ready to take over if the autonomous system failed, and another to keep an eye on what the computers were detecting. The second person was responsible for keeping track of system performance as well as labeling data on a laptop computer. Mr. Kallman, the Uber spokesman, said the second person was in the car for purely data related tasks, not safety.
Waymo had also moved from two operators at all times to one in some situations in late 2015, said Johnny Luu, a Waymo spokesman. Waymo still uses two test drivers when it is adding new systems or moving to a new location.
But Uber’s autonomous cars are not operating nearly as well as those of its competitors. Cruise reported to California regulators that it went more than 1,200 miles per intervention. After its strong California results, Waymo is now testing cars in Chandler, Ariz., a Phoenix suburb, with no safety drivers.
Mr. Kallman said miles per intervention was not a measure of safety but a rate of system improvement that could differ depending on where and how the cars were driven.
When Uber moved to a single operator, some employees expressed safety concerns to managers, according to the two people familiar with Uber’s operations. They were worried that going solo would make it harder to remain alert during hours of monotonous driving. Mr. Kallman said it delayed the start of its single-driver initiative to allow for more training and to make sure drivers felt comfortable for the new role.
Uber also developed an app, mounted on an iPad in the car’s middle console, for drivers to alert engineers to problems. Drivers could use the app anytime without shifting the car out of autonomous mode. Often, drivers would annotate data at a traffic light or a stop, but many did so while the car was moving, said the two people familiar with Uber’s operations. Mr. Kallman said it designed the app to meet government safety guidelines for in-car software to minimize distractions.
Waymo had a different solution when it moved to a single safety driver. It added a button on the steering wheel for drivers to create an audio explanation when they took the car out of autonomous mode.
Not all drivers followed Uber’s training. One was fired after falling asleep at the wheel and being spotted by a colleague. Another was spotted air drumming as the autonomous car passed through an intersection, according to the two people familiar with Uber’s operations.
Uber was planning to seek regulatory approval by December to start a self-driving car service in Arizona, according to company documents. Uber said the vehicles would have to be safer than human drivers before they would commercialize it. They would not operate around the clock and would stop for bad weather or traffic. And the service did not need to prove “longer-term financial viability.”
Already, one milestone will be missed. Mr. Khosrowshahi will not travel to Phoenix next month, because of scheduling problems that came up before the crash, Mr. Kallman said. But it is unclear how the crash will ultimately affect Uber’s plans for autonomous vehicles.
“The collection of bad news around Uber creates a reputation in people’s minds,” said Michael Ramsey, an automotive analyst at Gartner. “Every other company would get a black eye, too, but they might be forgiven. For Uber, it’s going to be hard to shake.”
Follow Daisuke Wakabayashi on Twitter: @daiwaka
Mike Isaac and Cade Metz contributed reporting.
A version of this article appears in print on March 24, 2018, on Page B1 of the New York edition with the headline: Flaws in Self-Driving Cars Before Fatal Crash. Order Reprints| Today's Paper|Subscribe