The Illusion of Autonomy
In cities like San Francisco, Phoenix, and Las Vegas, robot taxis are becoming a common sight. However, what many people don’t realize is that these driverless vehicles aren’t always as autonomous as they appear. Sometimes, even cars without steering wheels rely on human intervention to navigate.
This is a command center run by Zoox, an Amazon-owned self-driving car company, highlighting the human element behind the cutting-edge technology. With self-driving cars sometimes struggling to navigate on their own, technicians, often located hundreds of miles away, provide crucial assistance, as is the case for other robot taxi companies.
‘Wizard of Oz’ Effect
Within companies like Zoox, human assistance is standard practice. However, the public often remains unaware of this remote support. For years, companies downplayed the extent of this intervention, the lack of transparency helping attract investment.
“There is a ‘Wizard of Oz’ flavor to this,” said Gary Marcus, an entrepreneur and professor emeritus of psychology and neural science at New York University, who specializes in AI and autonomous machines. For instance, if one of Zoox’s robot taxis encounters a construction zone it hasn’t previously registered, a technician in the command center immediately gets an alert. Using their computer mouse, the technician can then draw a new route on their screen, guiding the car around the obstacle.
“We are not in full control of the vehicle,” said Marc Jennings, a 35-year-old Zoox remote technician. “We are providing guidance.”
The Role of Remote Technicians
One evening in San Francisco, a Zoox car encountered an emergency scene and needed assistance. Video feeds from multiple cameras installed on the car provide the technician with real-time views of the car’s surroundings. A graphical, bird’s-eye view from a dashboard on the computer screen also helps technicians monitor conditions on the road. In this case, the car got stalled near a fire truck and needed assistance. To bypass the obstruction, the technician used a computer mouse to plot a new path for the car. As the car began following the new route, the technician continued to manage the car’s course. The technician then noticed an ambulance behind the fire truck, an element that was obscured from the car’s initial camera views. The technician adjusted the path, and the car successfully navigated the new course, eventually returning to autonomous driving.
Industry Trends and Challenges
As companies such as Waymo (owned by Google’s parent company, Alphabet) and Cruise (owned by General Motors) have begun deploying driverless cars, their operations have faced increased scrutiny. After high-profile accidents, they have started acknowledging the need for human assistance.
While Zoox and others are disclosing more about human interventions in driverless cars, they haven’t revealed the size of their remote-assistance teams or associated costs. Zoox’s command center accommodates around three dozen people who oversee a relatively small number of driverless cars—two in Foster City and several more in Las Vegas—as well as a test fleet of roughly 200 cars that each still have a driver behind the wheel. When regulators ordered Cruise to shut down its fleet of 400 robot taxis in San Francisco, approximately 1.5 workers per vehicle, including remote assistance staff, supported those cars. These workers intervened to assist the vehicles every two and a half to five miles.
Remote assistance expenses pose a challenge to the widespread adoption of robot taxis, particularly when compared to traditional ride-hailing services such as Uber and Lyft. Companies like Zoox still spend money on human assistance, whereas it might be cheaper to have a human driver in the car.
The Zoox Vehicle
While Waymo and Cruise have adapted existing vehicles for self-driving, Zoox is testing a different kind of vehicle in Foster City and Las Vegas. These robot taxis are designed without a place for a driver, lacking a driver’s seat, steering wheel, dashboard, or gas pedal. The vehicle seats four passengers who sit on two bench seats facing each other. A touch screen allows passengers access to functions like opening and closing the door and controlling the air-conditioning or music. After testing the vehicles with employees and their families, Zoox plans to make the service available to the public this year. But, these robot taxis depend on human assistance too.
In Foster City, Zoox operates a “fusion center” that enables employees to monitor robot taxis operating both locally and in Las Vegas. Workers can view live feeds from vehicle cameras as well as data from the car’s sensors, which provides a detailed overhead view. These workers can also communicate with riders and assist the car if it encounters situations it can’t handle on its own.
“These are situations that don’t necessarily fit the mold,” said Jayne Aclan, who oversees a team of Zoox technicians that provide cars with remote assistance. Even though self-driving cars can handle routine maneuvers like turns and lane changes, unusual or unexpected events require human support.
The Human Element
Even with remote assistance, driverless cars on occasion make mistakes. In Las Vegas, a Zoox robot taxi initially failed to yield for fire trucks with their sirens on, which caused a delay in their turning, showing the difficulties these vehicles have in so-called edge cases. Zoox noted that its car struggled to recognize the fire trucks because they were yellow, not red. “We continue to test and refine our driving software,” said Whitney Jencks, a company spokeswoman.
“We think that computers should be able to replicate humans and replace humans in all ways,” Dr. Thomas W. Malone, the M.I.T. professor, said. “It is possible that might happen. But it hasn’t yet.”
