Aerial view of Ann Arbor, MI on September 16, 2015. Image credit: Marcin Szczepanski, Michigan Engineering

ANN ARBOR—Combining human and artificial intelligence in autonomous vehicles could push driverless cars more quickly toward wide-scale adoption, University of Michigan researchers say.

That’s the goal of a new project that relies on a technique called instantaneous crowdsourcing to provide a cost-effective, real-time remote backup for onboard autonomous systems without the need for a human to be physically in the driver’s seat. The research is taking place at the U-M Transportation Research Institute (UMTRI).

Today’s autonomous vehicles can drive relatively well in typical settings, but they fail in exceptional situations—and it’s those situations that are the most dangerous.

Walter Lasecki

The need for human safety drivers in vehicles like Waymo’s recently introduced autonomous taxis undermines their cost advantage compared to traditional ride sharing services, the researchers say. It also keeps the era of cars as autonomous rolling living rooms tantalizingly out of reach. And most researchers agree that machines won’t be able to completely take over driving duties for years or even decades.

Walter Lasecki

“Today’s autonomous vehicles can drive relatively well in typical settings, but they fail in exceptional situations—and it’s those situations that are the most dangerous,” said Walter Lasecki, an assistant professor of computer science and engineering and a leader of the project. “Designing autonomous systems that can handle those exceptional situations could take decades, and in the meantime, we’re going to need something to fill the gap.”

Instantaneous crowdsourcing differs from earlier efforts at remote human backup in that it can provide human responses in just a few milliseconds—potentially fast enough to help dodge a swerving vehicle or maneuver around a piece of roadway debris. It utilizes connected vehicle technology and a remotely located control center.

Here’s how it would work—all within the space of five seconds or less:

• Software in the vehicle would analyze real-time vehicle data and electronically guesses 10-30 seconds into the future to estimate the likelihood of a “disengagement”—a situation where the car’s automated systems could need human help.
• If the likelihood exceeds a pre-set threshold, the system contacts a remotely located control center and sends data from the car.
• The control center’s system analyzes the car’s data, generates several possible scenarios and shows them to several human supervisors, who are situated in driving simulators.
• The humans respond to the simulations and their responses are sent back to the vehicle.
• The vehicle now has a library of human-generated responses that it can choose from instantaneously, based on information from on-board sensors.

Robert Hampshire

Such a system might sound expensive and cumbersome, but Robert Hampshire, a research professor at UMTRI and U-M’s Ford School of Public Policy, says it would be far less expensive than having a human driver in every vehicle. This could make it particularly valuable to ride sharing and fleet operators. And the huge volume of miles driven combined with the fact that autonomous vehicles only rarely need human assistance could drive economies of scale that would bring down the cost per vehicle.

There were 3.2 trillion miles driven in the US last year, and the best autonomous vehicles averaged one disengagement every 5,000 miles.

Robert Hampshire

“There were 3.2 trillion miles driven in the US last year, and the best autonomous vehicles averaged one disengagement every 5,000 miles,” Hampshire said. “We estimate that if all those miles were automated, you’d need around 50,000 to 100,000 employees, distributed city by city. A network like that could operate as a subscription service, or it could be a government entity, similar to today’s air traffic control system.”

The algorithm-based screening at the beginning of the process makes it more useful than earlier attempts at remote human assistance, which required the vehicle to stop, contact a remote call center and get instructions before proceeding.

Another key to making the system work on the ground will be designing it in a way that’s workable for the large number of employees, says Hampshire.

“Like the job of air traffic controllers, this work could be stressful and cognitively complex. So we’ll be looking at ways to make it less intense and mentally fatiguing.”

The developers are currently working on the software platform. They hope to have humans testing the system by the end of the project’s first year, with the system capturing data from actual vehicles by the end of the second year.

The basic premise behind instantaneous crowdsourcing was validated in a paper titled “Bolt: Instantaneous Crowdsourcing via Just-in-Time Training,” which was presented at the ACM CHI 2018 conference. The USDOT project aims to adapt it for use in autonomous vehicles. In addition to USDOT, this project is funded by the Center for Connected and Automated Transportation at UMTRI and U-M’s Mcubed.