By Contributor,John Werner
Copyright forbes
SAN FRANCISCO, CALIFORNIA – Waymo
Getty Images
Transportation is such a big, wide field. Go watch the vehicles creeping through any city street in America, or any suburban enclave, for that matter, and you’re going to see how many warm bodies are piloting vehicles around in seemingly endless rows, all hours of the day and night.
To date, we haven’t seen a lot of self-driving vehicles in most American communities. There’s been the general feeling that this stuff isn’t ready for prime-time, although China is far ahead, with the projection that a full 30% of new vehicles there will be self-driving by 2030.
Again, America has been a long way behind: but that may be all about to change.
We know that foundation models are accelerating what’s possible with AI engines. We know that when these systems reach a certain critical mass, where they can be safety-conscious and accurate, we’re going to see a self-driving takeoff.
When will that be?
The Advent of Waymo’s Autonomous Fleets
In a recent Imagination in Action segment at Stanford, I talked to Vincent Vanhoucke at Waymo about what’s actually going on under the hood there, and how that company is poised to roll out self-driving in over a dozen cities very soon.
MORE FOR YOU
The Timeline
Waymo is about 15 years old, and Vanhoucke noted how it has lived through a lot of the last few technology shifts, estimating that there were four successive generations of engineering involved, until now, AI is close to mature enough to be deployable for self-driving vehicles.
“What’s changed today is that modern AI has enabled us to bring together all of the components of autonomous agents,” he said, citing planning, semantic and geometric understanding, the ability to interpret visual data, and higher-level reasoning. “All of that is (now) in one framework that allows you to merge all of these disparate components,” he said.
Safety, Critical Systems and Visibility
Addressing the “future of mobility,” Vanhoucke said it’s important to be “the most trusted driver,” which, he added, involves transparency, respectability, and being a “good citizen” in a complex sector.
As for the volume of rides happening right now, he estimated 250,000 paid trips a week, with Waymo programs rolling out soon in cities like San Francisco, Phoenix, Atlanta, Miami and more.
The company’s self-driving vehicles, he said, have now racked up a collective 71,000,000 miles, which is a lot of data to crunch numbers on. Here’s some of what researchers have found. Waymo driving involves:
88% fewer serious injuries than trips with human drivers
79% less airbag deployment
78% fewer injury-causing accidents overall
“We’re here to improve the lives of people in a material way,” Vanhoucke said.
Demos of Waymo Machines
Then Vanhoucke showed some actual demonstration footage of the Waymo car.
A Waymo vehicle glides smoothly on a major street. Suddenly, the car swerves. The AI is able to identify pedestrians and other obstacles accurately, for those safer outcomes noted above.
Making it Happen
Explaining the methodology behind safe self-driving, Vanhoucke noted two challenges: building the driver, and validating the driver. The latter, he said, can be more difficult.
Vanhoucke listed three aspects of goal-setting:
Mastery of a complex physical environment
High performance requirements
Need for accuracy in real-world computation
These systems, he suggested, need to handle “black swan” events, or in other words, low-probability scenarios.
The technologies need to be generalized, he added, and they need to leverage world knowledge, as well as scaling training and inference.
Reviewing Waymo’s end-to-end multimodal system named Emma, which is built on Gemini, Vanhoucke explained that it’s important to have a good offboard teacher for an online driver, and a system that features long-term memory, robust reasoning without hallucination, and accurate spatial reasoning.
That’s a tall order, as those are three things that did not exist prior to new research on LLMs.
Vanhoucke also said one thing that many would consider a little chilling: models, he said, like to be creative.
“They like to imagine new things that are not necessarily what exists on the ground,” he said. That’s an important consideration that needs to be dealt with.
In any case, citing a sensor fusion network, a VLM, and an action decoder, Vanhoucke showed off more of the inner workings of AI engines that will keep passengers safe on the road, and save commuters from sitting “in the box” in the driver’s seat, sacrificing non-productive time to travel.
More from Vanhoucke’s talk included the idea of multiple agents behaving like networked parties having a “conversation” and the importance of scaling laws and simulations in autonomous vehicle engineering.
What’s Next?
Questioners asked Vanhoucke, simply, what’s next for Waymo?
His answer? Rapid scaling, now that a lot of the heavy lifting is done.
“You’ll see Waymo in a neighborhood near you very soon,” he said.
As for job displacement around Uber and Lyft, that offer so many Americans independent income, Vanhoucke didn’t really have an answer.
“It’s going to be one of those transitions,” he said, pointing out that the ride-sharing companies themselves disrupted the taxicab industry.
Look for more Waymo cars soon on the American road, and stay tuned for more out of our Imagination in Action Stanford event.
Editorial StandardsReprints & Permissions
