Toyota CTO: Autonomous Vehicles Will Transform Our Streets & Built Environment

The Toyota Institute was founded about nine months ago, and we now have more than 150 employees and have opened three offices. Our goal is to partner with academic institutions, our industrial partners in the Toyota family of companies, and government to bring about new technologies related to advanced safety and development. Our key focus areas are automotive safety and autonomy, but we’re also exploring new forms of mobility.

Our philosophy is to leverage state-of-the-art cloud computing and software tools to explore new forms of energy, battery technology, and materials that could help transform the next generation of mobility solutions.

A lot of Silicon Valley companies talk about “moonshot research”—how to make great leaps in technology. How to do that is still an unsolved problem, but we know some of the ingredients. There’s historical precedent.

Take the car, for example. In 1885, we had the first gas-powered engines. They tended to explode and were not very safe; they were also very expensive. Over the next 20-30 years, there was incredible advancement in transmission design, and manufacturing technology brought costs down to the point where it became possible to personally own vehicles. That changed the world.

In fact, our cities are now designed for cars. Many people have said that if aliens observed our planet from afar, they would say, “That place is populated by metal beings with four wheels, and every once in a while, some carbon-based parasites enter and exit them.”

Another historical precedent is the computer. In 1945, we had very expensive, unreliable, vacuum-tube-based computers. Over the next 20-30 years, we saw rapid advances in hardware, transistor storage, and displays. In the early 1980s, personal computers became widely affordable, and that, again, transformed our society.

Then there’s the transformative shift that all of us have lived through: the mobile phone. Back in 1983, if you had $4,000, you could buy a big brick that made occasionally good phone calls. Over the next 20-30 years, incredible advances in wireless transmission speeds, reliability, and battery technology led to the development of smart phones. Smart phones have now overtaken PC desktops worldwide. In fact, the phone you carry around in your pocket is more powerful than the most powerful supercomputer of 20 years ago.

So what’s next?

We think that the confluence of the car, the computer, and connectivity and data in the cloud are going to cause a huge shift. Cloud-connected intelligent vehicles and robots powered by machine-learning and AI—a breakthrough that happened in the last decade—are areas that the Toyota Research Institute is working hard to explore.

The ingredients for disruption come down to strong partnerships among academia, industry, and government—plus a critical mass of people and capital to realize it.

The DARPA Grand Challenge is a good example. For two decades, DARPA had invested in autonomy for vehicles, with mediocre results. So instead of funding individual contracts, they flipped it around. They decided to have a prize bounty, and encourage people to enter a competition to build a car that could navigate itself successfully through a desert track.

The first year of the contest, there was no winner. The entry from Carnegie Mellon—the university that I was teaching at the time—went the furthest, but only about seven miles. Within 12 months, several cars finished the race, including CMU and Stanford’s cars. Two years later, they upped the bounty and had the car handle urban environments. Many cars finished then; Carnegie Mellon was the winner.

That led to the Google self-driving car project, where I helped build the initial prototype. By that time, the technology was proven enough that companies started to invest in it. Now, many more companies are working hard to bring about autonomous cars. There’s incredible activity in this space.

What does all this mean for our cities? Let’s speculate on how intelligent vehicle technology could impact the design of future cities—when we could have driverless cities.

Most transportation would be on-demand—mobility as a service. Ride-share vehicles could be fleet-managed, zero-emission, and eco-friendly, leading to a dramatic reduction in overall traffic, noise, and pollution.

The utilization of each car would also be much higher. Most vehicles wouldn’t be parked all the time, but moving around. We would be able to convert a lot of land currently dedicated into parking lots into useful space.

This is a problem that the Bay Area faces a lot: There is no more space left to develop more housing in San Francisco. But what if you had cars that no longer needed to be parked on the curbside and they could park offsite?

Let’s think about today’s realities. The average car in the US is parked about 95 percent of the time. There’s only a 5-percent utilization of the car. Worldwide, urban drivers spend an average of 20 minutes per trip looking for a parking spot. Yet the United States has more than a billion parking spots—for only 253 million cars. There are four times more parking spaces than there are vehicles. We could better utilize this resource.

In the County of Los Angeles, over 200 square miles of land is dedicated to parking. There are 18.6 million spaces for about 5 million cars—again, roughly four times. That’s 14 percent of all the land area in Los Angeles County.

Now let’s think about what could happen to urban centers if autonomous cars were dropping people off at work downtown, then driving away to an offsite parking garage. Parking structures could be located away from urban centers. There would be no need for giant parking garages on valuable real estate in downtown areas.

A lot of curbside parking could also be removed. That space could become another lane of traffic, or be converted into bike lanes or expanded sidewalks. We would not only reclaim retail, commercial, and residential space in our urban centers, but also increase throughput and encourage more eco-friendly forms of transportation.

We could also more efficiently utilize the space of parking structures themselves. If they were managed robotically, they wouldn’t need stairs and elevators and wide alleyways for pedestrian access. They could be more dense, packed, and efficient.

These robotically managed, centrally located parking structures could double as charging stations. If every time the vehicle parked, it automatically recharged or refueled itself, then we could reclaim the land currently utilized for gas stations. Even car services—such as cleaning, repairs, safety checks, and maintenance—could be performed at the parking lot facilities. If all the parking lots and gas stations could be converted to green spaces, how would that change our cities? I think that would be wonderful.

What else? You can imagine that there are many delivery vehicles on our roads today. Everyone is getting packages delivered to their home. But what if the parking lot were smart enough to become a logistics hub? When your on-demand ridesharing car picked you up from work, loaded in the backseat of your car could be all the packages you ordered.

If the parking system were connected to all the cars, and could use machine-learning to model traffic flows and demand curves, we could do dynamic load-balancing—the same way that Internet routers load-balance traffic and packets of information among servers. We could even proactively dispatch cars from the remote garage in order to meet demand and have very low latency.

Our cities have, until now, been designed around traditional cars. Intelligent vehicles will enable new forms of design.

What if we moved all the vehicle traffic underground? All the above-ground streets currently used for cars could be converted to Fußgängerzone. I love this tradition in Europe of having pedestrian-only zones in the central part of the city. It’s very quiet, there’s better air quality, and of course, it’s much safer.

Disneyland has an incredible network of underground tunnels for Mickey Mouse to get go from one place to another easily. They have an incredible underground logistics network supporting the park. The city of Amsterdam is moving all of its roadways underneath its canals—digging new roadways underneath the city’s existing waterways. The construction doesn’t disrupt the existing roads, and once the cars are moved, those roads will be converted to pedestrian and bicycle zones. We could absolutely do the same thing.

Realizing this incredible potential will require the cooperation of governments, forward-thinking industry leaders, and people in academia. If our society comes together and makes this a reality, we could dramatically improve the quality of life of millions of people around the world—as urban areas are becoming the places where people live, and rural areas are losing population.

Toyota is doing test programs in Grenoble, France, and Toyota City, Japan on a concept vehicle—a zero-emissions electric vehicle with a compact freewheel design. It has active balancing and stability control, so it’s very fun to drive. It’s strong and lightweight, and it has batteries that average 50 kilometers on a single charge.

There are versions of the car that can support two passengers. But the sad fact is that in California, at any given time, 93 percent of the cars on the road have a single passenger in them.

One of the coolest things about this vehicle is that you can fit four of them in a traditional parking space. Imagine the incredible density this could achieve. There’s an opportunity to dramatically increase the capacity of existing roadway infrastructure. If all vehicles were this size, we could have twice the number of lanes.

Toyota is also investing in a “true zero” emissions future for our transportation. The Toyota Mirai hydrogen fuel-cell vehicle has 312 EPA-rated miles per tank, and it only takes five minutes to refuel. Wheel to well, it is one of the most environmentally friendly solutions you can build.

It essentially uses the Hybrid Synergy Drive—which became popular with our Prius line and hybrid models—but instead of a gas engine, it uses a hydrogen fuel-cell stack. The only emission is water vapor. You can push the H20 button and water will come out of the tailpipe. If you were stuck in the desert, you could get some water out of your car.

The incredible thing about it, though, is that the car itself could power an average home for about a week. If for some reason you had no power, you could hook up your home to your car. That’s truly off the grid. You can buy this today (and there are incredible tax incentives because it’s so wonderful for the environment.)

Of course, lots of infrastructure needs to be built to support this type of vehicle. But if our future mobility solutions are moving toward centrally managed fleets, this actually is very synergistic: The smart parking garage could simply have a hydrogen-fuel station.

There is an additional promise to connected cars in terms of their software. My former employer Google has open-sourced Tensorflow, a state-of-the-art models for machine learning, and many other open-source solutions now exist, like Caffe or Torch or Theano. Anyone can use these state-of-the-art algorithms. What is valuable, then, is the data.

The data collected from connected cars is going to be really powerful. They’ll be able to upload new exemplars to a training set. In the cloud, we’ll be able to train these new models and broadcast updates, so that the entire fleet can download a much more reliable and efficient perception system. Recognizing cars and pedestrians could be crowdsourced. That data will enable a much more reliable next generation of intelligent vehicles.

This could be used, for example, on something that a lot of companies are investing in right now: maps of our road infrastructure. Some companies are spending billions of dollars to collect data about all the roads in the world.

But as soon as you dispatch a car equipped with sensors to collect data, that data immediately becomes out of date and stale. Maintaining the freshness of the data and the map is very challenging, given that every day, lane markings get repainted, there’s construction, new roads get built, etc.

A connected car could notice street signs, speed markings, lane markings, and everything around it as it’s driving. When it notices a difference between its prior map and what it just collected, it could send that differential information up to the cloud. The cloud can reassemble and stitch together dynamically a map of the entire road surface. This is a truly scalable way of maintaining a fresh map of the world—using existing vehicles that are already out there.

Toyota sells 10 million vehicles a year. Each vehicle lasts about 10 years. That means that at any given time, there are 100 million Toyota vehicles driving on the road. We think that that should be the way that we crowdsource and gather information.

And this information will not only be about drivable surfaces, weather conditions, and traffic patterns. It will enable a new era of managing traffic and managing our cities on a much bigger scale.

I’m excited about the next generation of autonomous vehicles that will enable all of us to rethink how our urban centers are designed and built. I also think that cloud computing, big data, machine learning, and the ubiquitous connectivity that is coming will dramatically advance, not only the capability of our vehicle autonomy, but also the safety and the access that comes along with it.

In order to realize this future, all of us who are passionate about it need to come together. It will require cooperation and partnerships between the government, the industry, and our academic partners. That is key to successful solutions.