Tesla full self-driving
On average, 1.35 million people die in road crashes each year around the world
. An additional 20-50 million people suffer long-term injuries from vehicle crashes globally. In the United States alone, 38,000 people die in crashes each year.We weigh these costs against all the benefits driving adds to civilization like the freedoms that come with owning a personal vehicle, the ability to travel further away from our homes, and the exponential increases in productivity for our economies. Progressive technologists and engineers are now seeking to mitigate and eliminate some of the 2nd order effects of driving. Driving is dangerous, and human drivers are imperfect. Drunk drivers, distracted drivers, pissed-off drivers, IDGAF drivers, the list goes on. Vehicle safety is a multidimensional issue, and cars throughout the decades have been fitted with safety features like airbags and seatbelts to make driving less dangerous. Still, tens of thousands of Americans die each year in car crashes, and human error is unfortunately often to blame.
In 1908, the Ford Model T was released for sale to the public. Since the creation of the first gas-powered automobile in 1885 by Karl Benz
, the Model T was by far the most affordable car ever made. Ford Motor Company began mass manufacturing the Model T, with their new breakthroughs implementing vertical integration and assembly lines into their factories and manufacturing processes. This leap, making vehicles via mass production as opposed to handmaking each individual vehicle, made cars affordable for middle-class individuals and families. Before this, car ownership was reserved for wealthy people. In 1927, Henry Ford watched the 15 millionth Model T come off the assembly line at a Ford factory in Michign. As a greater percentage of society, especially in the US, embraced vehicle ownership - the world began being built around individual vehicle ownership and individual transportation via personally owned vehicles. Ford’s Model T and their new breakthroughs in assembly lines, mass manufacturing, and economies of scale were a significant opportunity for the world, but again, especially the US.Over the last century, our society has outweighed all the things driving has given us against these driving accidents suffered - but is there a better way? From search engines to brain-machine interfaces, AI can have unlimited functions, though its application to self-driving and fully autonomous vehicles could eradicate one of the largest causes of death in the world - driving accidents.
Andrej Karpathy, the director of AI at Tesla, and their team of engineers led by Elon Musk are leading the world’s most significant self-driving and fully autonomous vehicle project. Other US companies like Waymo (Google), Cruise (GM), and Comma ai are also working on autonomous vehicles, but are taking a slightly different approach, and traditional OEMs like Ford, Volkswagen, BMW, and Honda are introducing driver-assist features into their newest cars. But not everyone believes in a future where vehicles will be fully autonomous and some critics, like financial analyst Gordon Johnson, are particularly livid about the effort Tesla is making to realize a fully autonomous fleet.
Gordon and other financial analysts like TESLAQ have been insistent that full self-driving, and particularly Tesla’s full self-driving suite, is vaporware. Gordon consistently cites, as in the tweet above, a website tesladeaths.com where Anti-Teslaians passionately track each crash, injury, and death in Teslas.
Here’s what they’d like you to know about Tesla (autopilot released Oct 14, 2015):
Note: In the US, 38,680 people died in motor vehicle traffic crashes in 2020
Tesla deaths globally since 2013: 201
2013: 3
2014: 6
2015: 5
2016: 16
2017: 11
2018: 21
2019: 50
2020: 44
2021: 45
In Q2 2020, Tesla officially had over 1 million active vehicles in their fleet
, and the cars in that fleet had driven over 3 billion miles on autopilot. Today, the Tesla fleet is approaching double the autopilot miles from Q2 2020.
I agree with Gordon that we want to make our roads safe, but I think he’s pointing his finger at the wrong thing. In fact, I think he’s pointing his finger at the engineers trying to find a solution to the millions of global auto deaths suffered each year.
Tesla FSD (full self-driving)
Tesla’s approach to fully autonomous vehicles is fundamentally different than the likes of Cruise and Waymo. Waymo and Cruise deploy vehicles with Lidar sensors and pay engineers and drivers to sit in them while they go around cities mapping the world around them. Over time, these maps turn into simulations and the simulations are used to guide their fleet of autonomous vehicles. Lidar has many applications. For example, SpaceX uses Lidar to autonomously land boosters from launched rockets, and by mapping a simulation of the pad, landing, and the world around it, its boosters can return to earth and land back on their pads.
Lidar is an important technology but is limited in scope. For rocket boosters returning to earth for landing, pads are controlled sites and are not consistently changing. For cars on roads, variables are infinite and changing rapidly. Roadways evolve, signage is different in unconventional places, roads are different in various countries, drivers all act differently, and the list goes on. In this sense, Gordon Johnson and TESLAQ are correct, solving for fully autonomous vehicles is really fucking hard.
So Tesla has taken a slightly different approach than relying on Lidar to make their fleet fully autonomous. While Lidar can still be useful (more on that later), to actually make vehicles fully autonomous Tesla’s engineers argue you must solve for ‘vision’.
Humans drive relying most on our eyes, with minor inputs from sensory items like sound to drive. We rely on our biological neural networks, our brain, to make all the decisions necessary. Humans use their past experiences and ‘data’ to train our brains for upcoming decisions.
Now we are building artificial neural networks, broadly referred to as AI, to simulate the biological brain. Artificial neural nets, with the objective of simulating the biological brain’s decision-making functions, are trained on data to build a non-human brain. Google’s search engine, as an example, uses artificial neural networks (basically a non-human brain) to remember all the data we’ve given it over the years to give us better search results. No matter what problem statement you are building an artificial neural network to solve for, an important ingredient is the data set you’re using to train your fake brain.
This is what the annual Tesla AI day that happened last week was about. AI projects get more exciting over time as the progress you’re making compounds, so each year the announcements get more and more compelling. This year, the Tesla AI team announced their roadmap for full autonomy.
Hardware
In many ways, we can compare Tesla vehicles to traditional computers. In order to support a full self-driving suite, the vehicle is equipped with chips measured in compute power, just as smartphones and computers are. To train a neural net for FSD, massive amounts of real-world data and then a computer is powerful enough to use that data are required. Tesla engineers cite a list of reasons why they were forced to build a chip in the house (not simple), but the core issue was that other AI chips on the market were not specialized for the problem statement of full self-driving.
(Tesla’s first chips: hardware 1.0 was built on MobilEye EyeQ3, 2.0 on NVIDIA DRIVE PX 2 AI, and 2.5 on NVIDIA DRIVE PX 2 with secondary node enabled
)At autonomy day in 2019, Tesla AI announced their first in-house chip hardware 3.
This year they outdid themselves again, announcing the D1 chip to support Project Dojo. Ganesh Venkataramanan, leader of the Tesla Dojo project
, stated the goal of Dojo to be:1) Achieve best AI training performance
2) Enable larger & more complex neural net models
3) Power efficient and cost-effective compute
Dojo is the name of the entire hardware stack. From the smallest functional unit of compute, hardware is placed together in a modular form. Simply put, 25 D1 chips make 1 training tile, 12 training tiles make 1 cabinet or rack, and 10 racks make one Exa pod
(like a sandwich kind of).Both at a vehicle and training level, Dojo efforts from Tesla engineers will remove any bottlenecks in the way of scaling the exponentially growing neural net. At the vehicle level, Tesla’s on the road today are equipped with the hardware necessary for full autonomy. At the training level, as the amount of vehicles in the fleet grows, the number of miles driven on autopilot will exponentially increase and the supercomputer Dojo will make it possible to train against the rapidly growing data set.
Tesla Brain
First, there are two important neural network types, Convolutional Neural Networks
and Recurrent Neural Networks. Basically, Tesla FSD uses recurrent neural networks to make ‘decisions’ from multiple feature inputs at once, a step up in complexity from convolutional neural networks which is possible with massive compute power. They use these recurrent neural networks to train for vector space (essentially 3D), time, and planning all simultaneously. Lex Friedman, a beloved podcaster, and Artificial Intelligence expert at MIT gives a good explanation.Next in the feedback loop is 3D labeling in vector space (3D). Tesla uses thousands of in-house manual labelers alongside their auto-labeling processes. Then comes the Lidar.
As Gordon giddily points out, Tesla has some cars mounted with Lidar to add a simulation to their FSD stack. As explained at AI day last week, these simulations are used to train for rare edge cases and for scenes where real-world labeling is really difficult.
Tesla vehicles on autopilot are still running on a neural net trained from the data of every single vehicle mile from other Teslas that are driving on autopilot over time but are also run through the improving simulation now as well. Vision, planning and control, manual labeling, auto labeling, simulation, hardware integration: Dojo.
In total Tesla FSD looks like this:
Tesla AI
Fully autonomous vehicles are difficult to achieve but important to pursue. Not to be distracted, solving for autonomous vehicles is an effort to reduce the number of people who suffer from driving accidents. There will be infinite applications for Tesla AI, but the most pressing is to save lives on public roads by making vehicles truly autonomous.
To make autonomous vehicles real, many components have to come together. General real-world robotics and perception problems, the human-robot interaction problem to make the experience more smooth and pleasurable, massive in-house data swaths, massive compute hardware to support the neural networks, and on.
Tesla is not the only company working on the future of driving, but it is the only fleet in the world built with the entire technology stack necessary to achieve full (level 5) autonomy. As their fleet of computers (aka Tesla vehicles) grows, the data swath to train the fleet against will continue to grow exponentially, and data supporting the safety of cars with Tesla autopilot will be available to monitor. Everyone is watching, including US regulators:
‘Good regulation’ is important for our newest industries. Competent oversight should be done on those introducing extremely powerful technologies, and these technologies should be monitored with rigor. In the case of autonomous vehicles, especially in the United States, our regulators should lean on engineers to help understand the landscape. Tesla has demonstrated their ability to make groundbreaking technology that is not only impressive but will save countless American lives in the future. These efforts should be rewarded and celebrated.
Ford’s Blue Cruise driver assist, and GM’s Cruise made it to the White House for their recent EV event, but Tesla didn’t make the cut. American’s have a lot to gain from celebrating our best engineers and their work. They are the American’s in the arena. They are the American’s trying to reduce driving accidents, electrifying our car market, and bringing the world’s most sophisticated manufacturing to factories in the United States. They are American’s saving lives. The White House, Gordon Johnson, and TESLAQ are talking about the engineers in the arena.
-LUCAS
https://www.asirt.org/safe-travel/road-safety-facts/
https://www.asirt.org/safe-travel/road-safety-facts/
https://en.wikipedia.org/wiki/Benz_Patent-Motorwagen
https://en.wikipedia.org/wiki/Ford_Model_T
https://www.nhtsa.gov/press-releases/2020-fatality-data-show-increased-traffic-fatalities-during-pandemic
https://lexfridman.com/tesla-autopilot-miles-and-vehicles/#Tesla_Vehicle_Deliveries_and_Autopilot_Hardware
https://en.wikipedia.org/wiki/Tesla_Autopilot#cite_note-163
https://perspectives.mvdirona.com/2021/08/tesla-project-dojo-overview/
https://perspectives.mvdirona.com/2021/08/tesla-project-dojo-overview/
https://www.upgrad.com/blog/basic-cnn-architecture/
https://www.upgrad.com/blog/cnn-vs-rnn/
Great read