Self-Driving Cars
A technology where the hardest problems aren't highway driving, but figuring out what a confused pedestrian is about to do next.
By Greg Goodwin · Published July 3, 2026 · Updated October 24, 2026
Cheat Sheet
- Self-driving cars use a combination of sensors — cameras, radar, and often lidar — feeding a real-time software system that perceives the environment and decides how to act.
- The industry uses a 6-level scale (0-5) defined by SAE International to describe autonomy, from Level 0 (no automation) to Level 5 (full autonomy in all conditions, no human needed).
- Most "self-driving" features on the road today, including Tesla's Autopilot and GM's Super Cruise, are actually Level 2 — they still require an attentive human driver ready to take over.
- Waymo, Google's self-driving spinoff, operates genuinely driverless (no safety driver) robotaxi services in several US cities, generally considered Level 4 autonomy within a defined operating area.
- Edge cases — unusual scenarios like unpredictable pedestrian behavior, construction zones, or extreme weather — remain the hardest unsolved problem for full autonomy, more than routine highway driving.
- Lidar (a sensor using laser pulses to build a 3D map of surroundings) has been a major point of industry disagreement — Tesla notably avoids it in favor of cameras alone, while most competitors consider it essential.
The 60-Second Version
Self-driving cars use a combination of sensors — cameras, radar, and often lidar — feeding a real-time software system that perceives the surrounding environment and decides how to act. The industry describes autonomy using a 6-level scale (0-5) defined by SAE International, ranging from Level 0 (no automation at all) to Level 5 (full autonomy in every condition, with no human needed at all). Most "self-driving" features actually on the road today, including Tesla's Autopilot and GM's Super Cruise, are really Level 2 systems that still require an attentive human driver ready to take back control at any moment. Waymo, Google's self-driving spinoff, operates genuinely driverless robotaxi services (no safety driver at all) in several US cities, generally considered Level 4 autonomy within a specifically defined operating area. Edge cases — unusual scenarios like unpredictable pedestrian behavior, active construction zones, or extreme weather — remain the hardest unsolved problem for full autonomy, harder by far than routine highway driving. Lidar, a sensor using laser pulses to build a precise 3D map of the surroundings, has been a genuine point of industry disagreement: Tesla notably avoids it in favor of cameras alone, while most competitors consider it essential.
The Long Version
The Sensor Stack
Autonomous vehicles typically combine several types of sensors to build a picture of their surroundings: cameras capture visual detail useful for recognizing lane markings, traffic signs, and pedestrians; radar detects the distance and speed of other objects, working reliably even in poor weather or low light; and lidar, where used, bounces laser pulses off surrounding objects to build a precise 3D map of the environment. Combining these different data sources, a process called sensor fusion, is meant to compensate for each individual sensor type's weaknesses with the strengths of the others.
The SAE 0-5 Autonomy Scale
SAE International's widely adopted scale defines six levels of driving automation: Level 0 has no automation at all; Levels 1 and 2 offer driver assistance features (like adaptive cruise control or lane centering) but still require full human attention and control; Level 3 allows the car to handle full driving in specific conditions while still expecting a human to take over when prompted; Level 4 allows genuinely driverless operation, but only within a clearly defined area or condition set; and Level 5 describes full autonomy anywhere, in any condition, with no human involvement ever required, a level no commercially deployed system has yet achieved.
Who's Actually Doing What Today
Despite marketing names that suggest otherwise, most consumer-available systems, including Tesla's Autopilot and Full Self-Driving, GM's Super Cruise, and similar systems from other automakers, are Level 2 systems requiring constant driver attention and readiness to intervene. Waymo stands out as one of the few operators running genuinely driverless Level 4 services, offering robotaxi rides with no safety driver in several US cities, though strictly within carefully mapped and geographically limited operating areas rather than anywhere a human could drive.
Why Edge Cases Are the Real Bottleneck
Routine highway driving in clear weather is, relatively speaking, one of the easier problems for autonomous systems to solve, since conditions are predictable and well-mapped. The much harder, still largely unsolved challenge is handling edge cases: a pedestrian who starts to cross then hesitates, an unmarked construction detour, an unusual hand signal from a traffic officer, or heavy snow obscuring lane markings. These low-frequency but high-stakes scenarios are exactly why full Level 5 autonomy, capable of handling literally any situation a human driver might encounter, remains an unsolved engineering challenge despite over a decade of rapid progress.
Why People Care
Self-driving technology sits right at the boundary between genuinely impressive and still meaningfully limited, and understanding the real gap between marketing terms like "Autopilot" and the industry's own formal autonomy levels is essential to evaluating both the technology's current safety and its realistic near-term future.
Glossary
- SAE Levels of Automation
- A 0-5 scale defining how much of the driving task is handled by the vehicle versus a human.
- Lidar
- A sensor that uses laser pulses to build a precise 3D map of a vehicle's surroundings.
- Edge case
- An unusual or rare driving scenario that's especially difficult for autonomous systems to handle safely.
- Robotaxi
- A self-driving vehicle operated as an on-demand ride service without a human driver.
- Sensor fusion
- Combining data from multiple sensor types, such as cameras, radar, and lidar, into a single, more reliable picture of the environment.
Go Deeper
More to Explore
- Generative AI
A very well-read autocomplete that learned to write essays, paint pictures, and occasionally make things up with a straight face.
- Computer Vision
Teaching a computer to see turned out to be much harder than teaching it to play chess, and much more useful.
- AI Ethics and Bias
An AI hiring tool taught itself to penalize resumes hinting at being a woman, and no programmer asked it to. It just learned from biased history.
- Neural Networks
Neural networks loosely mimic the brain and can give accurate answers that even researchers struggle to fully explain.
- Machine Learning Basics
Software that gets better at a task not because a programmer rewrote its rules, but because it saw enough examples to figure out the pattern itself.