Computer Vision

Teaching a computer to see turned out to be much harder than teaching it to play chess, and much more useful.

By Greg Goodwin · Published October 17, 2026 · Updated October 24, 2026

Cheat Sheet

Definition: the AI field of getting programs to identify what is in digital images from cameras; also called machine vision.
Early milestone: Larry Roberts's 1963 PhD thesis, which some historians prefer to call the field's true starting point.
Summer Vision Project: MIT AI memo AIM-100 by Seymour Papert, dated July 7, 1966.
ImageNet: debuted as a poster at CVPR 2009; its 2010 contest used 1,000 classes and about 1.2 million training images.
AlexNet: won the 2012 ImageNet contest with 15.3% top-5 error, against 26.2% for the runner-up.
Core tasks: classification (what is it), detection (where is it), segmentation (which pixels belong to it).

The 60-Second Version

Computer vision is the branch of artificial intelligence that lets machines make sense of images and video. To a computer, a photo is just a grid of numbers, one set per pixel, so the challenge is turning those numbers into something meaningful like a face, a stop sign or a tumor on a scan. For decades, researchers tried to write rules by hand, with limited success. The breakthrough came when programs learned patterns from huge sets of labeled pictures instead. Around 2012, deep neural networks trained on a dataset called ImageNet beat older methods by a wide margin, and the field has been dominated by deep learning ever since. Britannica notes that technologies such as facial recognition and augmented reality rely on it. Today it helps unlock phones, read license plates, sort photos, guide robots and assist doctors, though it can still be fooled by unusual lighting, angles or situations it never saw in training.

The Long Version

Seeing Is Hard

People recognize a friend from across a street in a split second, so it feels easy. For a computer, a photograph is millions of numbers describing the color and brightness of each pixel, and the same object can look entirely different under changed lighting, angle, distance or background. Vision systems therefore tackle several tasks. Classification labels a whole image, detection finds and boxes the objects in it, and segmentation decides which pixels belong to which object. Britannica describes the field as programs attempting to identify objects in digitized images from cameras, with much recent work using deep learning and neural networks.

Early Dreams, Early Reality

Computer vision is often said to have begun in 1966, when MIT's Seymour Papert wrote a memo called The Summer Vision Project, issued as AIM-100 and dated July 7, 1966. It set out a plan for programs that would split a scene into likely objects, background and chaos. A popular legend says it assumed vision could be solved in a summer. The memo itself is narrower than the legend, and researcher Matthew Turk has argued that Larry Roberts's 1963 PhD thesis is the better candidate for the field's birth. Either way, the early optimism ran into an unglamorous truth: seeing is deeply difficult, and hand-written rules never captured the messiness of real scenes.

Data Changes Everything

The turning point was data. Fei-Fei Li began developing the idea for ImageNet around 2006, and in 2007 she met Princeton's Christiane Fellbaum, a creator of WordNet, to discuss organizing images using WordNet's structure. The team used crowdsourcing through Amazon Mechanical Turk to label images and presented the project as a poster at the CVPR conference in 2009; Li has recalled that the conference offered only a poster slot. The dataset grew to over 14 million labeled images by some counts, though figures vary by source. The ImageNet Large Scale Visual Recognition Challenge began in 2010, with 1,000 classes and about 1.2 million training images, and gave researchers a shared yardstick.

The 2012 Jolt

In 2012, a deep convolutional network built by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton, later known as AlexNet, won the ImageNet challenge with a top-5 error of 15.3 percent. The second-best entry scored 26.2 percent, a gap of nearly 11 percentage points. The result convinced much of the field that deep learning was the way forward. According to one account, by the contest's final year in 2017 the winning accuracy had climbed from 71.8 percent to 97.3 percent, though such figures depend on how the task is defined.

Where It Works and Where It Doesn't

Today computer vision shows up in photo search, phone unlocking, medical image analysis, factory inspection and the cameras that help cars detect lanes and pedestrians. It also raises hard questions. Models can be fooled by small changes, may perform unevenly across lighting conditions or demographic groups, and can enable surveillance that people never agreed to. Accuracy on a benchmark does not guarantee reliability in the messy real world. For anyone evaluating a vision product, the sensible questions are what data it was trained on, how it was tested and what happens when it is wrong.

Why People Care

Computer vision sits behind face unlock, driver assistance, medical scans and photo apps, and it is central to debates over surveillance and bias. Knowing how it learns, and where it fails, helps you judge both the hype and the risks.

Glossary

Pixel: One tiny square of an image, stored as numbers for color and brightness.
Convolutional neural network: A neural network that scans small patches of an image to detect edges, textures and shapes, then builds up to whole objects.
Image classification: Assigning a label to a whole image, such as 'cat'.
Object detection: Finding objects in an image and drawing boxes around them.
Top-5 error: The share of test images where the correct label is not among a model's five best guesses.

Go Deeper

More to Explore

🎲 Take Me Somewhere Random