Edge cases in computer vision: why the rare events matter most
Edge cases in computer vision are the rare scenes a model barely saw in training, and they are where it fails. What the research shows about the long tail and where the data comes from.
By Paul Botha,

An edge case is a scene a model rarely or never saw in training: a deer on the road at night, a car driving against traffic, a pushchair in the carriageway. Ordinary data collection misses them, and that is where models fail. A 2022 benchmark found standard detectors fell to no more than 12.8% recall on such corner cases.
What are edge cases in computer vision?
Most real-world data follows a long tail. A few situations happen constantly and thousands happen rarely. A model trained on what is common learns the head of that tail well and the end of it badly.
The CODA benchmark (ECCV 2022) collected 1,500 driving scenes with about four corner cases each. Standard object detectors dropped to no more than 12.8% mean average recall, a fall of 30% to 50% against normal scenes.
What is the long-tail problem in AI?
In the nuScenes driving dataset the rarest class has 1,670 times fewer examples than the most common, according to a 2022 study. In Argoverse 2 the gap is 2,500 times.
The same study raised detection of pushchairs from 0.1 to 31.6 average precision by changing how rare classes were handled.
ImageNet-LT (2019) was built to study the problem, with classes ranging from 1,280 images down to 5.
Why do self-driving cars struggle with rare events?
Because they cannot drive enough to meet them all. In 2016 RAND estimated that proving autonomous vehicle safety by driving alone could take hundreds of millions, sometimes hundreds of billions, of miles. Waymo's WOD-E2E dataset (2025) instead collects 4,021 segments of scenarios that occur less than 0.03% of the time.
The cost of a missed edge case can be high. In the 2018 Tempe crash, the US National Transportation Safety Board found the system detected the pedestrian 5.6 seconds before impact but kept changing its classification of her.
It never accurately classified her as a pedestrian or predicted her path.
The NTSB put the probable cause on the safety driver's distraction, not on perception alone, but the classification failure is a textbook edge case.
Why can't you just collect more data?
Because the rarest cases stay rare however much you collect. Doubling the hours of ordinary driving doubles the hours of ordinary driving. Waymo's examples of long-tail scenes include a deer on the road at night and a car travelling against traffic at 70 miles an hour: events most fleets will record a handful of times, if ever. Finding them means searching for them deliberately, or licensing footage that already caught them.
Where do you find training data for edge cases in computer vision?
Hard benchmarks such as CODA and WOD-E2E, to measure the problem.
Fleet mining: searching large volumes of recorded footage for rare moments.
Targeted synthetic data, kept to a small share and tested on real footage.
Real footage of real incidents, filmed by people who happened to be there.
Held-out real edge cases for evaluation, never used in training.
Each source has a weakness. Benchmarks are small, fleet mining finds only what your own cameras saw, and synthetic scenes inherit the blind spots of whatever generated them. Real footage filmed by many different people, in many places, on many devices, covers ground no single fleet or simulator will.
Motion56 licenses real footage of rare events: crashes, near-misses, falls, fires, storms and crowds, filmed on dashcams, CCTV and phones by the people who were there, with what happens described on every clip. See dashcam training data and synthetic vs real training data, or send a brief through training data.
Rare events are on the agenda tonight at our panel, Out of the lab: what happens when video AI meets the real world, in London on Thursday 19 November 2026.