Technology
How do robots recognise people?
A robot vacuum that steers round a sock, a doorbell that tells a person from a cat, a drone that keeps its distance from you: all of them rely on object detection. A camera that can do that at home also raises questions a distance sensor does not.
In short. Object detection answers two questions about a picture: what is in it, and where. A neural network trained on many labelled images draws a box round each object it recognises, with a label and a confidence score. It is measured on images it has not seen, counting misses and false alarms, and lowering one raises the other. Detectors struggle with light, angles and things unlike their training pictures, and a camera that looks at your home raises privacy questions that a distance sensor does not.
What, and where
Recognising that a photo contains a cat is classification. Detection goes further: it finds every object of the kinds it knows, draws a box round each, and gives each a label and a score between 0 and 1 for how sure it is. A robot then decides what to do with boxes above a chosen score: slow down for a person, steer round a sock.
Learning from labelled pictures
Detectors learn from examples. A widely used research dataset, COCO, published in 2014, holds 328,000 images of everyday scenes with 2.5 million labelled objects of 91 types that, in its authors’ words, a four-year-old would easily recognise. A network shown enough labelled examples learns the patterns that make a person a person, from many angles and in many settings.
A 2019 survey by Zou and colleagues traces the field over a quarter of a century, from the hand-designed features of the 1990s to today’s deep learning detectors. The progress is real, and so are the limits below.
Misses and false alarms
Every detector makes two kinds of mistake: missing something that is there, and seeing something that is not. The score threshold trades one against the other. Set it low and fewer people are missed, but a coat on a chair may be taken for one; set it high and the reverse. Where a miss matters more than a false alarm, as with a person near a robot, it makes sense to set the threshold low and to back the camera up with other sensors.
Why a robot’s camera is harder
Most training photos are taken by people, at eye level, in decent light, with the subject framed. A robot sees the world from near the floor or near the ceiling, in a dim hallway, through motion blur, with people cut off at the edge of the frame. A detector that does well on photos can do worse from a robot’s point of view, which is why it has to be tested with pictures from the robot’s own camera and height: why robots are tested in simulation first.
A camera in your home
A camera that can recognise people also records them. Before you bring one home, ask where its images are processed, whether they leave your home, who can see them, and whether you can turn the camera off: home robots and camera privacy.
How Alkyon is designed
Alkyon, our concept for a flying home companion, is designed with a front camera whose images are processed on its dock, in your home. A detector looks for people, pets and curtains: a person nearby makes it slow down and widen its margins, and fabric is kept at a distance. The detector only adds caution: the distance sensors and fixed rules, such as keeping 1 m from people, work without it. It is a concept in development, for Europe only: an autonomous indoor drone.
Sources, read 4 November 2026: Lin and colleagues, Microsoft COCO: common objects in context, 2014; Zou and colleagues, object detection in 20 years: a survey, 2019, revised 2023 (both arXiv). General information.
Questions
What is object detection?
Finding things in an image and saying what they are: the software draws a box round each object it recognises, with a label such as person or cat and a score for how sure it is.
Why do object detectors make mistakes?
They learn from labelled pictures, so unusual light, angles, poses and objects they rarely saw cause misses and false alarms. Every detector also trades one against the other: catching more real objects means more false alarms.