Technology
Sim-to-real: why robots are tested in simulation first
In a simulation, a crash costs nothing, thousands of trials can run overnight, and a gust of wind arrives on demand. That is why robots are trained and tested in software before they move for real. The catch is the gap between the simulation and the world.
In short. Simulation lets engineers test a robot in far more situations than they could build, including dangerous and rare ones. But every simulator is wrong somewhere, and a controller tuned too closely to it can fail on real hardware: the reality gap. One answer is to randomise the simulation so widely that the real world looks like one more variation. The other is to measure the real robot and correct the simulation. A result from simulation is only as good as the physics behind it.
Why simulate
- Crashes are free. A drone can hit a wall a thousand times without a single broken propeller.
- Scale. Many simulated robots can run side by side, and faster than real time.
- Rare events on demand. A person stepping into the path, a draft from an open window, a sensor that drops out.
- The truth is known. The simulator knows exactly where the robot really is, so its own estimate can be checked to the millimetre.
The reality gap
Tobin and colleagues called it the “reality gap” that separates simulated robotics from experiments on hardware. Friction, motor delays, moving air, the noise of each sensor and reflections are all approximated. A controller that quietly depends on a detail the simulator gets wrong will fail when it meets the real thing.
Randomise everything
Their 2017 study trained an object detector only on simulated images with random, deliberately unrealistic textures. On real images it located objects to within 1.5 cm, coped with distractions and partly hidden objects, and was good enough to guide a robot grasping things in clutter. The idea, in the authors’ words: with enough variability in the simulator, “the real world may appear to the model as just another variation”.
The same idea works for physics. Vary the mass, the motors, the delays and the sensor noise across thousands of simulated robots, and a controller that works for all of them is more likely to work for the one you build.
Close the gap by measuring
Randomising has a limit: if the real robot sits outside every range you tried, the simulation has nothing to say about it. The other half of the work is measurement: put the real motors, propellers and sensors on a test stand, measure how they behave, update the simulation, and test again.
What a simulation result means
A result from simulation is a statement about the simulation. Before you trust one, ask what in it was measured, what came from the literature and what was assumed, and whether the result holds when the assumptions change.
How we work on Alkyon
Alkyon, our concept for a flying home companion, is developed simulation first. Its flight controller is tested on large numbers of randomly varied robots: slower or faster motors, small differences in weight and balance, noisy sensors, and drafts from fans and open windows. Every physics constant in that simulation is still an estimate from the literature or an assumption until we measure the real parts on a test stand, and we say so with each result. It is a concept in development, for Europe only: an autonomous indoor drone.
Source, read 4 November 2026: Tobin and colleagues, domain randomization for transferring deep neural networks from simulation to the real world, 2017 (arXiv).
Questions
What is the reality gap in robotics?
The difference between a simulated robot and a real one: friction, motor delays, air, sensor noise and light are all approximated in a simulator. A controller or a model that relies on something the simulator gets wrong can fail on real hardware.
What is domain randomisation?
Training or testing across many randomly varied versions of the simulation, such as textures, lighting, masses, delays and sensor noise, so that the real world looks like just one more variation and what works across all of them is more likely to work for real.