The Sim-to-Real Gap Explained: Why It Happens and How to Close It (2026)

The sim-to-real gap is the performance degradation that occurs when a robot policy trained entirely (or primarily) in simulation is deployed on physical hardware. It is one of the central unsolved problems in robotics research, and it gets more severe as tasks become more complex.

Simulation offers compelling advantages for robot training: infinite data, no hardware wear, parallelizable compute, safe exploration of failure modes, and the ability to reset environments instantly. Isaac Gym and its successor Isaac Lab from NVIDIA can run thousands of parallel simulation environments simultaneously, enabling reinforcement learning training that would take years in the real world to complete in hours. MuJoCo, PyBullet, and Webots are similarly used across manipulation research.

The problem is that simulation is a model, and every model has approximation errors. When a policy trained in simulation encounters the real world, it faces inputs it was never trained on — real physics, real lighting, real sensor noise, and real situations that the simulation did not generate. The policy was trained to be optimal for the simulated world, not for the real one.

The gap is not one thing. It is four distinct phenomena with different causes and different mitigations. Treating it as a single problem is why many sim-to-real transfer attempts fail: a team applies domain randomization to address the visual gap, sees no improvement, and concludes that sim-to-real doesn't work — when in reality they were ignoring the physics approximation or sensor noise gaps that were the actual failure modes.

Physics simulation approximates reality at the level of contact mechanics, friction, and deformable body dynamics. For rigid-body manipulation in constrained environments (e.g., pick-and-place with simple geometry), the approximations are reasonable. For contact-rich manipulation — assembly, folding, pouring, cutting — they diverge significantly.

MuJoCo (Multi-Joint dynamics with Contact) uses a convex optimization approach to contact forces that produces stable simulation but does not match real contact dynamics for soft materials. Isaac Gym's PhysX engine handles rigid bodies well but has well-documented limitations for deformable objects and slip-contact interactions. Neither simulator accurately models the behavior of a compliant parallel gripper grasping a soft object like a foam cup or a piece of fruit.

Domain randomization over physics parameters (mass, friction coefficients, joint damping) partially compensates for this by forcing policies to handle a range of dynamics rather than a single simulated point estimate. But there is a systematic limit: if the real contact physics falls outside the randomization range — which it often does for novel object geometries or materials — the policy trained with randomized physics will still fail.

The only reliable fix for physics approximation error is real interaction data: robot demonstrations or teleoperation logs that capture actual contact dynamics, including failure cases and recovery behaviors. This is why even simulation-heavy pipelines like GR00T N1's training include real robot demonstration data — the simulation provides breadth and scale, while the real data provides physics accuracy.

Simulation generates scenarios from a designed distribution. Real environments contain everything that distribution did not anticipate: a cup on a tilted surface, a person walking past during manipulation, an object that looks like a training object but is made of a different material, a workspace that is partially occluded by something that was not there during collection.

This long-tail failure mode is arguably the hardest to address through simulation alone, because it requires anticipating the unanticipated. Procedural generation can add surface clutter and varied object positions, but it cannot generate the genuinely unexpected. The more diverse the deployment environment — kitchen versus lab versus warehouse versus outdoor construction site — the larger this gap becomes.

The implication for data: long-tail coverage requires real-world data collected at scale across diverse environments and conditions. Large-scale egocentric video datasets provide exactly this — they capture the visual diversity of real human manipulation across hundreds of location types, lighting conditions, object configurations, and task contexts that simulation cannot enumerate.

Claru's 500K+ egocentric clips, captured by 10,000+ contributors across 100+ cities on 6 continents, cover the visual long-tail of real manipulation environments. Each clip captures a genuine human in a genuine environment performing genuine tasks — the kind of distributional diversity that simulation cannot procedurally generate. For robotics teams training models that will deploy into unstructured real-world environments, this egocentric data directly addresses the long-tail scenario gap that domain randomization and simulation cannot.

Domain randomization achieved a real milestone with Dactyl — successfully transferring a dexterous in-hand manipulation policy from MuJoCo simulation to a physical Shadow Dexterous Hand with no real-world training data. This result is frequently cited as proof that simulation is sufficient for complex manipulation.

It is important to understand what Dactyl actually demonstrated. The task (rotating a Rubik's cube to a target face configuration) is a continuous control problem over a rigid object with relatively predictable contact mechanics. It does not involve deformable objects, novel materials, environmental clutter, or human interaction. Domain randomization can close the sim-to-real gap for tasks with bounded visual and physical variation — and Dactyl is that type of task.

For tasks that do involve deformable objects, diverse environments, or real-world clutter, DR alone is insufficient. The 2023 results from Physical Intelligence and others on deformable object manipulation (laundry, food packaging, cable management) all required real robot demonstration data precisely because DR could not handle the contact physics divergence for these materials.

The emerging consensus in 2026 is a hybrid approach: use simulation + DR for maximum data volume and safe exploration of failure modes, then supplement with real-world data to close the residual gaps. The ratio depends on task complexity — for rigid pick-and-place, sim-to-real with DR can work with minimal real data; for dexterous manipulation of deformable objects in diverse environments, the real data component is substantial.