Real-World Data for SAPIEN
SAPIEN brings real object geometry to simulation. Real-world data adds the mechanical fidelity that geometry alone cannot capture.
SAPIEN at a Glance
PartNet-Mobility Object Categories
PartNet-Mobility provides 2,000+ articulated objects across 46 categories. The most commonly used categories for manipulation research are shown below.
| Category | Example Objects | Mobility Type | Sim-to-Real Gap |
|---|---|---|---|
| Cabinets & Storage | Kitchen cabinets, bathroom vanities | Revolute (doors) + prismatic (drawers) | Hinge friction, magnetic catches, dampening |
| Faucets & Taps | Kitchen faucets, bathroom taps | Revolute (handles) | Valve type variation, water resistance |
| Appliances | Microwaves, ovens, refrigerators | Revolute (doors) + buttons | Latch mechanisms, heavy doors, seals |
| Laptops & Devices | Laptops, tablet cases | Revolute (hinge) | Variable damping, magnetic closure |
| Switches & Controls | Light switches, knobs, dials | Revolute + prismatic | Detent positions, spring return |
SAPIEN vs. Related Simulation Platforms
| Feature | SAPIEN | MuJoCo | CoppeliaSim | Isaac Gym/Sim |
|---|---|---|---|---|
| Articulated objects | 2,000+ real scans (PartNet) | Manual modeling | Manual modeling | Manual modeling + URDF |
| GPU parallel | Yes (via ManiSkill3) | MuJoCo XLA | No | Yes (native) |
| Rendering | Ray-traced | Rasterized | Rasterized | RTX ray-traced |
| Part-level mobility | Annotated per-part joints | Manual joint definition | Manual joint definition | URDF-based |
| Object diversity | Scan-based (46 categories) | Limited to modeled objects | Limited to modeled objects | Limited to modeled objects |
Benchmark Profile
SAPIEN is a physics simulation platform and benchmark for interactive 3D environments. Built by UC San Diego and Stanford researchers, it provides articulated object simulation with PartNet-Mobility assets — real object meshes with annotated part mobility (hinges, sliders, handles). SAPIEN powers ManiSkill and enables research on articulated object manipulation.
The Sim-to-Real Gap
SAPIEN provides the most detailed articulated object models available, but real mechanical assemblies have friction, backlash, wear, and manufacturing variation that the PartNet-Mobility annotations capture imprecisely. Each real door handle has unique resistance; each drawer slide has unique friction profile.
Real-World Data Needed
Manipulation recordings of real articulated objects with joint angle measurements and force feedback. Diverse mechanical variation across the same object category (e.g., 50 different real drawer pulls). Material property calibration data for simulation parameter tuning.
Complementary Claru Datasets
Manipulation Trajectory Dataset
Real-world articulated object manipulation provides the authentic mechanical variation that SAPIEN's parameterized models approximate.
Custom Articulated Object Collection
Purpose-collected data on real doors, drawers, and appliances captures the manufacturing variation and wear that simulated models cannot reproduce.
Egocentric Activity Dataset
Human demonstrations of interacting with household articulated objects provide visual pretraining for SAPIEN-trained policies.
Bridging the Gap: Technical Analysis
SAPIEN's unique contribution is bringing real object geometry into simulation through PartNet-Mobility. Unlike benchmarks that use procedurally generated shapes, SAPIEN objects come from 3D scans of real products with annotated movable parts. This provides geometric fidelity but not mechanical fidelity.
The gap between SAPIEN's articulated models and real objects is subtle but significant. A simulated cabinet door has a parameterized hinge with constant friction. A real cabinet door has a hinge with friction that varies with angle, a magnetic catch at the closed position, and dampening that changes with temperature. These mechanical details affect grasp strategy, force requirements, and timing.
The PartNet-Mobility dataset contains 2,000+ articulated objects across 46 categories, but the distribution is biased toward Chinese household products (the scans were collected primarily in China). Western household objects differ in style, mechanism design, and standard dimensions. Balanced global coverage requires data from diverse geographic markets.
Claru's global collector network can capture manipulation data with real articulated objects across diverse locations, providing the mechanical variation and geographic diversity that complements SAPIEN's geometric fidelity.
Key Papers
- [1]Xiang et al.. “SAPIEN: A SimulAted Part-based Interactive ENvironment.” CVPR 2020, 2020. Link
- [2]Mo et al.. “PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding.” CVPR 2019, 2019. Link
- [3]Geng et al.. “GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation.” CVPR 2023, 2023. Link
Frequently Asked Questions
SAPIEN uses 3D scans of real products with annotated movable parts from PartNet-Mobility — real door handles, drawer slides, and appliance controls. This provides geometric fidelity that procedurally generated objects cannot match, though mechanical properties are still approximated.
A perfect 3D model of a drawer does not capture the drawer's friction profile, stick-slip behavior, or manufacturing tolerances. These mechanical properties affect grasp strategy and force requirements. Real-world manipulation data measures these properties directly.
PartNet-Mobility objects are primarily Chinese household products. Drawer handles, door mechanisms, and appliance controls differ significantly across markets — American soft-close cabinet hinges, European lever door handles, and Japanese sliding doors present different mechanical challenges. Global manipulation data ensures policies generalize to diverse product designs and mechanical conventions.
Ray-tracing produces more realistic images than rasterized rendering, with accurate reflections and shadows. But it still cannot capture real sensor artifacts — auto-exposure, motion blur, rolling shutter, dust on lenses, and specular reflections from unexpected light sources. The visual gap narrows but does not close, making real-world visual data still essential for robust policy deployment.
SAPIEN is the physics engine that powers all three versions of ManiSkill. ManiSkill adds GPU parallelization, standardized task definitions, and evaluation protocols on top of SAPIEN's articulated object simulation. Understanding SAPIEN's strengths (geometric fidelity from PartNet-Mobility) and limitations (approximate mechanical properties) is essential for interpreting ManiSkill benchmark results.
Get Real Articulated Object Data
Discuss purpose-collected data with real doors, drawers, and appliances for SAPIEN research.