Stereo Outdoor Dataset

Outdoor mobile robots -- delivery bots, agricultural rovers, search-and-rescue platforms, military ground vehicles, and planetary exploration rovers -- must perceive 3D terrain structure to navigate safely. Unlike indoor robots that can assume flat floors and regular geometry, outdoor robots encounter slopes, ditches, loose gravel, vegetation of varying density, water hazards, and terrain that changes dramatically with weather and season. Stereo vision provides dense, real-time depth estimation that is more reliable than monocular approaches in the textureless and repetitive environments common outdoors.

Depth estimation from calibrated stereo pairs is fundamentally different from monocular depth prediction. Stereo provides metric depth (actual distances in meters) through geometric triangulation, while monocular methods produce relative depth that requires scale estimation and frequently fails on textureless surfaces, repetitive patterns (crop rows, paving stones), and at the far-range distances critical for outdoor navigation. Training stereo-specific depth networks requires calibrated stereo image pairs with precise ground-truth disparity maps -- data that single-camera datasets simply cannot provide.

Existing stereo outdoor datasets like KITTI are limited to urban driving scenes from a single geographic region. They lack the terrain diversity that field robots encounter: forest trails, rocky hillsides, muddy paths, sand dunes, snow-covered ground, agricultural fields between rows, construction sites, riverbanks, and the transitional zones between terrain types that are often the most challenging for traversability estimation. Claru's stereo outdoor dataset captures 15+ terrain types across seasons and weather conditions with calibrated stereo pairs and synchronized IMU data.

Research from ICRA 2024 and the Field Robotics workshop at RSS 2023 shows that stereo depth networks trained on diverse outdoor terrain data improve traversability prediction accuracy by 35-50% compared to networks trained on urban-only stereo datasets, with the improvement driven primarily by exposure to non-rigid surfaces (vegetation, mud, snow) and unstructured geometry (rock fields, root systems) absent from structured road environments.

The stereo rig uses a pair of FLIR Blackfly S cameras (1280x960, global shutter, 12mm lenses) mounted on a rigid carbon-fiber baseline bar with 120mm separation, factory-calibrated for stereo rectification. Global shutter is essential for outdoor stereo -- rolling shutter causes temporal skew between scanlines that corrupts disparity computation, especially during rapid platform motion over rough terrain. Stereo images are hardware-triggered to sub-microsecond synchronization.

An Xsens MTi-630 IMU (9-axis, 400Hz) is rigidly mounted to the stereo baseline bar, providing synchronized inertial data for visual-inertial odometry and motion compensation. GPS/RTK position is recorded at 10Hz for global localization, with RTK corrections providing centimeter-level accuracy when available. The complete sensor package weighs under 800g and mounts to standard robot platforms, survey poles, or chest harnesses for human-carried collection.

Collection spans 15+ terrain types across multiple geographic regions and all four seasons. Terrain categories include: paved trails, gravel paths, forest floors (deciduous and coniferous), grasslands, agricultural rows, rocky hillsides, sandy beaches and dunes, muddy paths, snow-covered ground, urban sidewalks with vegetation, construction sites, riverbanks, wetlands, desert hardpack, and mixed transitional zones. Each terrain type is captured in multiple weather conditions (dry, wet, after rain, during light rain, overcast, direct sun) and times of day (morning, midday, evening) to cover the full range of lighting conditions.

Environmental metadata per session includes terrain type, weather conditions, GPS track, time of day, season, recent precipitation history, vegetation state (full leaf, partial, bare), ground moisture level estimate, and notable hazards encountered (water crossings, steep slopes, loose surfaces). This metadata enables researchers to condition navigation models on environmental context and study how traversability changes with conditions.

Stage one automated processing generates dense disparity maps from the calibrated stereo pairs using RAFT-Stereo, with confidence-weighted filtering to suppress errors in textureless regions. Semi-global matching (SGM) provides a second disparity estimate, and pixels where RAFT-Stereo and SGM agree within 1 pixel are marked as high-confidence ground truth. IMU-integrated visual odometry provides camera pose for each frame, enabling temporal consistency checks on the depth estimates.

Stage two human annotation adds semantic and traversability labels: terrain type classification per region (15+ classes including path, grass, rock, mud, water, vegetation, obstacle, structure), traversability scoring on a 5-point scale (clear, traversable, difficult, dangerous, impassable), obstacle detection and classification (static obstacles like rocks and posts, dynamic obstacles like people and animals, negative obstacles like holes and ditches), and path boundary segmentation for trail-following applications.

Stage three QA combines automated geometric checks with human review. Disparity maps are validated against IMU-derived motion estimates (if the platform moved forward at known speed, the depth change between frames must be consistent). Traversability annotations are reviewed by a second annotator with field robotics experience. Agreement targets: 96%+ on terrain classification, 93%+ on traversability scoring, and 95%+ on obstacle detection. Clips with poor stereo quality (due to rain drops on lenses, extreme sun flare, or insufficient texture) are flagged with quality scores rather than discarded, enabling researchers to train robust stereo networks that handle degraded conditions.

The complete annotation taxonomy covers 15+ terrain classes, 5 traversability levels, 20+ obstacle categories (static, dynamic, and negative), path boundary delineations, weather and visibility condition tags, surface roughness estimates, and slope angle derived from stereo reconstruction. This enables training navigation systems that assess terrain safety from visual appearance before committing to traverse it.

Claru's collector network deploys calibrated stereo rigs across diverse outdoor environments -- from forest trails in the Pacific Northwest to sandy coastal paths in the Southeast, rocky alpine terrain in the Rockies, and agricultural access roads in the Midwest. This geographic diversity captures the terrain variation that outdoor robots will encounter in real deployment, not just the controlled paths used in academic field robotics experiments.

Custom campaigns can target specific terrain types (agricultural, urban trails, wilderness), weather conditions (specifically rain or snow collection for degraded-condition robustness), seasons (fall leaf cover, spring mud, winter snow), or locomotion platforms (mounted on wheeled robots, legged platforms, or human-carried for different viewpoints). Turnaround is typically 4-6 weeks for standard terrain campaigns.

Data is delivered with full stereo calibration parameters, pre-computed disparity maps (with confidence scores), IMU data at 400Hz, and GPS/RTK trajectories. All streams are time-synchronized to a common clock. Format options include RLDS, HDF5, WebDataset, and standard stereo dataset formats (KITTI-compatible if desired).