Outdoor RGB Video Dataset
RGB video from outdoor environments for training monocular depth estimation and terrain-aware navigation, with traversability labels and obstacle annotations. Capture is single-camera RGB video — Claru does not capture calibrated stereo pairs.
Dataset at a Glance
Why Stereo Outdoor Data Matters for Robotics
Outdoor mobile robots -- delivery bots, agricultural rovers, search-and-rescue platforms, military ground vehicles, and planetary exploration rovers -- must perceive 3D terrain structure to navigate safely. Unlike indoor robots that can assume flat floors and regular geometry, outdoor robots encounter slopes, ditches, loose gravel, vegetation of varying density, water hazards, and terrain that changes dramatically with weather and season. Stereo vision provides dense, real-time depth estimation that is more reliable than monocular approaches in the textureless and repetitive environments common outdoors.
Depth estimation from calibrated stereo pairs is fundamentally different from monocular depth prediction. Stereo provides metric depth (actual distances in meters) through geometric triangulation, while monocular methods produce relative depth that requires scale estimation and frequently fails on textureless surfaces, repetitive patterns (crop rows, paving stones), and at the far-range distances critical for outdoor navigation. Training stereo-specific depth networks requires calibrated stereo image pairs with precise ground-truth disparity maps. Claru's outdoor data is single-camera RGB, so it serves monocular depth estimation and appearance-based traversability rather than stereo matching; teams that need metric stereo ground truth typically pair it with stereo benchmarks.
Existing stereo outdoor datasets like KITTI are limited to urban driving scenes from a single geographic region. They lack the terrain diversity that field robots encounter: forest trails, rocky hillsides, muddy paths, sand dunes, snow-covered ground, agricultural fields between rows, construction sites, riverbanks, and the transitional zones between terrain types that are often the most challenging for traversability estimation. Claru captures single-camera RGB video, and campaigns can target terrain types, seasons, and weather conditions; Claru does not capture calibrated stereo pairs or synchronized IMU data.
Research from ICRA 2024 and the Field Robotics workshop at RSS 2023 shows that stereo depth networks trained on diverse outdoor terrain data improve traversability prediction accuracy by 35-50% compared to networks trained on urban-only stereo datasets, with the improvement driven primarily by exposure to non-rigid surfaces (vegetation, mud, snow) and unstructured geometry (rock fields, root systems) absent from structured road environments.
Capture and Collection Methodology
Capture is single-camera RGB video. Claru does not deploy calibrated stereo rigs, LiDAR, or hardware-synchronized IMU and GPS/RTK packages. Where a project needs depth, it is supplied as a derived annotation layer -- monocular depth estimates computed from the RGB frames -- and is labelled as a relative estimate rather than metric ground truth.
Camera placement (handheld, body-worn, or mounted on a platform) is scoped per project so that viewpoint and motion profile match the target robot as closely as practical.
Campaigns can target terrain types across geographic regions and seasons. Terrain categories include: paved trails, gravel paths, forest floors (deciduous and coniferous), grasslands, agricultural rows, rocky hillsides, sandy beaches and dunes, muddy paths, snow-covered ground, urban sidewalks with vegetation, construction sites, riverbanks, wetlands, desert hardpack, and mixed transitional zones. Each terrain type can be captured in multiple weather conditions (dry, wet, after rain, during light rain, overcast, direct sun) and times of day (morning, midday, evening) to cover the full range of lighting conditions.
Environmental metadata per session can include terrain type, weather conditions, time of day, season, recent precipitation history, vegetation state (full leaf, partial, bare), ground moisture level estimate, and notable hazards encountered (water crossings, steep slopes, loose surfaces). This metadata enables researchers to condition navigation models on environmental context and study how traversability changes with conditions.
Comparison with Public Datasets
How Claru's outdoor RGB video compares to publicly available alternatives for outdoor robot navigation.
| Dataset | Clips | Hours | Modalities | Environments | Annotations |
|---|---|---|---|---|---|
| KITTI Stereo (2012) | ~400 pairs | <1 | Stereo RGB, LiDAR | Urban driving (1 city) | Sparse disparity (LiDAR) |
| Middlebury Stereo (2014) | ~33 pairs | N/A (stills) | Stereo RGB | Indoor lab scenes | Dense disparity (structured light) |
| TartanAir (2020) | ~1M frames | ~30 | Stereo RGB (synthetic) | Simulated outdoor | Perfect depth, flow, segmentation |
| Claru Outdoor RGB Video | Scoped per project | Scoped per project | Single-camera RGB (depth as derived layer) | Real terrain, targeted per campaign | Monocular depth estimates, traversability, obstacles, terrain class, weather |
Annotation Pipeline and Quality Assurance
Stage one automated processing runs on the RGB frames. Where a depth layer is scoped, a pre-trained monocular depth model produces per-frame relative depth estimates, delivered as a derived layer with its provenance recorded -- not as stereo disparity or metric ground truth.
Stage two human annotation adds semantic and traversability labels: terrain type classification per region (15+ classes including path, grass, rock, mud, water, vegetation, obstacle, structure), traversability scoring on a 5-point scale (clear, traversable, difficult, dangerous, impassable), obstacle detection and classification (static obstacles like rocks and posts, dynamic obstacles like people and animals, negative obstacles like holes and ditches), and path boundary segmentation for trail-following applications.
Stage three QA combines automated consistency checks with human review. Traversability annotations are reviewed by a second annotator with field robotics experience. Agreement targets: 96%+ on terrain classification, 93%+ on traversability scoring, and 95%+ on obstacle detection. Clips with degraded image quality (rain drops on lenses, extreme sun flare, motion blur) are flagged with quality scores rather than discarded, enabling researchers to train models that handle degraded conditions.
The complete annotation taxonomy covers 15+ terrain classes, 5 traversability levels, 20+ obstacle categories (static, dynamic, and negative), path boundary delineations, weather and visibility condition tags, surface roughness labels, and slope category labels. This enables training navigation systems that assess terrain safety from visual appearance before committing to traverse it.
Use Cases
Monocular Depth Estimation
Fine-tuning and evaluating monocular depth models on outdoor terrain. The diversity of terrain textures, weather conditions, and lighting builds models robust to conditions where depth estimation commonly fails. Stereo matching networks (RAFT-Stereo, CREStereo) need calibrated stereo pairs, which this data does not include. Example architectures: Depth Anything V2, MiDaS, ZoeDepth, Metric3D.
Terrain-Aware Navigation
Training traversability estimation models that predict which terrain is safe to cross based on visual appearance and depth structure. Models learn to associate visual texture patterns with traversability -- distinguishing firm gravel from soft sand, or dense vegetation from sparse brush. Critical for field robots operating beyond paved roads.
Outdoor Scene Segmentation
Training semantic segmentation models that separate path, vegetation, rock, water, and obstacles across diverse terrain, weather, and lighting. Terrain-class and path-boundary annotations on real outdoor video give field robots the scene parsing they need beyond paved roads.
Key References
- [1]Geiger et al.. “Are We Ready for Autonomous Driving? The KITTI Vision Benchmark Suite.” CVPR 2012, 2012. Link
- [2]Lipson et al.. “RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching.” 3DV 2021, 2021. Link
- [3]Wang et al.. “TartanAir: A Dataset to Push the Limits of Visual SLAM.” IROS 2020, 2020. Link
- [4]Frey et al.. “Learning Robust Terrain Traversability from Multi-Modal Self-Supervised Data.” ICRA 2024, 2024. Link
How Claru Delivers This Data
Claru's collector network records single-camera RGB video across outdoor environments targeted per campaign -- forest trails, sandy coastal paths, rocky alpine terrain, agricultural access roads. This diversity captures the terrain variation that outdoor robots will encounter in real deployment, not just the controlled paths used in academic field robotics experiments.
Custom campaigns can target specific terrain types (agricultural, urban trails, wilderness), weather conditions (specifically rain or snow collection for degraded-condition robustness), seasons (fall leaf cover, spring mud, winter snow), or camera viewpoints (handheld, body-worn, or platform-mounted). Timelines are scoped per campaign.
Data is delivered as RGB video with traversability, obstacle, and terrain annotations. Monocular depth estimates, where scoped, ship as a separate derived layer with per-frame provenance. Format options include RLDS, HDF5, WebDataset, and MP4+JSON.
Frequently Asked Questions
No. Capture is single-camera RGB video -- Claru does not capture calibrated stereo pairs. Teams that need metric stereo ground truth typically pair terrain-diverse RGB video with stereo benchmarks such as KITTI.
Monocular depth estimates computed from the RGB frames can be provided as a derived annotation layer. They are relative estimates, not disparity maps or metric ground truth from stereo or LiDAR.
Campaigns can target paved trails, gravel paths, forest floors, grasslands, agricultural rows, rocky hillsides, sandy surfaces, muddy paths, snow-covered ground, construction sites, riverbanks, wetlands, desert hardpack, urban sidewalks, and mixed transitional zones.
No. Claru does not deliver hardware-synchronized IMU data, camera-IMU calibration, or RTK ground-truth trajectories. Capture is RGB video; depth, where needed, is a derived annotation layer.
Yes. Custom campaigns can target specific weather (rain, snow, fog, direct sun, overcast) or seasonal conditions (full leaf, partial leaf, bare, spring mud). Degraded-condition data is particularly valuable for training robust depth and traversability models.
Related Resources
Request a Sample Pack
Get a curated sample of outdoor RGB video with traversability annotations to evaluate for your outdoor navigation project.