RGB Kitchen Video Dataset

RGB video from real kitchen environments with monocular depth estimates provided as an annotation layer for training depth-aware kitchen robots. Capture is RGB video — Claru does not capture hardware depth.

Dataset at a Glance

RGB
Capture modality
Derived
Monocular depth layer
Scoped
Volume per project
Custom
Annotation layers

Comparison with Public Datasets

How Claru's dataset compares to publicly available alternatives.

DatasetClipsHoursModalitiesEnvironmentsAnnotations
ScanNet1.5K scans~5RGB-D707 rooms3D segmentation
NYU Depth V21.4K~2RGB-D464 scenesDepth, segmentation
Claru RGB Kitchen VideoScoped per projectScoped per projectRGB (depth as derived layer)Real kitchensMonocular depth estimates, objects, surfaces

Use Cases

Depth-Aware Manipulation

Using derived depth estimates for reach planning and collision avoidance in cluttered kitchens. Example models: UniDepth, DepthAnything, ICP grasping.

3D Scene Understanding

Building 3D kitchen representations for spatial reasoning. Example models: ScanNet++, Habitat, iGibson.

Transparent Object Detection

Labelling glass and clear objects from RGB appearance, where depth estimates are least reliable. Example models: ClearGrasp, TransCG, Dex-NeRF.

Key References

  1. [1]Dai et al.. “ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes.” CVPR 2017, 2017. Link
  2. [2]Yang et al.. “Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data.” CVPR 2024, 2024. Link
  3. [3]Sajjan et al.. “ClearGrasp: 3D Shape Estimation of Transparent Objects.” ICRA 2020, 2020. Link

How Claru Delivers This Data

Capture is RGB video. Monocular depth estimates may be provided as an annotation layer. Contact Claru to scope RGB capture and derived annotation requirements.

Frequently Asked Questions

No. Capture is RGB video — Claru does not deploy depth sensors. Monocular depth estimates computed from the RGB frames may be provided as an annotation layer; they are relative estimates, not registered sensor depth.

Monocular depth estimates are least reliable on glass, mirrors, and polished steel. Where a depth layer is scoped, those regions can be flagged as low-confidence, and transparent objects can receive boundary annotations.

Only approximately. Monocular depth estimates are relative, not metric, so point clouds back-projected from them carry scale ambiguity. Teams that need metric 3D reconstruction should treat the RGB video as input to their own reconstruction pipeline rather than as sensor ground truth.

Request a Sample Pack

Get a curated sample of RGB kitchen video with derived depth and annotations to evaluate for your project.