Agricultural Robotics Training Data

Claru collects agricultural robotics data using ruggedized sensor rigs equipped with RGB-D cameras, 5-band multispectral cameras (MicaSense RedEdge-P), GPS-RTK receivers with centimeter accuracy, and force/torque sensors for harvesting tasks. Our collection teams deploy to farms during target growth windows, following crop-specific protocols that mandate coverage across growth stages, lighting conditions, and weather states using a real-time diversity dashboard. Annotations are performed by team members with agricultural science training using crop-specific label guides with photographic references. We deliver datasets in RLDS, HDF5, or custom formats with full sensor calibration, GPS field maps, multispectral band data, and stratified splits by growth stage, cultivar, and environmental condition. Typical turnaround is 4-8 weeks aligned to the crop's seasonal window.

Agricultural robotics encompasses autonomous systems that perform farming tasks including harvesting, weeding, pruning, spraying, planting, and crop monitoring. The sector is driven by a structural labor crisis: farm labor costs have risen 20% over the past decade in the US and EU, seasonal worker availability has dropped sharply due to immigration policy changes, and the average age of farmers exceeds 57 in most developed nations. Companies like Agrobot (strawberry harvesting), Carbon Robotics (laser weeding), Abundant Robotics (apple picking), and SAGA Robotics (UV-C treatment) have raised hundreds of millions in venture capital to address this gap.

The data challenge in agricultural robotics is dominated by environmental variability. Unlike factory floors or warehouses, agricultural environments change continuously: plants grow and change shape daily, lighting shifts with sun angle and cloud cover, soil moisture affects terrain traversal, and the appearance of crops changes dramatically across growth stages. A strawberry at 30% ripeness looks entirely different from one at 90% ripeness, and the grasping strategy differs accordingly. A weeding robot must distinguish crop seedlings from weed seedlings that may differ by only a few millimeters in leaf shape at early growth stages. This variability means that datasets collected in one week may be partially obsolete by the next, and data from one growing season may not transfer well to the next due to different cultivar selections, planting densities, or weather patterns.

The modality requirements are also distinct from indoor robotics. Multispectral imaging (capturing near-infrared, red edge, and visible bands) reveals plant health information invisible to RGB cameras — NDVI (Normalized Difference Vegetation Index) computed from NIR and red bands indicates chlorophyll content and can distinguish stressed plants from healthy ones. GPS-RTK positioning with centimeter accuracy is essential for row-following navigation and for building field maps that persist across visits. Force sensing is needed for harvesting tasks where the robot must detach fruit without damaging the plant or the produce — a ripe tomato requires 1-3 N of pull force, while an unripe one resists at 5-10 N, and exceeding the damage threshold bruises the fruit and reduces shelf life.

Harvesting robots represent the most commercially advanced category. Agrobot's strawberry harvester uses machine vision to locate ripe berries, estimate stem attachment points, and execute a twist-and-pull motion through a custom end-effector with 24 picking arms operating in parallel. Their system requires per-cultivar training data because berry size, cluster density, and stem rigidity vary by variety. Abundant Robotics (now acquired by Wärtsilä) built an apple-picking system using vacuum suction that achieved 1 apple per second pick rates, but required extensive training data on apple detection under varying canopy occlusion. The FFRobotics multi-arm harvester demonstrated kiwifruit and apple picking with 3-finger grippers, publishing success rates of 85% on exposed fruit but dropping to 50% when fruit was partially hidden by leaves — a direct consequence of insufficient occlusion training data.

Weeding is the second major application area. Carbon Robotics deploys the LaserWeeder, a tractor-towed system using computer vision to distinguish crop from weed and firing 150 W CO2 lasers at weed centers with millimeter precision. Their system processes 100,000+ plants per hour and requires massive labeled image datasets — typically 50,000-200,000 annotated images per crop-weed combination covering multiple growth stages. The training data must include dawn, midday, and dusk lighting (weeds look different under each), wet and dry soil conditions (water droplets on leaves change appearance), and all growth stages from cotyledon to mature plant. Blue River Technology (acquired by John Deere for $305M) uses a similar see-and-spray approach with herbicide microdosing rather than lasers.

Navigation and field mapping represent the foundation layer. Autonomous tractors from companies like Monarch Tractor, Bear Flag Robotics (acquired by John Deere), and Sabanto use GPS-RTK combined with visual odometry and LiDAR for row-following and headland turning. The data requirements for navigation are less about manipulation and more about coverage: thousands of hours of driving data across different field geometries (straight rows, curved rows, terraced hillsides), soil conditions (dry, muddy, frozen), and obstacle types (irrigation equipment, rocks, workers, animals). Crop monitoring drones (DJI Agras, senseFly eBee) generate aerial multispectral imagery for field health mapping, producing datasets that can exceed 1 TB per field per season at survey-grade resolution.

Agricultural data collection is fundamentally seasonal and location-dependent, requiring planning months in advance of the target growth stage. A strawberry harvesting dataset must be collected during a 4-8 week harvest window, and if quality issues are discovered mid-collection, there may not be time to re-collect before the season ends. This makes protocol validation on small pilot collections (50-100 recordings in the first few days of the season) critical — the cost of discovering a sensor calibration error in week 6 of a 8-week season is effectively the loss of the entire dataset.

The sensor stack for field robotics differs from indoor setups. Cameras must be ruggedized against dust, moisture, and temperature extremes (0-45 C operating range for most growing environments). Multispectral cameras (e.g., MicaSense RedEdge-P with 5 bands or Sequoia+ with 4 bands) capture NIR, red edge, green, and red channels alongside RGB, enabling NDVI computation for plant health assessment. GPS-RTK base stations (e.g., Emlid Reach RS3) must be set up within 10 km of the collection site and provide real-time corrections via LoRa or cellular link. For harvesting tasks, a 6-axis force/torque sensor (ATI Mini45 or equivalent) between the arm wrist and end-effector captures the detachment force profile — this is the critical signal for learning gentle harvesting without damaging fruit or plant.

Diversity axes for agricultural data include: growth stage (early vegetative, flowering, fruit set, green fruit, ripening, mature), time of day (dawn, morning, midday, afternoon, dusk — each producing different shadow patterns and color temperature), weather conditions (clear, overcast, after rain with wet leaves), field geometry (row spacing, plant density, trellising system for vine crops), and cultivar variation (different varieties of the same crop differ in size, color, growth habit, and detachment force). Claru's agricultural collection protocol mandates coverage across all these axes using a diversity matrix: for each crop, we define minimum representation requirements per growth stage, lighting condition, and weather state, tracked through a real-time collection dashboard.

Annotation for agricultural datasets requires domain expertise that generic labeling workforces lack. Ripeness must be scored on a crop-specific scale (e.g., strawberry: white, turning, three-quarter, full ripe, overripe), not a generic 1-5 rating. Weed species must be identified by a trained agronomist or from a reference library, not just marked as 'weed.' Disease symptoms must be classified by pathogen type (fungal, bacterial, viral, nutrient deficiency) because the robot's response differs — diseased fruit should be removed to prevent spread, while nutrient-deficient plants need treatment, not removal. Claru employs annotators with agricultural science backgrounds and maintains crop-specific annotation guides with photographic references for each label category.