Construction Robotics Training Data

Claru collects construction robotics data using ruggedized, IP67-rated sensor rigs built for active construction sites. Our team holds OSHA safety certifications and coordinates collection schedules with general contractors to align with construction phases. We deploy longitudinal collection programs with recurring site visits, capturing data as the environment evolves from foundation through finishing. Each session includes a fresh LiDAR site scan registered to the BIM model, providing spatial context for all manipulation and navigation data. Annotations include structural element labels, assembly state, weld quality grades, safety zone compliance, and BIM deviation measurements — all performed by annotators trained in construction terminology and safety requirements. We deliver in RLDS, HDF5, or custom formats with full sensor calibration, BIM registration data, and stratified splits by construction phase, task type, and environmental condition.

Construction robotics encompasses autonomous and semi-autonomous machines that perform building tasks: bricklaying, welding, rebar tying, concrete dispensing, painting, demolition, site surveying, and material transport. The construction industry is the least digitized major sector of the global economy, with labor productivity largely flat for the past 30 years while manufacturing productivity doubled. The US construction sector faces a shortage of 650,000 workers as of 2024, and fatalities remain the highest of any industry at roughly 1,000 deaths per year from falls, struck-by, electrocution, and caught-in-between incidents. This combination of labor scarcity, low productivity, and high danger makes construction one of the most compelling robotics deployment targets.

Companies are investing heavily. Construction Robotics (SAM, the Semi-Automated Mason, lays 3,000 bricks per day versus 500 for a human mason), Dusty Robotics (autonomous layout marking), Hilti Jaibot (overhead drilling), Built Robotics (autonomous heavy equipment), Canvas (drywall finishing), Scaled Robotics (site progress monitoring), and Toggle (rebar assembly) represent a small fraction of the startups in this space. The global construction robotics market is projected to reach $166 million by 2028, though many industry analysts believe this underestimates the opportunity given the scale of the labor gap.

The data challenge in construction is distinct from factory or warehouse robotics for several reasons. First, construction sites are unstructured and constantly changing — the environment today (bare concrete slab) bears no resemblance to the environment next month (partially erected steel frame with temporary scaffolding). Second, construction materials are heavy (a standard brick weighs 2-3 kg, a rebar bundle 50+ kg, a steel beam hundreds of kilograms), requiring robots with high payload capacity and force-controlled contact. Third, safety constraints are paramount — a welding robot operating near humans must respect exclusion zones, detect personal protective equipment (PPE), and halt operations if safety violations are detected. Training data must capture all of these conditions: changing site geometry, heavy material handling dynamics, and safety-critical human-robot coexistence.

Most deployed construction robots use classical control rather than learned policies. SAM100 (the bricklaying robot from Construction Robotics) uses a pre-programmed pattern based on architectural drawings, with a human mason loading bricks and applying mortar while the robot places them. The system uses laser levels and encoder feedback for positioning but does not learn from demonstration data. Similarly, Hilti Jaibot drills ceiling holes from a pre-loaded BIM (Building Information Model) coordinate set. These systems work for repetitive, well-defined tasks but fail when site conditions deviate from the digital plan, which happens constantly in real construction.

The learning-based frontier is emerging in three areas. First, robotic welding is adopting vision-guided seam tracking where a neural network detects the joint line in real time from an arc-camera image and adjusts the torch position. Companies like Path Robotics have developed deep learning weld planning systems that generate weld paths from 3D scans of steel assemblies, but their training data requirements are substantial: 5,000-10,000 annotated weld seams across different joint types (butt, fillet, lap, groove) and material thicknesses (3-25 mm steel plate). Second, autonomous heavy equipment (excavators, bulldozers, loaders) is using imitation learning from teleoperated demonstrations. Built Robotics has collected thousands of hours of expert operator data for trench digging and grading tasks, using this to train policies that match 80-90% of expert performance on routine operations. Third, site progress monitoring uses 3D change detection: compare a daily LiDAR scan against the BIM model to automatically compute construction progress, detect deviations, and flag safety issues.

The key research challenge is adapting manipulation policies to the evolving construction environment. A bricklaying robot must handle the wall height increasing by one course after each row (changing the end-effector height requirement), mortar consistency varying with temperature and age (affecting adhesion force), and wind loads on tall wall sections (requiring additional bracing that changes the workspace geometry). This type of progressive environmental change is unique to construction and is not well represented in standard manipulation datasets. Training data must be collected longitudinally across the construction timeline, not just at a single point in time.

Construction site data collection requires industrial-grade sensors that withstand dust, vibration, temperature extremes (-10 to 50 C), and occasional impacts. Consumer-grade RealSense cameras fail within days on active construction sites due to dust infiltration. The recommended sensor stack includes: IP67-rated industrial cameras (e.g., FLIR Blackfly S or Basler ace2 in ruggedized enclosures), 3D LiDAR scanners (Leica BLK360 G2 for static scans or Velodyne VLP-16 for mobile mapping), GPS-RTK receivers (Trimble R12i or equivalent with centimeter accuracy), 6-axis force/torque sensors rated for the payload range (ATI Omega series for heavy-duty applications up to 10,000 N), and IMUs for equipment tilt and vibration monitoring. For welding applications, add a through-arc camera (Xiris XVC-O) that images the weld pool through a bandpass filter, a thermal camera (FLIR A700) for heat-affected zone monitoring, and weld current/voltage sensors on the power supply.

Collection logistics on construction sites are complex. Access requires safety training (OSHA 10/30 certification in the US), PPE (hard hat, high-visibility vest, steel-toe boots, safety glasses), and site-specific orientation. Data collection must be coordinated with the construction schedule — you cannot record bricklaying data if the masons are not working that day, and you cannot set up sensors in an area where concrete is being poured. Claru's construction data protocol begins with a site survey 2-4 weeks before collection to map the environment, identify power sources for sensors, establish safe equipment staging areas, and coordinate with the general contractor on the collection schedule.

Diversity axes for construction data include: construction phase (foundation, framing, enclosure, MEP rough-in, finishing), material type (concrete, steel, wood, masonry, drywall), weather conditions (clear, rain, wind, temperature extremes — each affecting material properties and worker behavior), time of day (lighting changes dramatically on open sites), and operator variation. For material transport tasks, diversity must also cover terrain conditions (gravel, mud, paved, graded, sloped) and load configurations (empty, partial, full, oversize). Annotations include structural element identification (beam, column, slab, wall segment), assembly state (placed, fastened, welded, inspected), safety zone compliance (workers within exclusion zone, PPE detection), and quality metrics (level, plumb, alignment to BIM coordinates). All annotations reference the BIM model coordinate system so that as-built data can be compared to the as-designed model.