Data & ML Infrastructure
Data pipelines, simulation & MLOps — the data stack behind physical AI.
Prerequisites
Why it matters in robotics
As robotics shifts from hand-tuned controllers to learning-based, fleet-scale systems, the infrastructure that makes scaled robot learning possible becomes the bottleneck: data pipelines, simulation, and MLOps. It spans collecting and standardizing heterogeneous robot data from many robots and logging formats into shared training datasets, generating training data in simulation across different simulators suited to different needs and crossing the gap between simulated and real-world behavior, and deploying, monitoring, and updating policies across a fleet of robots. Concrete tooling choices trade off throughput, cost, reproducibility, and safety — infrastructure is often the real limit on how fast robot learning scales, not the model.
What to study
- ✓Robot data pipelines and dataset formats: ROS2 bags (rosbag2/MCAP) for raw logging, and standardized training formats like RLDS and LeRobotDataset (Parquet+MP4); how Open X-Embodiment pooled 60+ datasets across 22 embodiments into one schema.
- ✓Simulation platforms and when to use each: GPU-parallel Isaac Sim/Isaac Lab for massively parallel RL, MuJoCo for fast contact-rich dynamics, Gazebo for ROS-integrated robot/sensor simulation.
- ✓Sim-to-real transfer: domain randomization (dynamics, textures, lighting), system identification, and how parallel-environment RL scaling changes wall-clock training cost and data economics.
- ✓Fleet MLOps for robotics: teleoperation and data-flywheel collection, OTA policy deployment, observability/logging, and reproducible training plus rigorous (often real-world) evaluation.
Where to study
Cheatsheet
Raw logs vs. training datasets
Robots log raw, timestamped sensor and control data as ROS2 bags (rosbag2), increasingly in the MCAP container format — good for full-fidelity replay and debugging, but heterogeneous across robots and logging setups.
Training pipelines instead want a standardized schema: RLDS (RL Dataset format, TFRecord-based) or LeRobotDataset (Parquet for low-dim data + MP4 for video) store episodes of (observation, action, reward) tuples in a consistent shape regardless of which robot produced them. Open X-Embodiment pooled 60+ separately-collected datasets across 22 robot embodiments into one shared schema, which is what made cross-embodiment policy training possible at all.
Choosing a simulator
Isaac Sim / Isaac Lab: GPU-parallel (thousands of environments at once), built for massively parallel RL and photorealistic sensor rendering — the throughput leader when you need millions of episodes.
MuJoCo: fast, accurate contact-rich dynamics on CPU, the standard for manipulation/locomotion research where physical realism of contact matters more than visual realism or environment count.
Gazebo: tightly ROS-integrated, simulates full robot + sensor stacks (not just dynamics) — the default choice when the goal is testing the actual ROS2 nodes and message flow, not training a policy at scale.
Sim-to-real gap and domain randomization
A policy trained purely in sim overfits to the simulator's specific dynamics, textures, and lighting — the sim-to-real gap. Domain randomization deliberately varies friction, mass, latency, textures, and lighting during training so the policy generalizes instead of memorizing one simulated world.
System identification (fitting the sim's dynamics parameters to match real measured behavior) narrows the gap from the other direction — instead of randomizing blindly, make the nominal sim more accurate first, then randomize around that better nominal point.
Parallelism changes the economics, not just the speed
Running thousands of GPU-parallel environments turns wall-clock training time from weeks to hours for the same sample count — which changes what's economical to try (more seeds, more randomization ranges, more architecture sweeps) rather than just making one fixed experiment faster.
Fleet MLOps loop
Teleoperation and on-robot data collection feed a data flywheel: deploy a policy, log where it struggles, collect targeted teleop demonstrations for those cases, retrain, redeploy. OTA (over-the-air) update mechanisms push new policy weights to a fleet without a truck roll, alongside observability (logging, dashboards, alerting on anomalous behavior) so a regression is caught from telemetry, not from a robot getting stuck in the field. Reproducible training (pinned data versions, configs, seeds) is what makes it possible to tell whether a fleet-wide behavior change came from the new policy or from a silent data/config drift.
| Isaac Sim / Isaac Lab | MuJoCo | Gazebo | |
|---|---|---|---|
| Parallelism | Thousands of GPU envs | Single/CPU-parallel | Single instance, real-time-ish |
| Strength | Scale + photorealism | Fast, accurate contact dynamics | ROS2 + full sensor stack integration |
| Typical use | Large-scale RL training | Manipulation/locomotion research | Testing the real ROS2 stack in sim |