Imitation Learning

Supervised learning on expert (state, action) pairs: minimize $\mathbb{E}_{s \sim \pi^*}\big[\, \ell(\pi_\theta(s), \pi^*(s))\big]$.

It only sees states the expert visits. Once $\pi_\theta$ makes a small error and drifts to a state the expert rarely visited, it has no training signal there — errors compound and worst-case regret grows like $O(\epsilon T^2)$ with the horizon.

The training distribution (expert's state visitation) and the test-time distribution (the learned policy's own state visitation) diverge as soon as $\pi_\theta \ne \pi^*$ — this mismatch, not a lack of model capacity, is the core failure mode of naive BC.

Interactive imitation breaks the mismatch: roll out the current $\pi_\theta$, query the expert for the correct action on the states it actually visits, aggregate those labels into the dataset, and retrain.

Because training data now comes from the learner's own distribution, regret grows linearly in $T$ instead of quadratically — at the cost of needing a queryable expert online rather than a fixed offline dataset.

Predicting a short sequence of future actions per inference step (instead of one action at a time) and blending overlapping chunks at execution time (temporal ensembling) smooths the trajectory and reduces how often compounding error gets a chance to accumulate, without needing an online expert.

A single instant can have more than one reasonable next move (go left or right around an obstacle). Averaging those modes under an MSE loss produces an invalid action in between them.

Diffusion and flow-matching policies model the full action distribution instead of its mean, so they can represent and sample from genuinely multimodal behavior.

Real-robot demonstrations are usually gathered via teleoperation (joystick, VR controller, or a kinesthetic/leader-follower rig). Teleop data quality — smoothness, coverage of recovery behavior, consistency across operators — directly bounds how good any imitation-learned policy can be.