Multi-Object Tracking

Each frame: predict every existing track forward with its own filter, associate new detections to predicted tracks, update matched tracks with their assigned detection, then manage the track lifecycle — birth a new track for unmatched detections, coast (predict without updating) briefly-unmatched tracks, and delete tracks that go too long without a match.

A track only considers detections inside its validation gate:

$d^2(k,j) = (z_j - h(x_k))^\top S_k^{-1} (z_j - h(x_k))$

Detections with $d^2$ above a chi-squared threshold are excluded before assignment — this prunes obviously-wrong pairings and keeps the association problem small, using either Mahalanobis distance or IoU as the cost.

Global nearest neighbor treats association as bipartite matching: build a track-by-detection cost matrix (Mahalanobis distance or $1 - IoU$), then the Hungarian algorithm finds the minimum-total-cost one-to-one assignment in $O(n^3)$.

It's optimal for a single hard assignment per frame, but commits early — it can't recover from an assignment that looks right now but is wrong once more frames arrive.

Each track runs its own Kalman filter, typically a constant-velocity or constant-acceleration model on position and velocity (and sometimes size/orientation). For targets that switch between motion patterns (cruising vs turning), an IMM blends several motion models' estimates weighted by how well each currently fits.

Occlusion: a track goes unmatched for several frames — coasting on prediction alone risks losing it, especially under a poor motion model.

Clutter / false positives: spurious detections can hijack an assignment or spawn a phantom track.

ID switches: two crossing tracks swap identities when the association picks the wrong (but locally cheaper) pairing — the leading cause of tracking errors near occlusions and crossings.

SORT associates using only motion (IoU between predicted and detected boxes) — fast, but fragile through occlusion. DeepSORT adds an appearance embedding to the cost, so a track can re-identify its object even after a gap where motion alone would be ambiguous.

Evaluation uses MOTA (accuracy — penalizes misses, false positives, and ID switches), MOTP (localization precision of matched boxes), and IDF1 (how consistently identity is preserved over the whole track, not just per-frame).