Robotics
← Roadmap/Perception

Computer Vision

Camera models, features, geometry — robots that see.

Progress0/17 questions

Prerequisites

Why it matters in robotics

Most robots perceive the world through cameras, and computer vision is the bridge from pixels to a metric 3-D world model. Camera calibration, projection, and triangulation turn images into geometry, while matching the same points across images, discarding bad matches, and recovering camera motion from two or more views underpin visual odometry, mapping from images, and visual SLAM. The classical geometry beneath these pipelines is what makes modern perception robust rather than a black box.

What to study

  • Camera models: pinhole/perspective projection, intrinsic & extrinsic matrices, lens distortion, and calibration
  • Features and matching: detectors/descriptors (SIFT, ORB), correspondence, and RANSAC for robust outlier rejection
  • Two-view geometry: epipolar constraint, fundamental & essential matrices, pose recovery, and triangulation
  • Structure from Motion & visual SLAM: bundle adjustment, multi-view reconstruction, and visual odometry

Where to study

Cheatsheet

Symbols used below

fffocal length
ZZdepth (distance along the camera's optical axis)
BBstereo baseline (distance between the two camera centers)
dddisparity (pixel shift of a point between left/right images)
KKcamera intrinsic matrix
R,tR, textrinsic rotation and translation (camera pose)
x,xx, x'a point's pixel coordinates in image 1 and image 2 (homogeneous)
FFfundamental matrix (uncalibrated two-view geometry)
EEessential matrix (calibrated two-view geometry)
HHhomography (planar or rotation-only mapping between two views)

Pinhole projection

A 3-D point at depth ZZ projects to the image plane by similar triangles: x=fX/Zx = fX/Z, y=fY/Zy = fY/Z.

In pixels this is packaged as x~=K[Rt]X\tilde{x} = K [R \mid t] X, where KK holds focal length and principal point, and [Rt][R \mid t] places the point in the camera frame.

Stereo depth from disparity

Z=fBdZ = \dfrac{f B}{d}

Closer points have larger disparity; depth resolution degrades quadratically with distance since ZZ is inversely proportional to dd.

Epipolar constraint

For a correspondence (x,x)(x, x') between two views: xFx=0x'^\top F x = 0.

This says xx' must lie on the epipolar line l=Fxl' = Fx — it collapses a 2-D search for correspondences into a 1-D search along a line. E=KFKE = K'^\top F K relates the calibrated and uncalibrated forms.

RANSAC for robust fitting

Repeatedly: sample a minimal set of correspondences, fit a model (FF, EE, or HH), count inliers within a threshold, keep the best-scoring model.

The number of trials needed grows sharply as the outlier fraction rises, which is why a good initial match filter (e.g. descriptor ratio test) matters before RANSAC runs.

Homography vs epipolar geometry

xHxx' \simeq H x holds when points lie on a common plane, or when the camera only rotates between views (no baseline, no parallax).

General 3-D scenes with camera translation need the epipolar constraint instead — a single HH can't explain parallax.

Triangulation

Given two calibrated views and a matched pixel pair, the 3-D point is the intersection of the two back-projected rays.

In practice the rays rarely intersect exactly due to noise, so triangulation finds the point minimizing reprojection error in both images (linear least squares or iterative refinement).

object (H)pinholeimage (h)Zfh / f = H / Z
The pinhole model: rays through a single center invert the scene onto the image plane. Similar triangles give h/f = H/Z.
left imageright imagexx′ (match)epipolar line l′ = F xOₗOᵣbaselinex′ᵀ F x = 0
Epipolar geometry: a point x in one image must lie on the line l′ = F x in the other. The constraint x′ᵀF x = 0 turns matching into a 1-D search.

Application focus

Select an application above.

Practice questions (17)