Ever since I first learned about computer vision at university I've been the most intrigued about this topic. I think because vision is one of the most basic interfaces with the physical world that you can have without abstractions. Also it's very relatable to our main sense, which is vision through our eyes.
At the time of writing this, I've completed my studies and worked with cameras and worked on vision models. But I feel like I only work in my own layer of abstraction, I develop models, train them and deploy them. Never thinking again about what all makes this possible.
This is my escape attempt, I will attempt to walk through how to create a computer vision model from the starting point of a light sensor...
Thinking about it again, a light sensor might be the wrong starting point. We need to first understand what light rays are and how we can perceive different colours. Then we can look at how cameras work.
The pinhole camera
Let's get back to basics. We all learned about pinhole cameras in high school, but let's revisit:
Because there is only a single point for light rays to enter, the light rays will form an image on the wall opposite the hole inside the box. Notice how the image is inverted — light from the top of the object travels through the pinhole and hits the bottom of the image plane, and vice versa.
Try it yourself: Drag the sliders or the colored handles in the diagram to see how changing distances affects the image!
What the projected image looks like depends on the distance from the hole to the inner box wall, also referred to as the image plane:
- Image plane close to pinhole: Objects appear smaller, but you get a wider field of view
- Image plane far from pinhole: Objects appear larger (more magnification), but narrower field of view
The relationship is captured by the magnification formula:
$$M = \frac{d_i}{d_o}$$
Where $d_i$ is the image distance (pinhole to image plane) and $d_o$ is the object distance (object to pinhole). When magnification is less than 1, the image is smaller than the object. When greater than 1, the image is larger.
The problem with pinholes
Pinhole cameras have a fundamental limitation: the pinhole must be extremely small to produce a sharp image. A larger opening lets in more light (brighter image), but the rays from each point on the object spread out, creating a blurry image.
This is where lenses come in.
Adding a convergent lens
A convergent (convex) lens solves this problem by bending light rays so they converge to a single point. This allows us to have a larger aperture (more light) while still maintaining a sharp, focused image.
Toggle the lens to see the difference:
- Without lens: Light rays from each point on the object pass through different parts of the aperture and hit different spots on the image plane — resulting in blur.
- With lens: The lens refracts (bends) the light rays so they all converge to a single point on the image plane — producing a sharp image.
The lens follows the thin lens equation:
$$\frac{1}{f} = \frac{1}{d_o} + \frac{1}{d_i}$$
Where $f$ is the focal length of the lens. The focal length determines how strongly the lens bends light — a shorter focal length means more bending power.