FaceID & Filters: Computer Vision 101

You hold up your phone. A 3D mesh of lasers scans your face. The phone unlocks. You open Instagram. A filter smooths your skin and adds sunglasses. To you, it’s an image. To a computer, it’s a grid of numbers.

Seeing in Grids

A digital image is just a matrix of pixels.

• Each pixel has a value: R (Red), G (Green), B (Blue).
• Your face is just a cluster of pixels where “Skin Color” meets “Background Color.”

Feature Extraction (Finding the Face)

How does Snapchat know where your nose is? It uses Feature Detection.

1. Haar Cascades (The Old Way): It looks for patterns of light and dark.
   • “Eyes are darker than foreheads.”
   • “Nose bridges are lighter than cheeks.”
   • If it finds these rectangles in the right order, it says “Face Detected.”
2. Landmarks: Once the face is found, it finds 68 specific points (tip of nose, corner of mouth).
3. The Mesh: It draws a 3D wireframe connecting these points.
4. The Filter: It textures the wireframe with the “Dog Ears” graphic. If you turn your head, the wireframe rotates, and the ears move with you.

FaceID (The 3D Map)

Apple’s FaceID is more advanced. It doesn’t just use a camera; it uses a Dot Projector.

• It blasts 30,000 invisible Infrared dots onto your face.
• An IR Camera reads the pattern.
• If the dots on your nose look big (close) and the dots on your ears look small (far), it builds a topographical map.
• This is why photographs don’t unlock FaceID—they are flat.

Robotics: Detecting Game Pieces

We use the same tech.

• We train a Neural Network (using tools like TensorFlow or YOLO - You Only Look Once).
• Instead of “Faces,” we train it on “Yellow Hexagons.”
• The robot scans the field.
  • “I see a Hexagon.”
  • “Confidence: 98%.”
  • “Coordinates: (X,Y).”
• The robot drives to it. Snapchat filters seem silly, but they rely on the same heavy math that allows autonomous drones to track targets.