What is Computer Vision in Robotics?

Robots are inherently blind. While a human can look at a messy room and instantly identify a chair, a ball, and a door, a robot’s camera only sees a grid of millions of colored pixels.

Computer Vision (CV) is the field of software engineering dedicated to teaching computers how to extract meaning from those pixels. It is the core technology behind self-driving cars, facial recognition, and modern autonomous robotics.

How Do Robots See objects?

Traditionally, Computer Vision relies on math-heavy pipelines to filter images. If a robot is looking for a bright yellow tennis ball, the software will:

1. Filter by Color: Delete any pixel that isn’t bright yellow.
2. Find Contours: Draw geometric outlines around the remaining blobs of yellow pixels.
3. Check Shapes: Ignore rectangular blobs and only keep perfect circles.
4. Calculate Position: Find the center coordinate of the circle and drive towards it.

Libraries like OpenCV make these operations accessible, allowing developers to write complex visual processing logic without building the calculus from scratch.

The Rise of Machine Learning and AprilTags

Today, vision systems are becoming even smarter. Instead of hard-coding color filters, engineers use Machine Learning models (like TensorFlow) that are “trained” on thousands of photos to recognize complex objects, like pedestrians or street signs, regardless of lighting conditions.

In industrial settings, robots also frequently rely on fiducial markers like QR Codes or AprilTags. These high-contrast square targets are mathematically designed to be instantly recognized by cameras, providing the robot with its exact distance and 3D angle relative to the tag.

Computer Vision in FIRST® Robotics

In the FIRST® Tech Challenge (FTC), autonomous computer vision is heavily utilized. At the start of a match, the robot must use a webcam to “look” at randomized game elements and instantly decide which pre-programmed route to take.

Teams write OpenCV pipelines or leverage built-in TensorFlow models to track customized game pieces, locate targets on the field, and dynamically adjust their pathing. For high school students, there is nothing more satisfying than watching a robot autonomously identify a target, line up perfectly, and score!