3D Computer Vision in Python: A Nutshell Guide with Code & History

Welcome to the World of 3D Vision!

Ever wondered how robots see the world in 3D? Or how your phone knows your face is your face and not a poorly drawn stick figure? Welcome to 3D Computer Vision—where math, magic, and Python code come together to give computers eyeballs.

A Quick History Lesson: From Pinhole Cameras to AI Eyes

The Ancient Beginnings (a.k.a. “Ye Olde Optics”)

Before computers, humans tried to understand depth perception with pinhole cameras and stereoscopic imaging. The idea? If we have two eyes to perceive depth, why not give cameras the same ability?

Leonardo da Vinci, around the 1500s, was already thinking about how humans perceive depth. But, of course, Python wasn’t around yet (sad times).

The 20th Century: Math Enters the Chat

Fast forward to the 20th century, and we get all the fancy math that drives modern 3D vision: epipolar geometry, stereo disparity, and structure-from-motion. Researchers started using computers to reconstruct 3D shapes from 2D images. This was also the era when robots stopped bumping into walls (well, most of the time).

The AI Boom: Deep Learning Takes Over

Enter deep learning and neural networks. Instead of manually designing feature detectors, researchers trained AI models to infer depth, recognize objects, and even hallucinate 3D scenes from a single 2D image. Today, 3D vision is everywhere—from self-driving cars to augmented reality filters that put dog ears on your selfies.

Applications of 3D Computer Vision

• Self-Driving Cars – LiDAR, stereo cameras, and depth estimation help cars “see” the road.
• Medical Imaging – 3D scans from CT and MRI help doctors visualize organs in 3D.
• Augmented Reality (AR) – Apps like Snapchat and Pokémon GO use 3D vision for cool effects.
• Robotics – Robots use 3D vision to avoid obstacles and grab objects.
• 3D Reconstruction – Convert 2D photos into full 3D models (like photogrammetry software does).

Let’s Get Coding: 3D Vision in Python

1. Setting Up the Environment

We’ll use OpenCV and NumPy for this adventure. Install them with:

1

pip install opencv-python numpy matplotlib

2. Depth Estimation with Stereo Vision

Let’s create a simple stereo vision depth estimator using OpenCV.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

import cv2
import numpy as np

# Load left and right images (example stereo pair)
left_img = cv2.imread("left_image.jpg", cv2.IMREAD_GRAYSCALE)
right_img = cv2.imread("right_image.jpg", cv2.IMREAD_GRAYSCALE)

# Create stereo block matcher
stereo = cv2.StereoBM_create(numDisparities=16, blockSize=15)
depth_map = stereo.compute(left_img, right_img)

# Normalize and display
depth_map = cv2.normalize(depth_map, None, 0, 255, cv2.NORM_MINMAX)
cv2.imshow("Depth Map", depth_map.astype(np.uint8))
cv2.waitKey(0)
cv2.destroyAllWindows()

What’s happening here? The stereo block matcher compares the left and right images, finds corresponding pixels, and estimates depth.

3. 3D Point Cloud from Depth Data

Want to reconstruct a 3D scene? Let’s generate a point cloud from a depth map.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import numpy as np
import open3d as o3d

def generate_point_cloud(depth_map):
    h, w = depth_map.shape
    fx, fy = 500, 500  # Camera focal lengths
    cx, cy = w // 2, h // 2
    
    points = []
    for v in range(h):
        for u in range(w):
            z = depth_map[v, u] / 255.0  # Normalize depth
            x = (u - cx) * z / fx
            y = (v - cy) * z / fy
            points.append([x, y, z])
    
    return np.array(points)

# Generate point cloud
depth_map = np.random.randint(0, 255, (480, 640), dtype=np.uint8)
point_cloud = generate_point_cloud(depth_map)

# Visualize
pcd = o3d.geometry.PointCloud()
pcd.points = o3d.utility.Vector3dVector(point_cloud)
o3d.visualization.draw_geometries([pcd])

This script converts a 2D depth map into a 3D point cloud. Try replacing depth_map with real data for cooler results.

Key Ideas

References

• OpenCV Docs: https://docs.opencv.org/
• Open3D Library: http://www.open3d.org/
• 3D Vision in Robotics: https://robotics.stackexchange.com/