Computer Vision Examples

Code examples for the Computer Vision subject of the Robotics Software Engineering Degree at URJC, using C++, OpenCV, and the Point Cloud Library (PCL).

First, set the environment variable so examples can find the data folder. Run this from the repository root:

echo "export OPENCV_SAMPLES_DATA_PATH=$(pwd)/data/" >> ~/.bashrc
source ~/.bashrc

Note: The examples use modern features and require a compiler that supports C++17 or higher.

Note: Developed against OpenCV 4.6.0 and PCL 1.14.0. The build requires OpenCV 4 and at least PCL 1.10, which is what the top-level CMakeLists.txt asks for.

Note: Some examples require the opencv_contrib modules (ximgproc, aruco, surface_matching, viz). If you installed OpenCV from the distribution package these are usually included; if you built OpenCV from source, follow the Installation from source section below and pass OPENCV_EXTRA_MODULES_PATH. The examples that need them are: 11_05_skeletonization, 14_02_pose_estimation, 15_02_stereo_disparity and 15_04_opencv_icp.

A top-level CMakeLists.txt compiles every example in one step and places all executables in the vision_examples/bin/ folder, named after their source directory:

cmake -B vision_examples/build
cmake --build vision_examples/build

Executables are in vision_examples/bin/. For example:

./vision_examples/bin/03_01_read_image
./vision_examples/bin/06_01_dft_frequencies
./vision_examples/bin/15_02_stereo_disparity

Note: the default inputs are written as ../../data/..., which resolves to the data/ folder of the repository both from vision_examples/bin/ and from the folder of the example itself. Either way of building works without touching the paths.

From the repository root, every example follows the same convention, so any of them can be run without reading its source first:

./vision_examples/bin/example                 # runs with its default input, taken from data/
./vision_examples/bin/example my_image.jpg    # overrides the input
./vision_examples/bin/example --help          # prints what the example accepts and its defaults

OpenCV examples use cv::CommandLineParser; PCL examples use PCL's own pcl::console parser. Both accept -h and --help. Inputs are positional and optional: an example with no arguments always works. The few examples that write a file take the destination on the command line and default to the current directory: --out in 13_01, 13_03, 14_03 and 16_02, and --dst_path (plus --dst_raw_path and --dst_conf_path) in 14_02.

Each OpenCV example also has its own Makefile. The executable takes the name of its folder, exactly like the one the top-level build produces, so both ways of compiling give the same binary:

cd 08_edge_detection/08_02_canny_edges
make
./08_02_canny_edges

Each PCL example has its own CMakeLists.txt:

cd 15_3d_and_point_clouds/15_09_pcl_icp
cmake -B build
cmake --build build
./build/15_09_pcl_icp

The Chapter 19 examples live here like every other chapter, under 19_vision_ros2/, but they are not built with the rest. They are ROS 2 packages, not standalone programs: they need a workspace, they are built with colcon and they are run with ros2 run. The top-level CMakeLists.txt ignores that folder on purpose, so the repository still builds for anyone who only wants the OpenCV examples and has no ROS 2 installed.

There are five packages: opencv_demo, transport_demo, sync_demo and pcl_demo, one per piece developed in the chapter, plus launch_demo, which holds the launch file that chains the depth_image_proc nodes to produce the cloud pcl_demo consumes.

cd <repository root>
rosdep install --from-paths 19_vision_ros2 --ignore-src -r -y
colcon build --base-paths 19_vision_ros2 --symlink-install
source install/setup.bash

--base-paths is what keeps colcon from descending into the rest of the repository, where it would find the OpenCV/PCL project of the other chapters.

Requirements, beyond a current ROS 2 distribution: cv_bridge, image_transport (plus image-transport-plugins), message_filters, pcl_ros and depth_image_proc. rosdep installs them from the manifests, or by hand:

sudo apt install ros-${ROS_DISTRO}-cv-bridge \
                 ros-${ROS_DISTRO}-image-transport \
                 ros-${ROS_DISTRO}-image-transport-plugins \
                 ros-${ROS_DISTRO}-message-filters \
                 ros-${ROS_DISTRO}-pcl-ros \
                 ros-${ROS_DISTRO}-depth-image-proc

Every node works the same against a live camera or against a recording:

ros2 bag record /color/image /color/camera_info /stereo/depth -o session
ros2 bag play session

Things worth trying:

• opencv_demo: ask toCvCopy for BGR8 on a depth topic and watch the cv_bridge exception; then ask for BGR8 on an rgb8 camera and notice that nothing breaks, because cv_bridge converts.
• transport_demo: compare ros2 topic bw /image_processed with ros2 topic bw /image_processed/compressed. The saving depends on the scene, not on the format in the abstract.
• sync_demo: drop the queue size to 1 and count how many pairs are lost; switch the policy to ExactTime and watch the callback stop firing unless the cameras share a hardware trigger.
• pcl_demo: drop a VoxelGrid filter between the two conversions and compare ros2 topic hz on input and output.

A note on QoS: these nodes use plain rclcpp::SensorDataQoS(), which is best effort, on both ends. Forcing it to .reliable() on the subscriber makes it incompatible with any publisher that offers best effort, which is what most camera drivers do, and there is no error message when that happens: the topic is listed, ros2 topic hz reports data, and the callback simply never runs. Check the actual profiles with:

ros2 topic info /color/image --verbose

The examples default to the same photographs the book uses in its figures, so running one without arguments reproduces what the reader has just seen printed. data/building_facade.png, coins.png, chess.png, smarties.png, aerial_view.png, starry_night.png and futbol.png are the very files that the figure-generating scripts of the book read.

The optical flow examples of chapter 16 default to the same video, the overhead shot of a busy square that the book credits to Pexels 853889.

Those photographs are around 1400 px on the long side, and the video is Full HD, which does not fit on a normal screen once an example opens four or five windows. Every example that uses them reduces only what it sends to the screen, with INTER_AREA and a long side of 800 px; the processing always runs at full resolution. The reduction is a no-op on smaller inputs, so passing your own image changes nothing.

Text is written on the reduced copy, or with the font raised by the same factor when it is a label anchored to a region, so that it stays readable instead of shrinking with the picture.

The one exception to processing at full resolution is 16_03_dense_flow: Farneback costs 392 ms per frame at 1920x1080, ten times the 40 ms a 25 fps video allows, so it reduces the frames by --scale (0.5 by default, the same factor the book uses for its figures) before computing the flow. Pass --scale=1.0 to see the difference.

tools/check_repo.py verifies the things that drift when a chapter is renamed or an example moves: that every example on disk is built by the top-level CMakeLists.txt, that each one produces a binary named after its folder whichever way it is compiled, that no header cites an executable or an example that does not exist, that the default input paths point at files that are really there, and that every example answers -h and --help.

python3 tools/check_repo.py

It exits non-zero on the first inconsistency, so it can be used in CI.

The examples are organised by chapter and follow the order in which the book introduces the material. There are 80 in total: 75 numbered NN_MM examples, where NN is the book chapter, plus the 5 ROS 2 packages of chapter 19, which are named after the package instead of numbered because colcon builds them by name. The folder column below is the authoritative mapping between a book chapter and its code.

Every example is self-contained and runnable on its own: they can be run in any order and none of them needs another to have run first. Two of them are linked on purpose, and neither link is required:

• 15_03_stereo_to_pointcloud accepts --calib=stereo_calibration.yml, the file that 14_03_stereo_calibration writes. With it the pair is rectified and the cloud comes out in real units; without it the example falls back to an assumed rig and says so.
• 15_05_pcl_write writes the test_pcd.pcd that 15_06_pcl_read reads. That file is kept under version control, so 14_06 also works on a fresh clone. The generator of 14_05 is seeded, so running it rewrites the file byte for byte instead of producing a spurious change.

What does follow the book order is the material each one assumes you have already read, which is the reason for studying them from beginning to end.

OpenCV:

sudo apt update
sudo apt install libopencv-dev

Verify:

pkg-config --modversion opencv4

Every example but one runs on OpenCV 4.6, the version the Ubuntu 22.04 package installs. The exception is 17_02: its YOLO11 model in ONNX needs OpenCV 4.9 or newer, because earlier ONNX readers do not understand the Split node the way YOLO11 writes it. With an older OpenCV the example reports exactly that and exits, and 17_01 covers the same ground with a model that loads anywhere.

PCL:

sudo apt update
sudo apt install libpcl-dev

Verify:

dpkg -s libpcl-dev | grep Version

If you have ROS installed, OpenCV and PCL are likely already available through your ROS distribution.

Deep learning models (chapter 18, optional):

18_02_yolo_ultralytics and 18_03_semantic_segmentation export their ONNX model on first build via a download_model.sh/export_model.py pair. If the required Python packages aren't installed, the script prints a warning and skips the export instead of failing the whole build.

pip install --user --break-system-packages ultralytics onnx onnxruntime onnxslim  # 18_02
pip install --user --break-system-packages torch torchvision onnxscript           # 18_03

Re-run bash download_model.sh inside each example's folder (or rebuild its target) afterwards to generate the missing model files.

sudo apt install build-essential cmake git pkg-config libgtk-3-dev \
    libavcodec-dev libavformat-dev libswscale-dev libv4l-dev \
    libxvidcore-dev libx264-dev libjpeg-dev libpng-dev libtiff-dev \
    gfortran openexr libatlas-base-dev python3-dev python3-numpy \
    libtbb-dev libdc1394-dev

mkdir ~/opencv_build && cd ~/opencv_build
git clone https://github.com/opencv/opencv.git
git clone https://github.com/opencv/opencv_contrib.git

cd ~/opencv_build/opencv
mkdir build && cd build

cmake -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D INSTALL_C_EXAMPLES=ON \
    -D INSTALL_PYTHON_EXAMPLES=ON \
    -D OPENCV_GENERATE_PKGCONFIG=ON \
    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_build/opencv_contrib/modules \
    -D BUILD_EXAMPLES=ON \
    -D OPENCV_ENABLE_NONFREE=ON \
    ..

cmake -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D INSTALL_C_EXAMPLES=ON \
    -D INSTALL_PYTHON_EXAMPLES=ON \
    -D OPENCV_GENERATE_PKGCONFIG=ON \
    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_build/opencv_contrib/modules \
    -D BUILD_EXAMPLES=ON \
    -D OPENCV_ENABLE_NONFREE=ON \
    -D WITH_CUDA=ON \
    -D OPENCV_DNN_CUDA=ON \
    -D WITH_CUDNN=ON \
    -D ENABLE_FAST_MATH=1 \
    -D CUDA_FAST_MATH=1 \
    -D CUDA_ARCH_BIN=8.6 \
    -D WITH_CUBLAS=1 \
    ..

make -j$(nproc)
sudo make install
sudo ldconfig

echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/' >> ~/.bashrc
source ~/.bashrc

mkdir ~/pcl_build && cd ~/pcl_build
git clone --recursive https://github.com/PointCloudLibrary/pcl.git

cd ~/pcl_build/pcl
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j$(nproc)
sudo make install
sudo ldconfig

fatal error: opencv2/opencv.hpp: No such file or directory

OpenCV headers are not found. Make sure libopencv-dev is installed. If installed from source, add the pkg-config path:

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH

error while loading shared libraries: libopencv_core.so.X

The dynamic linker cannot find the OpenCV libraries. If you installed from source to a custom path, make sure it is in LD_LIBRARY_PATH. For standard source installations (/usr/local), simply update the linker cache:

sudo ldconfig

pkg-config: command not found

sudo apt install pkg-config

VideoCapture does not open the camera

1. Try different indices (0, 1, ...).
2. Add your user to the video group and log out/in:

   sudo usermod -aG video $USER

3. Make sure the v4l2 module is loaded:

   sudo modprobe v4l2

PCL visualizer window does not open or crashes

This is usually a VTK/OpenGL issue:

1. Install or update VTK: sudo apt install libvtk9-dev
2. On virtual machines or remote sessions, force software rendering:

   export LIBGL_ALWAYS_SOFTWARE=1

3. Compiling PCL from source (instead of packages) often resolves VTK 9.x compatibility issues.

CMake cannot find PCL (Could not find PCL)

Make sure libpcl-dev is installed. If installed from source, hint CMake to the correct path from inside your build folder:

mkdir build && cd build
cmake -DPCL_DIR=/usr/local/share/pcl-<version> ..

CUDA not detected when building OpenCV from source

Make sure nvcc is in your PATH and the CUDA libraries are accessible:

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

Also verify CUDA_ARCH_BIN matches your GPU's compute capability at https://developer.nvidia.com/cuda-gpus.

This project was made by Jose Miguel Guerrero, Associate Professor at Universidad Rey Juan Carlos.

Copyright © 2020-2026.

This work is licensed under the terms of the MIT license.