Skip to content
C++ Structure from Motion (SfM) pipeline with OpenGL visualization for Apolloscape Dataset
C++ HTML RPC COBOL C Python Other
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.vscode concurrent stuff Feb 11, 2019
data
include/cv_gl Added missing includes - <memory> and <functional> for g++ 8.3.0 (not… Jun 13, 2019
lib Added seerialization basics Jan 29, 2019
results Added visual gif Mar 1, 2019
shaders
src Added missing includes - <memory> and <functional> for g++ 8.3.0 (not… Jun 13, 2019
tests Rename and moved main.cpp to apps Feb 26, 2019
.gitignore Added missing includes - <memory> and <functional> for g++ 8.3.0 (not… Jun 13, 2019
3d_ones.sh Added readme and some params Mar 1, 2019
CMakeLists.txt Adding gflags Feb 14, 2019
README.md Added link to the blog post Mar 13, 2019
cv.code-workspace

README.md

SfM with OpenGL visualization on Apolloscape Dataset

3D reconstruction and Structure from Motion (SfM) are essential parts of many algorithms of today's research in Visual Odometry, SLAM and localization tasks. In this project, I've built the sparse 3D reconstruction from known poses implementing SfM pipeline with bundle adjustment optimization on C++ and used pure OpenGL for visualization.

3D Reconstruction Fly

More details available in the blog post https://capsulesbot.com/blog/2019/03/12/apolloscape-sfm.html

Previously, I've explored the Apolloscape dataset in the project for localization task via building Pytorch reader and training PoseNet to directly regress the car pose.

Dependencies & Build

Build dependencies:

  • Boost 1.59 Filesystem
  • OpenCV 3.4
  • Ceres Solver 1.14_04
  • gflags 2.2.2

Build commands:

mkdir build
cd build
cmake ..
make

Quick Run

Quick test of visualization on stored tiny reconstruction of just 10 images:

# from ./build directory
./bin/3d_recon --restore=../results/sfm_out_sample.bin

Control Navigation:

  • W,A,S,D - move camera around
  • Z,X - camera texture transparency
  • J,K,L - change camera forward, backward, lateral
  • 0 - return to the last camera (useful when you fly out and want to return)
  • ESC - close visualizer and store SfM Map to a file

Data Folder

Download Apolloscape ZPark dataset from its official page and unpack it into a folder. Then symbolically link it to ./data/apolloscape from the project root directory:

ln -s <DATA FOLDER>/apolloscape ./data

ZPark sample dataset should appear in the path ./data/apolloscape/zpark-sample as a result.

Run 3D Reconstruction

The main file to run is ./bin/3d_recon from ./build folder. Here is an example to reconstruct only first sequence alone with the simultaneous 3D visualization:

./bin/3d_recon --records="1"

Or the full ZPark sample dataset for all 14 records without visualization (will take some time):

make && ./bin/3d_recon --records="1,2,3,4,6,7,8,9,10,11,12,13,14" \
--pairs_look_back=2 --matches_num_thresh=60 --matches_line_dist_thresh=10.0 \
--sfm_repr_error_thresh=10.0 --sfm_max_merge_dist=5.0 --noviz \
--output=sfm_out_all_r1_14.bin

Cache

Extracted features and matched pairs of keypoints with descriptors are stored in a cache folder ./build/_features_cache so subsequent runs that do not introduce new image pairs are using pre-calculated values stored earlier. Its speed up my tests iterations dramatically.

Serialization

Cached featured and the final map serialization is implemented with Cereal. So the quickest way to see something is to restore the previous map. In the results folder there is sfm_out_sample.bin map that can be viewed without reconstruction step as:

cd build
./bin/3d_recon --restore=../results/sfm_out_sample.bin
You can’t perform that action at this time.