Skip to content

MCPTAM Roadmap

Jump to bottom
Steven Waslander edited this page Jul 27, 2016 · 11 revisions

MCPTAM Roadmap

We are building the next version of MCPTAM, and will focus on a few core improvements to help with future research project intergration. Foremost is a modularization of the code through top-down architecture design, so contributions can be made to every aspect of the code, or new packages from other sources can be easily integrated. Second is removal of the GPL licensing in favour of BSD, to aid adoption by our partners. Third is replacement of PTAM dependencies such as TooN, libCVD and gvars3 with more common tools such as OpenCV, eigen and rqt.

As this is an ambitious project for our lab, it needs an ambitious name: Project LaserTitan. To see the list of active development topics, go to [Laser Titan Activity list](LaserTitan Activity List).

Project LaserTitan will proceed in three phases:

LaserTitan 1.0

A nominal proof of concept phase with the major modifications included.

  • Fast feature detection and KLT matching from OpenCV
  • Cartesian and IDP feature parametrizations
  • EKF tracker which handles multi-camera measurement bundles
  • Additive pose/feature updates in SE(3)
  • Simple keyframe selection
  • GTSAM backend
  • Stock intrinsic calibration
  • Stereo extrinsic calibration
  • rqt for the gui
  • eigen for matrix math
  • Maintain tracker/mapper independence to run on separate computers
  • Identify and test options for parallelization (GPU usage)

LaserTitan 2.0

An expanded code base to include current research projects.

  • Extend EKF tracker to include multi-sensor measurement bundles
  • Add atan, Taylor camera models throughout (feature matching, calibration in particular)
  • Add non-overlapping cameras (calibration and tracking in particular)
  • Introduce gimballed cameras
  • Add sparse VO
  • Add dense tracking and depth estimation
  • Add Ceres and g2o if performance is better/different.

LaserTitan 3.0

The emphasis in the final stage of Project LaserTitan will be robustness.

  • Integrity monitoring of different sensor contributions to tracking
  • Outlier rejection and cost function robustification
  • Robust backend tuning
  • Tracking failure recovery
  • Mapping failure recovery

MCPTAM 1.0 Final Changelist

Simultaneously to Project LaserTitan, we will also continue to develop current research improvements for MCPTAM 1.0 as is. This will allow us to continue to publish while bringing new students up to speed on the newer codebase.

Top Priorities for current team

  • Integrate CPER keyframe selection as an option (for keyframe selection project)
  • Develop and integrate gimballed MCPTAM (for gimballed camera integration project)
  • Add the ability to change feature type using OpenCV feature detection
  • Add option to use GTSAM backend

Archived List of Future Improvements to be included in Project LaserTitan

Licensing Revisions
  • Switch to OpenCV and remove dependence on Gvars3 and libCVD
  • Switch to Eigen and remove dependence on TooN
  • Remove all PTAM code, requires front end rewrite.
  • Ensure all licenses are BSD or better
Front End Performance
Features
  • Ability to switch between feature types (SIFT, SURF, GPU-SURF, CENSURE, DAISY, ORB, all OpenCV options)
  • Improve use of features with poor depth, features at infinity
  • Robustify pruning and management of features
  • Add outlier rejection - parity/ransac on features
  • Point parametrizations
Cameras
  • Template the camera model so we can swap between Taylor, atan and pinhole.
  • Use sparse visual odometry too, not just features in the map
Tracker
  • Update map data during tracking, and maintain additional feature parameters to be used in backend
  • Tracker should be upgraded to EKF with interchangeable motion models.
  • Integrate multi-rate EKFs with delays
  • Integrate IMU and GPS into the tracker
  • Integrate continuous time SLAM
  • Integrate direct and dense visual odometry methods
  • Identify failures in translation tracking
  • Tracker integrity monitoring relative to other sensors
  • Active identification of loop closures, place recognition
Back end Performance
  • Keyframe selection methods
  • Integrate IMU/GPS measurements into bundle adjustments
  • Enable switch between g2o, GTSAM and CERES, using box+ manifolds.
  • Look at pose graph only optimizations
  • Dynamic covariance scaling and other robust costs
  • Local vs global bundle adjustments
  • Look at local maps and submapping instead of one big map.
  • Look at dense backend methods (DTAM, Sibley etc.)
Calibration
  • Improved robustness of calibration from cameras to IMU, GPS, gimbal, CG
  • Enhanced ability for field calibration (better instructions for existing tools, improve tools)
  • Automated calibration of GPS location, IMU location, altimeter, camera centers and CG of vehicle
  • Analyse sensitivity to errors in calibration parameters for all sensors
  • Calibration of gimbal axes of rotation and encoder values.
  • Automate system identification parameter optimization from flight data
  • Online calibration parameter updating
Initialization
  • Improve scale recovery during takeoff with non-overlap
  • Improve feature tracking and map updates in first seconds of flight
  • Consider heavy reliance on takeoff motion prior, IMU/GPS data should help significantly
  • Define best camera configurations for initialization
Processing platform customization
  • Parallelization of feature extraction
  • Parallelization of other parts of the code
  • Profiling on different hardware
  • TK1 experiments and other embedded platform experiments
Code refinements
  • Make code much more modular
  • Make code much better documented
  • Match coding practice for ROS release
  • Maintain wiki and instructional videos
  • Ensure many more parameters can be set in the launch file, and document all launch file parameters
Clone this wiki locally