CV_DL_Reading_List

A reading list related to our current project in SenseTime

• Section Links:
  • Learning Iterative Optimization Solver for SLAM
  • Learning Iterative Network for SLAM
  • RNN for Motion Estimation
  • CNN for Depth Estimation
  • Learning based Visual-Inertial Odometry
  • Depth Estimation from Partial Observation
  • Visual SLAM without Learning
  • Adaptive Frame Selection from Video
  • Robotic Manipulation and Perception
  • Other Interesting Papers

Learning Iterative Optimization Solver for SLAM

• Taking a Deeper Look at the Inverse Compositional Algorithm, CVPR 2019

  • paper, code
  • two-view feature encoder + convolutional M-estimator + trust region network, supervised, motion only

• BA-Net: Dense Bundle Adjustment Networks, ICLR 2019

  • paper, code
  • learning the feature pyramid + the LM damping factor + the basis depth maps, supervised, motion and depth

• SceneCode: Monocular Dense Semantic Reconstruction using Learned Encoded Scene Representation, CVPR 2019

  • paper
  • compact code for optimization, supervised, motion and segmentation and depth

• CodeSLAM-Learning a Compact, Optimisable Representation for Dense Visual SLAM, CVPR 2018

  • paper
  • compact code for optimization, supervised, motion and depth

Learning Iterative Network for SLAM

• Learning to Solve Nonlinear Least Squares for Monocular Stereo, ECCV 2018

  • paper
  • LSTM-RNN for GN solver updates prediction (consider jacobian and residual terms), supervised, motion and depth

• DeepTAM: Deep Tracking and Mapping, ECCV 2018

  • paper
  • TODO

• DeMon: Depth and Motion Network for Learning Monocular Stereo, CVPR 2017

  • paper
  • bootstrap net + iterative net (CNN, DOES NOT consider jacobian and residual terms), supervised, motion and depth and flow

RNN for Motion Estimation

• Beyond Tracking: Selecting Memory and Refining Poses for Deep Visual Odometry, CVPR 2019

  • paper
  • tracking + remembering + refining, memory augmented LSTM-RNN, supervised, motion only

• End-to-end, sequence-to-sequence probabilistic visual odometry through deep neural networks, IJRR 2017

  • paper
  • LSTM-RNN, supervised, motion only

CNN for Depth Estimation

• Digging Into Self-Supervised Monocular Depth Estimation, ICCV 2019

  • paper, code
  • per-pixel minimum reprojection loss, auto-masking stationary pixels, full-resolution multi-scale, self-supervised

• Neural RGB->D Sensing: Depth and Uncertainty from a Video Camera, CVPR 2019

  • paper, code
  • depth probability distribution + bayesian filters + adaptive damping + kalman filter, DPV fusion, supervised, motion and depth

• Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation, CVPR 2019

  • paper, code
  • TODO

• Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras, arXiv 2019

  • paper
  • unsupervised, motion and depth and camera intrinsics and occlusion

• Learning Depth from Monocular Videos using Direct Methods, CVPR 2018

  • paper, code
  • differentiable DVO, unsupervised, motion and depth

• GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose, CVPR 2018

  • paper, code
  • unsupervised, motion and depth and flow

• Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints, CVPR 2018

  • paper, code
  • unsupervised, 2D photometric + 3D ICP losses, motion and depth

• Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction, CVPR 2018

  • paper, code
  • unsupervised, stereo training, deep feature reconstruction loss, motion and depth (without scale ambiguity)

• Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry, ECCV 2018

  • paper
  • TODO

• UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learning, ICRA 2018

  • paper
  • unsupervised, stereo training, scale recovery, motion and depth (without scale ambiguity)

• Unsupervised Learning of Depth and Ego-Motion from Video, CVPR 2017

  • paper, code
  • unsupervised, monocular training, 2D photometric loss, motion and depth

• Unsupervised Monocular Depth Estimation With Left-Right Consistency, CVPR 2017

  • paper, code
  • unsupervised, stereo training, appearance matching & left-right disparity consistency loss, depth only

Learning based Visual-Inertial Odometry

• Selective Sensor Fusion for Neural Visual-Inertial Odometry, CVPR 2019

  • paper
  • visual-inertial fusion, LSTM-RNN, supervised, motion only

• Unsupervised Deep Visual-Inertial Odometry with Online Error Correction for RGB-D Imagery, TPAMI 2019

  • paper
  • iterative CNN, consider camera-imu synchronization errors, unsupervised

• VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem, AAAI 2017

  • paper
  • IMU-LSTM + Core-LSTM, consider camera-IMU calibration & synchronization Errors, supervised

Depth Estimation from Partial Observation

• Estimating Depth from RGB and Sparse Sensing, ECCV 2018

  • paper, code
  • TODO

• Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image, ICRA 2018

  • paper, code
  • TODO

• Sparse Geometry from a Line: Monocular Depth Estimation with Partial Laser Observation, ICRA 2017

  • paper

Visual SLAM without Learning

• BAD SLAM: Bundle Adjusted Direct RGB-D SLAM, CVPR 2019

  • paper
  • TODO

• ICE-BA: Incremental, Consistent and Efficient Bundle Adjustment for Visual-Inertial SLAM, CVPR 2018

  • paper, code
  • TODO

• Hybrid Camera Pose Estimation, CVPR 2018

  • paper
  • TODO

• Visual SLAM: Why Bundle Adjust?, ICRA 2019

  • paper
  • rotation averaging + known rotation BA

• (Add ORB_SLAM, DSO, VINS-Mono Here)

Adaptive Frame Selection from Video

• Multi-Agent Reinforcement Learning Based Frame Sampling for Effective Untrimmed Video Recognition, ICCV 2019

  • paper
  • TODO

• BubbleNets: Learning to Select the Guidance Frame in Video Object Segmentation by Deep Sorting Frames, CVPR 2019

  • paper, code
  • relative performance prediction + bubble sorting, video object segmentation using best annotation frame

• AdaFrame: Adaptive Frame Selection for Fast Video Recognition, CVPR 2019

  • paper
  • memory-augmented LSTM (selection, reward prediction, utility), video recognition using less frames

• Efficient Video Classification Using Fewer Frames, CVPR 2019

  • paper
  • TODO

• Watching a Small Portion could be as Good as Watching All: Towards Efficient Video Classification, IJCAI 2018

  • paper, code
  • TODO

Robotic Manipulation and Perception

• 6-DOF GraspNet: Variational Grasp Generation for Object Manipulation, ICCV 2019

  • paper
  • TODO

• U4D: Unsupervised 4D Dynamic Scene Understanding, ICCV 2019

  • paper
  • TODO

• Deep Hough Voting for 3D Object Detection in Point Clouds, ICCV 2019

  • paper
  • TODO

• Sim-To-Real via Sim-To-Sim: Data-Efficient Robotic Grasping via Randomized-To-Canonical Adaptation Networks, CVPR 2019

  • paper
  • TODO

• CRAVES: Controlling Robotic Arm With a Vision-Based Economic System, CVPR 2019

  • paper
  • TODO

• FlowNet3D: Learning Scene Flow in 3D Point Clouds, CVPR 2019

  • paper
  • TODO

• A Robust Local Spectral Descriptor for Matching Non-Rigid Shapes With Incompatible Shape Structures, CVPR 2019

  • paper
  • TODO

Other Interesting Papers

• DeepMapping: Unsupervised Map Estimation From Multiple Point Clouds, CVPR 2019

  • paper, code
  • TODO

• Understanding the Limitations of CNN-based Absolute Camera Pose Regression, CVPR 2019

  • paper, datasets
  • TODO

• Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving, CVPR 2019

  • paper, code
  • pseudo-LiDAR representations + mimick LiDAR signal