Skip to content
Depth Hints are complementary depth suggestions which improve monocular depth estimation algorithms trained from stereo pairs
Branch: master
Clone or download
Latest commit 4daf625 Oct 11, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
assets Initial commit Sep 19, 2019
README.md Update README.md Oct 11, 2019

README.md

Self-Supervised Monocular Depth Hints

Jamie Watson, Michael Firman, Gabriel J. Brostow and Daniyar Turmukhambetov – ICCV 2019

[Link to paper]

example input output gif

We introduce Depth Hints, which improve monocular depth estimation algorithms trained from stereo pairs.

We find that photometric reprojection losses used with self-supervised learning typically have multiple local minima.
This can restrict what a regression network learns, for example causing artifacts around thin structures.

Depth Hints are complementary depth suggestions obtained from simple off-the-shelf stereo algorithms, e.g. Semi-Global Matching. These hints are used during training to guide the network to learn better weights. They require no additional data, and are assumed to be right only sometimes.

Combined with other good practices, Depth Hints gives state-of-the-art depth predictions on the KITTI benchmark (see images above and results table below). We show additional monocular depth estimation results on the sceneflow dataset:

example input output gif

✏️ 📄 Citation

If you find our work useful or interesting, please consider citing our paper:

@inproceedings{watson-2019-depth-hints,
  title     = {Self-Supervised Monocular Depth Hints},
  author    = {Jamie Watson and
               Michael Firman and
               Gabriel J. Brostow and
               Daniyar Turmukhambetov},
  booktitle = {ICCV},
  year      = {2019}
}

📈 KITTI Results

Model name Training modality ImageNet pretrained Resolution Abs rel Sq rel 𝛿 < 1.25
Ours Resnet50 Stereo Yes 640 x 192 0.102 0.762 0.880
Ours Resnet50 no pt Stereo No 640 x 192 0.118 0.941 0.850
Ours HR Resnet50 Stereo Yes 1024 x 320 0.096 0.723 0.890
Ours HR Resnet50 no pt Stereo No 1024 x 320 0.112 0.857 0.861
Ours HR Mono + Stereo Yes 1024 x 320 0.098 0.702 0.887

Please see the paper for full results.

⚙️ Code

Coming soon!

In the meantime check out our monodepth2 repository for depth estimation from monocular and stereo training data.

You can’t perform that action at this time.