Skip to content

Python scripts performing stereo depth estimation using the CREStereo model in ONNX.

License

Notifications You must be signed in to change notification settings

ibaiGorordo/ONNX-CREStereo-Depth-Estimation

Repository files navigation

ONNX-CREStereo-Depth-Estimation

Python scripts performing stereo depth estimation using the CREStereo model in ONNX.

!CREStereo detph estimation Stereo depth estimation on the cones images from the Middlebury dataset (https://vision.middlebury.edu/stereo/data/scenes2003/)

Requirements

  • Check the requirements.txt file.
  • For ONNX, if you have a NVIDIA GPU, then install the onnxruntime-gpu, otherwise use the onnxruntime library.
  • For OAK-D host inference, you will need the depthai library.
  • Additionally, pafy and youtube-dl are required for youtube video inference.

Installation

git clone https://github.com/ibaiGorordo/ONNX-CREStereo-Depth-Estimation.git
cd ONNX-CREStereo-Depth-Estimation
pip install -r requirements.txt

ONNX Runtime

For Nvidia GPU computers: pip install onnxruntime-gpu

Otherwise: pip install onnxruntime

For youtube video inference

pip install youtube_dl
pip install git+https://github.com/zizo-pro/pafy@b8976f22c19e4ab5515cacbfae0a3970370c102b

OAK-D Host inference:

pip install depthai

You might need additional installations, check the depthai reference below for more details.

ONNX model

The models were converted from the Pytorch implementation below by PINTO0309, download the models from the download script in his repository and save them into the models folder.

Original MegEngine model

The original model was trained in the MegEngine framework: original repository.

Pytorch model

The original MegEngine model was converted to Pytorch with this repository: https://github.com/ibaiGorordo/CREStereo-Pytorch

Examples

  • Image inference:
python image_depth_estimation.py
  • Video inference:
python video_depth_estimation.py
python driving_stereo_test.py

!CREStereo depth estimation

Original video: Driving stereo dataset, reference below

python driving_stereo_point_cloud.py

!CREStereo depth estimation point cloud

Original video: Driving stereo dataset, reference below

  • Depthai inference:
python depthai_host_depth_estimation.py

Model option comparison (Nvidia 1660 Super)

In the graph below, the different model options, i.e. input shape, version (init or combined) and number of iterations are combined. The comparison is done compared to the results obtained with the largest model (720x1280 combined with 20 iters), as it is expected to provide the best results.

  • The size of the marker indicates the number of iterations, increasing with the number of iterations. !CREStereo model option comparison

References:

Releases

No releases published

Packages

No packages published

Languages