Merge VoteFlow into codebase

[Shiming94]

This pull request is to integrate VoteFlow into the OpenSceneFlow main branch. Readme and pretrained weights are also updated.

Core Code

• src/models/voteflow.py
• src/basic/voteflow_plugin
• config/model/voteflow.yaml

[Kin-Zhang]

It's not working in my environment somehow:

ch/include/ATen/CPUGeneratorImpl.h:3,
from /home/kin/mambaforge/envs/opensf/lib/python3.8/site-packages/torch/include/ATen/Context.h:3,
from /home/kin/mambaforge/envs/opensf/lib/python3.8/site-packages/torch/include/ATen/ATen.h:7,
from /home/kin/data/code_online/OpenSceneFlow/src/models/basic/voteflow_plugin/hough_transformation/cpp_im2ht/ht_cuda.cu:2:\n/home/kin/mambaforge/envs/opensf/include/thrust/system/cuda/config.h:122:2: error: #error The version of CUB in your include path is not compatible with this release of Thrust. CUB is now included in the CUDA Toolkit, so you no longer need to use your own checkout of CUB. Define THRUST_IGNORE_CUB_VERSION_CHECK to ignore this.
122 | #error The version of CUB in your include path is not compatible with this release of Thrust. CUB is now included in the CUDA Toolkit, so you no longer need to use your own checkout of CUB. Define THRUST_IGNORE_CUB_VERSION_CHECK to ignore this.\n      |  ^~~~~\nninja: build stopped: subcommand failed.\n')
full_key: model.target

I will check again later.

---

I see, it looks like we don't need to install these two packages:

conda install -c iopath iopath
conda install -c bottler nvidiacub

Another solution is to load with flags to ignore:

extra_cuda_cflags=["-DTHRUST_IGNORE_CUB_VERSION_CHECK"],

---

It's working for me now:

---

Validation results (Please note that this is evaluated in the fixed validation set, check more detail on fixing gt flow in #5 ):

The raw terminal result screenshot:

[Kin-Zhang]

Hi @Shiming94 , could you update the README again about the training setup and the hour for the model ckpt? Here: https://github.com/KTH-RPL/OpenSceneFlow/blob/feature/voteflow/README.md?plain=1#L148

[Shiming94]

It's not working in my environment somehow:

ch/include/ATen/CPUGeneratorImpl.h:3,
from /home/kin/mambaforge/envs/opensf/lib/python3.8/site-packages/torch/include/ATen/Context.h:3,
from /home/kin/mambaforge/envs/opensf/lib/python3.8/site-packages/torch/include/ATen/ATen.h:7,
from /home/kin/data/code_online/OpenSceneFlow/src/models/basic/voteflow_plugin/hough_transformation/cpp_im2ht/ht_cuda.cu:2:\n/home/kin/mambaforge/envs/opensf/include/thrust/system/cuda/config.h:122:2: error: #error The version of CUB in your include path is not compatible with this release of Thrust. CUB is now included in the CUDA Toolkit, so you no longer need to use your own checkout of CUB. Define THRUST_IGNORE_CUB_VERSION_CHECK to ignore this.
122 | #error The version of CUB in your include path is not compatible with this release of Thrust. CUB is now included in the CUDA Toolkit, so you no longer need to use your own checkout of CUB. Define THRUST_IGNORE_CUB_VERSION_CHECK to ignore this.\n      |  ^~~~~\nninja: build stopped: subcommand failed.\n')
full_key: model.target

I will check again later.

I see, it looks like we don't need to install these two packages:

conda install -c iopath iopath
conda install -c bottler nvidiacub

Great that you made it run, when I setup the environment, I did need to install liopath iopath bottler nvidiacub to install pytorch-3d, maybe you already installed it when you installed the packages for other methods. I am okay with deleting it.

It's working for me now:

Validation results (Please note that this is evaluated in the fixed validation set, check more detail on fixing gt flow in #5 ):

The raw terminal result screenshot:

I checked #5, since the performance of all methods decreases a little bit, the current results are okay. I have double-checked the checkpoint before I made the PR, it is the best checkpoint of VoteFlow.