Merge two optimization-based (NSFP, FastNSF) methods into codebase from HiMo project#14
Merged
Kin-Zhang merged 18 commits intoKTH-RPL:mainfrom Aug 11, 2025
Merged
Conversation
in case we need pip install -e rather than use setup.py directly.
in case old env cannot load, keep two options here.
* try to speed up if we have more gpus, but I haven't test on multi-gpu and multi-node session. Will test later once I got the setup. * upgrade training env to python 3.10 and with cuda 11.8 for afterward env upgrading etc.
* and fix typo from copilot reviewers.
There was a problem hiding this comment.
Pull Request Overview
This PR integrates two optimization-based scene flow estimation methods (NSFP and FastNSF) from the HiMo project into the existing codebase. The integration includes custom CUDA optimizations for improved performance and a new distributed inference runner to handle optimization-based models.
Key changes:
- Added NSFP and FastNSF model implementations with unified interface
- Implemented distributed inference runner for optimization-based methods
- Updated build configuration and dependency management
Reviewed Changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| src/runner.py | New distributed inference runner replacing PyTorch Lightning for optimization-based models |
| src/models/nsfp.py | NSFP model implementation with CUDA-accelerated Chamfer distance |
| src/models/fastnsf.py | FastNSF model implementation using FastGeodis for distance transforms |
| src/models/basic/nsfp_module.py | Added weight initialization methods to Neural_Prior class |
| src/models/init.py | Model registration with graceful handling of missing dependencies |
| conf/model/nsfp.yaml | Configuration file for NSFP hyperparameters |
| conf/model/fastnsf.yaml | Configuration file for FastNSF hyperparameters |
Kin-Zhang
changed the title
Kin-Zhang
changed the title
Member
Author
|
For FastNSF dependencies Reference to this issue. Manually solve by in my setup: (opensf) ➜ cp /usr/include/crypt.h /home/qingwen/miniforge3/envs/opensf/include/python3.8
(opensf) ➜ pip install FastGeodis --no-build-isolation --no-cache-dir |
Member
Author
Looks good for multi-gpu running on one node. (2x2080Ti, NSFP; VRAM: ~2G/gpu) where I noticed that the full cpu run. A **demo** result screenshot with timer on **only 2 scenes**, this is NOT THE FULL validation set.
|
* need set cuda device during spawn. * apply coplit pull request review. * finished checking on 2-gpu setting. (but not yet test on multi-nodes)
…r if we already have torch. * as dockerfile shown.
* update env file (check all good) * update dockerfile
…ion. * fix some issues on test on runner * but haven't test it on multi-gpu.
* update header info about HiMo
Member
Author
|
Almost finished, the final thing to check is about the multi-gpu test-mode output file to see if it can zip successfully at the end. save-mode also works well and results are correct. |
* visualization result also checked, corrected. * todo tested on multi-gpu to see if it works.
Kin-Zhang
force-pushed
the
feature/himo_baseline
branch
from
0f14594 to
abccd67
Compare
* style(header): update header info.
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces two new models, NSFP and FastNSF, into the codebase, along with their configuration files and supporting infrastructure. These are part of the HiMo project's benchmark code.
Model Integration
NSFPandFastNSFmodels, including their full implementations insrc/models/nsfp.pyandsrc/models/fastnsf.py, enabling new scene flow estimation methods. [1] [2]src/models/__init__.pyto register both models for import, with informative warnings if their dependencies are missing. [1] [2]Configuration
conf/model/nsfp.yamlandconf/model/fastnsf.yaml, specifying hyperparameters and training settings. [1] [2]CUDA Extension
TBD
...
Other notes: another optimization-based method (ICP-Flow) will be merged by another new pull request where I will ask Yancong (original author) to review the ICP-Flow pull request.