LET-NET implements an extremely lightweight network for feature point extraction and image consistency computation. The network can process a 240 x 320 image on a CPU in about 5ms. Combined with LK optical flow, it breaks the assumption of brightness consistency and performs well on dynamic lighting as well as blurred images.
- The LET-NET training code is released at https://github.com/linyicheng1/LET-NET-Train.
- Gray Image is also suport in LET-NET, you can get pytorch and onnx model tpye in
./model/
- LET-VINS Demo run on UMA-VI dataset is released.
- Our proposed LET-VINS won the second place in the VIO track of the ICCV2023SLAM Challenge, which is the best performance among the traditional methods.
- The preprinted paper was posted at here.
![](https://private-user-images.githubusercontent.com/50650063/277204080-f4e96620-925e-4ff8-a2a9-c8af202c3b99.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjI0MTc3ODUsIm5iZiI6MTcyMjQxNzQ4NSwicGF0aCI6Ii81MDY1MDA2My8yNzcyMDQwODAtZjRlOTY2MjAtOTI1ZS00ZmY4LWEyYTktYzhhZjIwMmMzYjk5LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MzElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzMxVDA5MTgwNVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWJhNDU4MWI5YmMwZGViZDZhMGM4MTJhZjNjZGRhOTMyMTk2ZmRkNmQwY2I4OGQ1NTM5ODgxNDRmNGVhN2Q4NTImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.l-ObRiowBbSc4Rh1h3wCzUb8o_C3kae9sqHUWEsWGUg)
- Breaking of brightness consistency in optical flow with a lightweight CNN network,Yicheng Lin, Shuo Wang, Yunlong Jiang, Bin Han, arXiv:2310.15655, pdf
- OpenCV (https://docs.opencv.org/3.4/d7/d9f/tutorial_linux_install.html)
- ncnn (https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-linux)
Notes: After installing ncnn, you need to change the path in CMakeLists.txt
set(ncnn_DIR "<your_path>/install/lib/cmake/ncnn" CACHE PATH "Directory that contains ncnnConfig.cmake")
mkdir build && cd build
cmake .. && make -j4
You can enter the path to a video or two images.
./build/demo <path_param> <path_bin> <path_video>
or
./build/demo <path_param> <path_bin> <path_img_1> <path_img_2>
For example using the data we provide:
./build/demo ./model/model.param ./model/model.bin ./assets/nyu_snippet.mp4
You should see the following output from the NYU sequence snippet:
The left is ours and the right is the original optical flow algorithm.
![]() |
![]() |
The left is ours and the right is the original optical flow algorithm.
![]() |
![]() |
The left is ours and the right is the original optical flow algorithm.
![]() |
![]() |
@misc{let-net,
title={Breaking of brightness consistency in optical flow with a lightweight CNN network},
author={Yicheng Lin and Shuo Wang and Yunlong Jiang and Bin Han},
year={2023},
eprint={2310.15655},
archivePrefix={arXiv}
}