New feature: SOLD2 line detector and descriptor #1507
Conversation
rpautrat
requested review from
ducha-aiki,
edgarriba,
lferraz and
shijianjian
as code owners
for more information, see https://pre-commit.ci
|
Wow, what a Christmas gift! |
|
Yes, I finally found some time to do it! |
|
@rpautrat awesome and thanks! I'll review over the next week and target to be merged for the release after new year's :) |
There was a problem hiding this comment.
some suggestions and mostly some functions that need to be re-implemented in pytorch instead of numpy. One general question: how much this method generalizes and how easy/difficult to fine-tune? just to understand how sensible is in front new data. For future after merge, a quick colab showing for example to reproduce that nice gif would be great.
|
Thanks for the detailed review and feedback! I might need a bit of time for the re-implementation from numpy to pytorch because there are some deadlines coming soon. The colab showing how to reproduce the GIF would be feasible, yes. Regarding generalization and fine-tuning, it generalizes fairly well indoors, but the performance is not as good outdoors. I was planning to release an outdoor model at some point, but it takes some time to train and tune it properly. Fine-tuning is indeed cumbersome unfortunately, because you need to generate the pseudo ground truth on the new dataset each time... But if I manage to get a good outdoor model, it might be less essential to fine-tune it. |
Sounds good ! No rush here from our side. Let us know if you need help with any part. |
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
|
Hi, I had time to work again on this PR. I have moved all numpy functions to Pytorch as suggested, and opted for the torchvision nms operation. It was the fastest among the kornia one, torchvision one and SuperPoint one. After consideration, releasing the code generating the GIF in a colab might not be possible, as it was partially based on the SuperPoint demo (https://github.com/magicleap/SuperPointPretrainedNetwork). The license stipulates that you can only use the code for research or non-profit purposes, so releasing it on Colab might be against it... |
There was a problem hiding this comment.
@ducha-aiki @shijianjian can you review
@rpautrat sounds good about the colab thing -- the example looks really nice
| @@ -0,0 +1,436 @@ | |||
| """Implements several backbone networks.""" | |||
There was a problem hiding this comment.
just as a meta-thought: @ducha-aiki @shijianjian we might plan at some point to make a low level-backbones-features API. Open to suggestions
| return state_dict | ||
|
|
||
|
|
||
| class WunschLineMatcher: |
There was a problem hiding this comment.
@ducha-aiki we could move this the matching API
There was a problem hiding this comment.
Yes, I will take a look after this PR merged
Remove cast to float before normalize_pixel_coordinates Co-authored-by: Edgar Riba <edgar.riba@gmail.com>
…S is no junction detected
for more information, see https://pre-commit.ci
Changes
Addition of the SOLD2 line detector and descriptor. I separated the detector and descriptor to allow to only detect lines, or detect&describe and perform line matching across images.
The added code does not require any special package outside of numpy and pytorch.
Fixes #1124
Type of change
Checklist