Use min views filter = 1

[pierotofy]

Opening draft for testing and possible merging.

So far I have not seen significant increases in noise, all while coverage for lower overlap areas is significantly improved.

[pierotofy]

I also propose to change the default values for pc-filter from 2.5 to 20 5 and to use dspsift by default as the feature-type; 2.5 is too aggressive and tends to remove points near facades (when shot nadir), so buildings don't get reconstructed properly. A value of 5 removes far outliers, but preserves more facade points. dspsift does not take advantage of the GPU, but performs better than sift, so it should be the default.

[smathermather]

dsp-sift is overdue. It's a harmless to useful change. And I'll remember to use it if its default, so bonus.

Haven't tested the changes in filtering yet, but min-consistent-views change looks fine, even in datasets with excessive existing overlap. Seems to reduce noise marginally in very high-overlap datasets because it fills gaps:

[smathermather]

I'll post these here too from https://community.opendronemap.org/t/processing-historical-aerial-photos-revisited/19273/44?u=smathermather

[pierotofy]

[smathermather]

20 was not aggressive enough, I assume?

Definitely nicer completeness on that ballfield.

[pierotofy]

20 was not aggressive enough, I assume?

Yes, as usual more testing yielded some counterexamples of where being too soft on filtering left some undesirable results.

[smathermather]

Leaf off forest example (just min-num change, not filtering change yet -- too many test datasets now running and a few hours in...):

[smathermather]

dspsift does not take advantage of the GPU, but performs better than sift, so it should be the default.

Should we continue to default to sift when GPU is available, or not?

[Saijin-Naib]

Mmm... I think GPU SIFT should be opt-in until it matches CPU SIFT/DSPSIFT reconstruction.

I think getting better results slower is a kinder default than less complete results faster.

Easier to explain and adjust folks to, as well, I would think.

[smathermather]

Hmm, good point. This will improve outcomes for GPU users as is. 🙂

[smathermather]

Nice to see vegetation much better handled (not yet using the new filter value, but re-running with that...:

[smathermather]

Confirmed that filter value does help with this relatively low overlap dataset on building edge:

(here in combination with the min views filter = 1)

[pierotofy]

I've run some tests on a few datasets and I'm good to merge this, unless there are objections or further testing presents some issues.

[smathermather]

I've got one more dataset to test and then I'm happy to bless it too.

[smathermather]

Hitting an OOM on a 200+ image dataset at DEM stage on a 768GB RAM machine. Probably something degenerate with the point cloud (will look tomorrow / later today).

@pierotofy -- have you done any memory profiling in your tests? I didn't think till now to try some docker instances with capped RAM for verifying.

[pierotofy]

Mm, good point. I did find an out of memory issue (during the depthmap fusion step). Will need to investigate a bit.

[Saijin-Naib]

The difference is subtle with my incredibly poorly collected mf_old-orchard, but it is better:

vs

---

vs

So, vegetation and built structures seem to benefit most, which is great, especially at the borders of the survey (I had no overfly in my survey designs back then).

There is an incredibly slight increase in total inaccuracy, but estimated GSD also very slightly improves.

I think it is overall a huge net-positive.

[pierotofy]

I've processed the dataset that was giving me memory troubles (I think I was running out of RAM because I had QGIS open, not necessarily because of the large increase in memory). Just to be safe, I've improved the mechanism for kicking in the clustering fusion logic.

This PR will increase the number of points for every dataset and thus the memory requirements a bit, but not severely. I processed all the large datasets that push the limits of my dated development machine without issues. So I'm OK to merge this.

[smathermather]

Cool. In the absence of the forthcoming test suite that I hope will A/B test changes in memory (time tbd, but after May), I have no objection to a merge.

It makes a big difference in a lot of datasets.