Improve time estimates

[smathermather]

Improving time estimates is a fool's errand, which makes it perfect for me.

I'd like to improve the proportion complete estimates. I used to know where these are set, but my brain is mush, and I cannot find it. Point me to the right place and I plan to update these values with a medium size dataset: something that for most folks will take a while to process (where the proportion of time between portions of the project matter the most). It won't be ideal for small datasets (for which we are already tuned), nor for massive ones. But it should help for projects that take long enough to notice and for which people are likely to be watching the progress bar.

Like I said. Fool's errand. But like a fool, I've processed a lot of data.

[pierotofy]

😄 I think it's more of a "guessing game"

Each stage in the ODM pipeline has a progress parameter: https://github.com/OpenDroneMap/ODM/blob/master/stages/odm_app.py#L38

It indicates the overall of a stage (e.g. the dataset stage, when finished, will show 5%).

Each stage in turn can update it's own internal progress:

https://github.com/OpenDroneMap/ODM/blob/master/stages/run_opensfm.py#L34

These values should go from 0 to 100 and they represent the fraction of progress.

So calling self.update_progress(50) inside the dataset stage will lead to an overall "global" progress of 2.5% when the function is called.

[pierotofy]

If no call to self.update_progress is made, the stage begins with 0 and automatically finishes with 100 (so you don't need to explicitly set update_progress(0) and update_progress(100) at the beginning/end of the stage.

[Saijin-Naib]

Would it be possible to have the progress bar divide into "bins" per step of the pipeline to sort of de-mystify the progress a bit?

Similar in concept to this?

A global progress bar, and then a secondary bar with stage/step progress/gates?

[pierotofy]

We used to have something like this several versions ago; I opted to remove it since it made the UI simpler (we have more than 7 stages and it seemed overwhelming).

Not against reintroducing it if somebody opens a PR with a clean UI...

[Saijin-Naib]

I think it could potentially address folks' consternation about what's happening if they have a more concrete idea of what step they're in, what step is next, etc, especially since we have some expectations about which steps are the heaviest and most likely to fail.

But yeah, I can see how it could look crowded.... hmm

[Saijin-Naib]

User-story for segmented Progress Bar to act as summary of Processing Log:
https://community.opendronemap.org/t/summary-task-output-please/10314?u=saijin_naib