You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, rust-parallel relies on indicatif to calculate ETAs for us, but the problem is that indicatif doesn't know the start times of the tasks, only the end times (when we increase the count). The problem with only knowing end times is that if a lot of tasks finish close to each other, then the ETA estimate decreases a lot suddenly & shoots back up too much when they don't complete for a while.
This probably affects us more since we've a lot of large tasks (biggest being 30m & smallest being 2m). But we know before hand how long each is gonna take, so we order them in descending order before hand. But since the first few groups are similar in size, they would complete very close to each other & the ETA will look much smaller.
I think a better approach might be to have a simple logic like elapsed * (total - done) / done or slightly more complicated logic like EMA(time per task) * (total - done).
The text was updated successfully, but these errors were encountered:
Currently, rust-parallel relies on indicatif to calculate ETAs for us, but the problem is that indicatif doesn't know the start times of the tasks, only the end times (when we increase the count). The problem with only knowing end times is that if a lot of tasks finish close to each other, then the ETA estimate decreases a lot suddenly & shoots back up too much when they don't complete for a while.
This probably affects us more since we've a lot of large tasks (biggest being 30m & smallest being 2m). But we know before hand how long each is gonna take, so we order them in descending order before hand. But since the first few groups are similar in size, they would complete very close to each other & the ETA will look much smaller.
Example:
I think a better approach might be to have a simple logic like
elapsed * (total - done) / done
or slightly more complicated logic likeEMA(time per task) * (total - done)
.The text was updated successfully, but these errors were encountered: