Suitable value for etaBuffer?

[reidrivenburgh]

I have a long-running job (about 24 hours) that updates every five to ten minutes. I'm trying to use {eta_formatted}, but after my first update, I get "INFs". I've been gradually increasing the value of etaBuffer to try to eliminate it as the docs suggest, and I'm currently at 100000000, but it's still happening. I've also tried setting etaAsynchronousUpdate to true with a fps of 1. Am I not approaching this correctly...?

Thank you.

[AndiDittrich]

i've never tried to use it for such long runs...

• you can try to set fps: 0.05 (20s) and increase the eta buffer to a value that at least 3...4 updates are includes.
• etaAsynchronousUpdate is not always useful - see https://github.com/AndiDittrich/Node.CLI-Progress/blob/master/lib/generic-bar.js#L74
• • take a look into the eta calculation for details: https://github.com/AndiDittrich/Node.CLI-Progress/blob/master/lib/eta.js

maybe you need a custom eta calculation (just use it as payload)

[reidrivenburgh]

Thanks for the information. Is this code causing it? Is there any reason for this check?

// > 100k s ?
}else if (eta > 100000){
this.eta = 'INF';

[AndiDittrich]

such long running eta's normally indicates some kind of error.

[reidrivenburgh]

It can be normal behavior, though, as in my case. If it was an error, it seems like it would sort itself out after some iterations. In any case, that's a pretty arbitrary cutoff and is only 27 hours. Could you at least increase that check by an order of magnitude?

[AndiDittrich]

i can increase it but i wouldn't call your use-case "normal"

[reidrivenburgh]

Thank you. Maybe "normal" isn't the right word, but it's correct behavior that shouldn't just be called an error. I work in an area where long-running programs aren't unusual. It's exactly why I wanted the ETA functionality.

[AndiDittrich]

the problem with the current eta calculation is the limitation of a linear interpolation - especially long running tasks are mostly non-linear which makes the calculated eta value useless. you'll find some discussion about this topic.

therefore i would recommend to use a "custom" eta calculation for serious usage.

[reidrivenburgh]

You're making a lot of assumptions. My task actually is fairly linear and should be fine, it just happens to be long. I would have written something similar for estimating ETA. Why break your code for me just because it may give inaccurate results for someone else? I would suggest that you don't artifically limit it, but warn the users in the docs that if their task is too nonlinear, the results may not be accurate/useful. That's true for a task of any duration.

[reidrivenburgh]

By the way, Microsoft's progress meters are pretty notorious for this, in my limited experience. A task might take 15 minutes, but the first five will sit at 0%, then it'll suddenly jump to 40%, take another five minutes, etc.

[AndiDittrich]

i fully understand your issue and your intention but on the other hand i've to be aware of a lot of other known use cases and issues.
and as said i'll change the limit to 1e6

[reidrivenburgh]

Understood, thank you. I'm just trying to be helpful with constructive suggestions.

[AndiDittrich]

changed in ea73371