--all-failed flag to restart all failed tasks.

[alextooter]

Hi @Nukesor,

I have many download tasks,few of them failed with exit code 1,so need to restart each of them by hand.

Can we enhance the restart features by this:
1,add command 'pueue restart all',that let user restart all the failed tasks.
2,add autoretry option in configure file.For example,auto retry failed task by X=10 times,then give up.But user can use 'pueue restart all' to restart them again,the autoretry count is reset.
[default]
autoretry = false //default is false
autoretrytimes = 10 //work when autoretry = true

Great thanks.

[Nukesor]

Hi.

I like the idea of an restart --all-failed/-a flag.

I'm not sure about the auto-retry config, though.
It feels like something that should be handled by the programs called by pueue.
wget for instance has a retry logic. curl as well and most other download programs I know of

[Nukesor]

On second thought, a --all-failed flag isn't really something that should be used often.

In most cases, retries should be handled by the called program. If all programs break due to a single prerequisite that hasn't been fulfilled, the user should probably use the pause_on_failure flag.

Closing for now until somebody comes up with a good use case.

[dmd]

I would love it if pueue had the ability to retry failed tasks, kind of like what https://github.com/rq/rq does with the retry option to enqueue. To me that's most of the purpose of a job scheduler - it explicitly should not be up to the called program to have to deal with scheduling retries! The called program has one job - do a thing. If the thing doesn't work, it should fail (and thus not block). Then the scheduler's job is to decide if, and when, to try again.

My particular use case, not that it matters, is doing sends of (thousands of) DICOM medical images from one machine to another, where the receiving machine is quite often unavailable. I want to be able to say "if the image doesn't send, wait 5 minutes ... 10 ... 30 .. 90 .. 720 .. 1440 ... then give up", or something like that.

[dmd]

Without this functionality I essentially have to write a scheduler-scheduler that runs pueue status over and over looking for failed jobs and keeping track of them, and scheduling restarts!

[Nukesor]

Hm.

A program/command that is designed to fail seems like an anti-pattern to me.
Especially on the commandline, tasks should either fail or succeed and that status should have some meaning.

Pueue's design goal is to be a commandline helper, which takes commands from your commandline that would otherwise run too long in your current shell.
You still have the whole feature-set of the shell at your hands.

If you expect a shell command to continously fail, until it succeeds at least once, you can simply do this:
pueue add 'while ! your_command; do; done'

That's it. That's a command that will indefinitely call itself, until it finally succeeds.

[hare1039]

I would like to see the restart all failed task function integrate into pueue itself.
Currently I can only achieve it by some shell hack pueue restart --in-place $(pueue status | grep Failed | awk '{print $1}').

In my use case is that I have a downloader for a particular website, and it do not have a retry function. I add a bunch of tasks for each download. Some tasks failed so I need to check the status manually and restart. I would like to see it integrate as a function in pueue.

[Nukesor]

I see how a --all flag can be useful for the restart command.