Allow to specify number of workers in verdi daemon start

[sphuber]

Fixes #2998

Now you can start more than just one worker when starting the daemon.

[coveralls]

Coverage increased (+0.2%) to 73.203% when pulling becb8b5 on sphuber:fix_2998_verdi_daemon_start_num into ababf1c on aiidateam:develop.

[ltalirz]

looks all good, just one question

[ltalirz]

Could this be an option instead?
The command currently is verdi start-circus 12 and I think verdi start-circus --nworkers 12 (or similar) would be more self-explanatory.

[sphuber]

One never calls verdi daemon start-circus it is a hidden command. But it would be verdi daemon start [NUMBER]. I chose the argument because that is inline with the already existing commands verdi daemon incr/decr [NUMBER].

[ltalirz]

Thanks, that looks already much better!

Sorry to keep nagging here - just two more questions:

• Is there a configuration setting for the default number of workers? Or is this something one has to put every time one starts the daemon? If there is no configuration setting for this, perhaps we should add an issue for this?
• One thing I never quite understood: Do we need to go via a hidden click command here? Can't we just spawn a process directly from a function like so?
  Anyhow, this is just out of curiosity.

[sphuber]

Is there a configuration setting for the default number of workers? Or is this something one has to put every time one starts the daemon?

There isn't a configuration setting yet, as this functionality is new, so yes, for now one would always have to specify an explicit number if one wants more than the default one. I don't think it is a problem, but adding a new issue with a feature request to make this configurable would be a good idea.

Do we need to go via a hidden click command here?

It sure is not necessary, it would certainly be possible to implement it differently. I think Rico implemented the very first command this way when migrating to circus. I just kept it and especially now that it is properly hidden, I don't see any harm

[ltalirz]

Ok, then I would kindly request to

• open an issue as discussed above
• mention somewhere in an appropriate docstring (or in a separate issue) the alternative of going through multiprocessing

Will approve then - am offline for 2h now

[sphuber]

open an issue as discussed above

Done, see #3007

mention somewhere in an appropriate docstring (or in a separate issue) the alternative of going through multiprocessing

I just realized why keeping the click route is preferable. Since we are launching a sub process, the correct profile needs to be loaded. This is currently taken care of automatically by the verdi click infrastructure that correctly parses and loads the desired profile. If we were to go through a normal function that we would call as a sub process, that function would have to provide some way to take a profile different from the default. You will basically end up implementing another command line parser.

[ltalirz]

thanks a lot @sphuber !