New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow the definition of process resources using a custom category annotation #623
Comments
Very good idea! |
This could also be extended to containers... imagine that I have a container for processes 1,2,3 and another for 4,5,6 etc... |
Yes, the point is to decouple the resource annotations from the
This should address would also address the issue #621. |
A first implementation of new process configuration via label annotations is ready to tested. This version allow a process to define one or more labels eg:
Then in the configuration you can use the the
See also the previous comment for other details. You can give a try using the version
@ewels @maxulysse @lucacozzuto @msmootsgi You feedback is welcome. |
Great, I'll look into it ASAP |
Same here :-) |
Nice! A simple question: why not to use the same tag: (i.e. label) for both process definitions (in both main and configuration file)?
This will be easier to remember. |
That's how it is supposed to work. |
Ahh, no sorry, I've misread. Actually I've tried also that, but at the I've found more readable |
Well, let's ask also to the other guys :) |
Don't really mind. |
Things to do to complete this issue:
|
Great initiative!
Are we talking about breaking backwards compatibility here? For the record, I also really like the current setup - I think it's very clear. The above would be a great addition, but personally I'd like to continue using the current syntax as well if possible. |
No, no breaking existing pipeline, but give a window to allow users to migrate to the new syntax eg
The problem with |
Ok, gotcha 👍 Sounds good! |
Nice! |
Question about the new syntax. Say I've got a label defined like this in
And then I use the label in a
|
Currently it overrides the |
This could just be how I write pipelines, but I put default values (and labels in the future) in I find these override hierarchies kind of tricky to get right, so I think it's worth some thought. And I make no claims here - I haven't given this a ton of thought! |
I tend to agree, that it could be useful it can be quite tricky to remember all these rules. Maybe just keep it simple, that setting defined in the config just overwrites the ones in the process is the a better choice. The Anyhow I will keep that as an open topic. |
As long as the rules are clearly stated, I'm OK with this. |
I am not sure whether we might also think about more specific settings overwriting general ones. I mean that configuration values defined at Also, what about processes with multiple labels that touch common options? I think it would be better to avoid it, but in case it won't be possible what would be the order of precedence? |
If I have understood well, that's already possible. Take in consideration the following config:
The above snippet defines:
Currently the last won. |
OK, another round adding the support for regulation expression and not operator! This allows to define label/process selectors with regex patterns, eg:
Please give it a try using the following command CAPSULE_RESET=1 NXF_VER=0.29.0-SNAPSHOT nextflow info # ONLY THE FIRST TIME
NXF_VER=0.29.0-SNAPSHOT nextflow run ...etc You can find the documentation at this links: |
Available in version Note: release |
@pditommaso is it possible to define a new label and after that assign a job to the new label?
Thanks, Ido |
Think it's related to #1786 |
@pditommaso but I'm not defining the label in a variable, that's plain text. |
It would be nice to have a way to classify processes according to their resources allocation for reducing the size of config file.
For example:
processes 1, 2,3,4,5 can be grouped as huge_demanding_ processes and processes 6,7,8,9,10 as little_processes. In this way you need to specify only two configurations within the config.file
The text was updated successfully, but these errors were encountered: