Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overwrite parameters via CLI #71

Closed
n-gao opened this issue Nov 4, 2021 · 3 comments
Closed

Overwrite parameters via CLI #71

n-gao opened this issue Nov 4, 2021 · 3 comments

Comments

@n-gao
Copy link
Collaborator

n-gao commented Nov 4, 2021

At the moment one has to edit the config file if one is interested in changing a single parameter. It would be great if we could do this via CLI. The case I most encounter is that I want to name different batches. Currently, I will have to either edit the yaml file and set a name there or put everything into a different MongoDB collection. Since it is much more comfortable to do this via the CLI I do the latter but the drawback is that my MongoDB is getting polluted with lots of collections.

My proposal would be to overwrite parameters via CLI similar to sacred. In sacred one can do

python script.py with param1=5

For seml I could envision something like

seml <collection> add <yaml> with param1=5

The syntax with with might be unfitting and up to discussion. Other points of discussion are the interactions with subconfigs and grid in general.

@gasteigerjo
Copy link
Collaborator

I think this is a good idea. We just need to be careful about 2 details:

  1. How the syntax should work, and how this would be possible with the argparse library.
  2. That we properly handle overriding grids/random blocks, and check duplicates accordingly.

@n-gao
Copy link
Collaborator Author

n-gao commented Nov 8, 2021

  1. I would propose a similar syntax to sacred by having seml <collection> add <yaml> [ADD Args] with [<param>=<value>]. Implementation-wise since we are splitting the CLI args by commands this should not inflict with chaining, so seml <collection> add <yaml1> with [<param>=<value>] add <yaml2> should work just fine. There are two ways to approach this either we use parser.parse_known_args() instead of parser.parse_args() or we exploit the fact that we already manually split the parameters to support chaining. The latter one seems a lot safer to do to prevent something like seml <collection> add <yaml> with [<param>=<value>] -f -nsc which, imho, is just confusing the syntax. If we split it manually we simply split CLI args assigned to add by with.
  2. I think the most sensible thing to do is to override the parameters in all experiments and then remove duplicates while printing a warning.

@n-gao
Copy link
Collaborator Author

n-gao commented Dec 13, 2021

Closed by #79

@n-gao n-gao closed this as completed Dec 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants