run: allow tracking of all params in a file with `-p`

[gcoter]

Bug Report

Please provide information about your setup

Output of dvc version:

DVC version: 1.0.2
Python version: 3.6.8
Platform: Linux-4.15.0-106-generic-x86_64-with-LinuxMint-18.3-sylvia
Binary: False
Package: pip
Supported remotes: http, https, ssh
Cache: reflink - not supported, hardlink - supported, symlink - supported
Repo: dvc, git

Additional Information (if any):

I tried to use dvc run with two -p arguments (because I have two parameters files). It gives me this error:

ERROR: dependency 'params.yaml' does not exist

If I remove one of the two arguments, it works. So it looks like DVC doesn't enjoy having two -p arguments. Is it expected? If DVC doesn't support more than one parameters file, the ERROR message should indicate that more explicitly.

[skshetry]

@gcoter, does the params.yaml exist? Can you give more verbose error message (and, your command that you tried)?

[gcoter]

@skshetry thanks for your quick answer! My command looks like this:

dvc run -d training.py -p config/params_1.json -p config/params_2.json:data.training,model -o trained_model/ -n training python training.py

I prefer to use JSON files so params.yaml does not exist in my project.

[skshetry]

@gcoter, is that a typo on -p config/params_1.json (i.e. there's no list of parameters eg: -p config/params_1.json:loss).

[gcoter]

@skshetry No, I thought that specifying config/params_1.json would select everything inside this file. Should I specify explicitly the keys I want to select?

[skshetry]

@gcoter, yes, you have to specify keys that you want.

[gcoter]

@skshetry Ok, indeed it works if I specify manually the keys. I thought that DVC would list all the keys by default. But I don't mind listing them myself. Thank you for your help!

[skshetry]

@gcoter, but, I can see it's usefulness. I'll log this as a feature request.

[skshetry]

@gcoter, another question, why do you choose to -p rather than tracking whole file? for granularity tracking of parameter values, right?

[gcoter]

@skshetry I am at the beginning of the project, so it is not very clear in my mind, but I would like to benefit from the dvc params diff command. Basically, if I use -d, I cannot diff the params. Well, I could use git diff but dvc params diff formats the diff in a nice way in my opinion. And since this file is actually used to store configuration settings, it makes sense to me to use -p.