get: show url to cache

[dmpetrov]

In some cases, users need a direct link to a data file in the cloud. It might be just a matter of convenience or when DVC is used from automation tools like CD4ML scenarios. We need to provide a way to get a direct link.

I suggest creating a new command dvc resolve. Other options can be also be considered. For example, it might be a part of dvc status (when it start supporting data files as argument).

$ dvc resolve file.txt df.csv
file.txt s3://dvc-temp/dataset1/2d10f00a05f1fe70eaa3c42aa9f44b95
df.csv s3://dvc-temp/dataset1/99370a5386cac00c93c9fdc836076a7d

It should support revisions (checksum, tag, branch or HEAD^^):

$ dvc resolve file.txt df.csv --rev ada9d973
file.txt s3://dvc-temp/dataset1/d30abac6950583e6a5f1dcd31cb1043b
df.csv gs://project42/cache/2290babda371e52eeca2a2065a358783

Note, the commands with revision options should resolve old remotes as well (see s3:// and gs:// above).

[efiop]

@dmpetrov Sounds like some automation helper, that can be worked around by simply reading the dvc file, extracting the checksum and then accessing the remote. Sounds like that wouldn't be a problem for anyone needing this for some automation script, not sure it is useful enough for many users though.

[dmpetrov]

Sounds like some automation helper, that can be worked around by simply reading the dvc file

This feature is required in automation scenarios where people prefer to write bash code. Also, in the case of using revisions, it won't be as simple to implement. A single command - would be really great to have.

In terms of convenience, if you ask a user to do that in a manual way - it requires quite deep DVC knowledge and it is easy to make a mistake. The user should do 4 operations: find the corresponding dvc-file, find a right checksum, find default remote in config, combine them. 2x more complex for old revisions - file copies, need to know get command.

This command will hide a lot of internals and will simplify the usage. It is about lowering the bar.

[Suor]

This issue has a remote config complication - do we use a remote from config from specified rev or current one? Both might be useful:

• use current if there was no default remote back then or if we changed default remote and moved all the data
• use the one from rev if we intentionally use different default remotes in e.g. different branches

[jorgeorpinel]

I think this already exists in dvc.api.get_url so may be a matter of wrapping it in a command 🙂 (and figuring the command arguments and their logic, etc.)

[efiop]

Change of plans. Talked with @dmpetrov @shcheklein , and they suggested that --show-url(or something similar) for dvc get would be better than introducing a separate command. Let's consider that instead.

Also, what to show for directories?

[skshetry]

@efiop, maybe I am missing something, but as per docs, get command downloads a file/directory from a DVC and Git repository. And, adding another responsibility to get, i.e. getting path/URL to the files in the cache, which in my view, is completely different from previous responsibility.