Skip to content

Environment variable or user path expansion in dvc files #5332

@coroa

Description

@coroa

Summary

I'd like to be able to use environment variables like $HOME in the path in .dvc fragments, ie.

md5: ae7728725f3ad3c123de3abbe545132f
frozen: true
deps:
- md5: 00ebf10af02368c6a2fbef8e997e7229
  size: 7203873
  path: $(HOME)/Box/projectdirectory/externalexcelsheet.xlsm
outs:
- md5: 00ebf10af02368c6a2fbef8e997e7229
  size: 7203873
  path: externalexcelsheet.xlsm

Rationale

When working as a team, we often keep input files like Excel data in project directories on Box (similar to Dropbox) and would like to import them (and keep them linked) into dvc pipelines, with dvc import-url, so that we can update them easily with dvc update. Box (or more precisely its extra tooling BoxDrive) by default mounts the remote shares into a Box folder in the home directory ie. /Users/user1 or /Users/user2 (on mac os x).

The immediate workarounds fail:

  1. using the absolute path to refer to the external file fails since the user slug is different
  2. using a relative path fails, since everyone is keeping their repositories in different folders

It's quite likely a variant of this feature which expands variable in paths for external outputs is also helpful, but I am unable to pin down how that would work yet.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions