Recursively add files by pattern

[pongnguy]

Hi, I save my model outputs using a specific name, but in different folders (e.g. by epoch). I would like to add all the files with the same name (but in different folders). Would it be possible to enable dvc add to add all files of the same name (but in different folders) to the cache? In general, it could add all files that have the given pattern.

[efiop]

Hi @pongnguy !

Are you talking about dvc add or dvc run/repro?

[pongnguy]

I am referring to dvc add. I just started using dvc and only use it on individual large files. I read in the docs that a whole directory can be added. However I would like to add selected **files** from the directory (and sub-directories).

…

On Sun, May 24, 2020 at 10:57 PM Ruslan Kuprieiev ***@***.***> wrote: Hi @pongnguy <https://github.com/pongnguy> ! Are you talking about dvc add or dvc run/repro? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3870 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAW5BLTXBJTBRGOD6SZNU5DRTHNBVANCNFSM4NJFCE3A> .

[skshetry]

@pongnguy, dvc add does not support patterns.

Does .dvcignore satisfy your requirements?

Or, for now, you can just depend on bash/zsh's glob,with something like:

dvc add **/name_prefix*

[pongnguy]

@skshetry .dvcignore doesn't fit my use case since I would like to positively add certain files, instead of adding all files and negatively excluding some. There are only a few files I want to add.

Thanks for the tip on glob. I split my development time between a Windows (local) environment and a Linux (server) environment. So I was looking for something built directly into dvc.

[efiop]

@pongnguy Do you use powershell or cmd.exe on windows? Usually people use something *sh-like on windows too (e.g. gitbash or conda shell), where globing is supported roughly to the same extent as on typical *nix shells.

[efiop]

Closing as stale.