[Python API] Have dvc.api.open support directories

[houssam7737]

It would be great to have a dvc.api.open or another method within the python api support downloading entire directories (like dvc get supports both files and directories).

That abstraction would be pretty useful to me right now, and would save some time. It's not clear to me how to achieve this right now in python without using subprocess to use dvc in the shell (I can't assume the repo is already present locally, so os.listdir is a no go if repo is remote only).

Cheers!

[skshetry]

You might be interested in #6850 which provides a pathlib-like API with which you can traverse a directory or a complete repository.

[houssam7737]

You might be interested in #6850 which provides a pathlib-like API with which you can traverse a directory or a complete repository.

Nice, subscribed to the PR to get updates and be notified when it lands. Thanks!

[JaviFuentes94]

This would be great for my use case as well

[skshetry]

We just published DvcFileSystem API in 2.27.0 that can download/list directories. Please check the docs for more details: https://dvc.org/doc/api-reference/dvcfilesystem.

I am going to close this issue, as now DvcFileSystem is preferred for more complex scenarios. 🙂
Please feel free to give any feedbacks, open any issues, etc.