Is there an UI ?

[seeb0h]

Hi, I am discovering dvc, so excuse me if I missed an obvious answer.

dvc looks great to use with the cli.
Now, I am considering to build a web ui in order to list dvc entities (and trigger experiments runs on cloud resources). Is there a convenient way (like an API) to access to the entities ?

Thanks

[efiop]

Hi @seeb0h !

We do have a python API(dvc/project.py), but it is not stable, not documented and thus not yet ready to be relied on. We will definitely make it stable and release to the public in the future. The only API that is stable and for which we guarantee backward compatibility is CLI. If you only need to access .dvc files, you can do that easily by using any yaml parser of your choice.

The web UI is a very interesting idea, could you please elaborate on it? We have been thinking about creating something like that and would be interested to hear more about what you would like to see it look like.

Thanks,
Ruslan

[elgalu]

By "web UI" @seeb0h probably means something like ModelDB's UI:
http://modeldb.csail.mit.edu:3000/projects

[AKuederle]

To expand on this question, It would be really cool if there would be a way to get specfic datasets stored with dvc without initializing the respective git repo. Often it is required to just share a specific version of your data with your client. I am thinking, it might be an interesting addition, if we could write a little webserver/webservice for self hosting, that given a specific data version just provides the dataset as a http download.

I would be interested on working on something like this (probably as a separate project)

[seeb0h]

I am asking myself in a very forward-looking way.

We are currently updating an ML R&D workflow and dvc is a good candidate for our versioning. We aim (in a more or less distant future) to automate part of the workflow through a web application. So, the presence of an API at dvc will become an important point.

There is many functions that could be done in a webUI.
Tracking metrics as @elgalu mentions. But also provide a convenient way to explore versionning and traceability of code, model and data.

[efiop]

Thank you all guys for the feedback!

@elgalu @seeb0h Thanks for the info! We definitely had similar thoughts on features that we wanted to see implemented in a WebUI on top of core dvc. We are also actively discussing it within our team and think that we will begin working on it by the end of the year. Btw, there is a mailing list in which we sometimes(no spam though) ask the community to answer some questions that help us improve dvc and we will definitely ask more detailed questions/polls there once we start working on the webUI, so please feel free to subscribe 🙂

@AKuederle

To expand on this question, It would be really cool if there would be a way to get specfic datasets stored with dvc without initializing the respective git repo. Often it is required to just share a specific version of your data with your client. I am thinking, it might be an interesting addition, if we could write a little webserver/webservice for self hosting, that given a specific data version just provides the dataset as a http download.

I would be interested on working on something like this (probably as a separate project)

Sorry, I'm not sure I follow. What you are describing sounds like any file sharing service(e.g. Dropbox, s3, etc) to me. Could you elaborate please?

Thanks,
Ruslan

[AKuederle]

Thank you for your response. What I basically meant, was that it would be nice to have a way to get the data, which corresponds from a specific project without installing git/dvc. Similar to downloading code from Github as .zip. Of course you could go right to the specific storage location in your AWS bucket, ssh-server, etc. but it would be nice to have a little webservice (with or without gui) that abstracts that in a way and simple terms is able to provide a simple file download given a specific commit hash of the corresponding git repository.
There would be some issues to be solve, e.g. potential credentials to access the data store, but I think it would be a really nice feature, that would allow dvc to be also used as a kind of "frontend" for a ML data archive.

I think this might still be a little confusing. I will try to expand on that and explain the big picture as soon as i have time.

[efiop]

@AKuederle Thank you for sharing your thoughts! This is indeed a very interesting idea! It would be a great addition to the WebUI mentioned above. I think it is definitely going to be implemented in the future.

Thanks,
Ruslan

[piccolbo]

You could use hug to simultaneously maintain a web and python API and a CLI. It seems to reduce the work of maintaining the three and keeping them in alignment, plus of course the bonus of being able to run dvc as a service. Just my 2c.

[efiop]

Hi @piccolbo !

Thank you for the tip! Hug looks very promising! We will be sure to take a closer look at it when developing WebUI.

Thanks,
Ruslan

[elgalu]

Hi @efiop , since you closed this issue, does it mean Hug is the solution we should look at?

[efiop]

@elgalu I don't know, we haven't been looking at it. I just felt that the discussion is done, since webui is not yet on our radar these days. Looks like I was wrong 🙂 Reopening.