Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/ext #52

Closed
wants to merge 5 commits into from
Closed

Feature/ext #52

wants to merge 5 commits into from

Conversation

mike0sv
Copy link
Contributor

@mike0sv mike0sv commented Mar 11, 2022

Draft for interface for mlem/dvc integration

Fast test (preferable with iterative/mlem#169 mlem version):

python -c "import mlem; a = lambda x: x; mlem.api.save(a, 'model')"
gto describe model

@mike0sv mike0sv requested a review from aguschin March 11, 2022 16:00
@mike0sv
Copy link
Contributor Author

mike0sv commented Mar 12, 2022

tagging #46

gto/ext_mlem.py Outdated Show resolved Hide resolved
This was referenced Mar 16, 2022
@aguschin
Copy link
Contributor

As mentioned by @shcheklein, when DVC is installed as binary, GTO may not be able to discover DVC Python API. We need to support CLI as a fallback then. As an option, we can list dvc and mlem in config file gto.yaml so GTO will know what to search for.

@mike0sv
Copy link
Contributor Author

mike0sv commented Mar 16, 2022

Added DVC integration implementation (as an example, it is actually stupid)
To test it you can run something like this

dvc init --subdir
echo "data" > file
dvc add file
gto describe file

@mike0sv
Copy link
Contributor Author

mike0sv commented Mar 16, 2022

Also added CLI implementation for the case we discussed. It will require for dvc/mlem to have special command that prints metadata, but you can test it like this
create a gto.yaml with this content:

enrichments:
  - type: cli
    config:
      cmd: "echo {\"size\":10,\"hash\":\"aa\"}"
      info_type: gto.ext_dvc.DVCEnrichmentInfo

and run gto describe ""
the "" is needed as argument but should be empty since it's content is added to the end of echo

@aguschin aguschin closed this May 19, 2023
@skshetry skshetry deleted the feature/ext branch March 26, 2024 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants