Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(artifacts): prototype for models api #6205

Merged
merged 35 commits into from
Oct 25, 2023

Conversation

ibindlish
Copy link
Contributor

@ibindlish ibindlish commented Aug 31, 2023

Fixes

Description

Prototype for model APIs:

  • log_model() (note: max_objects param while adding reference is not exposed on log_artifact, hence not exposed for log_model)
  • use_model() - returns path to directory where we download the artifact files. If there's only 1 file, then we return the path to that file.
  • link_model()

Re. Notion doc

🤖 Generated by Copilot at 78991d8

Add new functions to wandb/apis/models.py for logging, using, and linking model artifacts. These functions simplify the workflow of working with model artifacts in wandb.

Testing

tmpdir = "../../test-examples"

log_model(tmpdir, model_name="test-log-model")
data_dir = use_model("testing-log-model:latest")
link_model(
    local_path=tmpdir,
    linked_model_name="testing-link-model-api",
    aliases=["testing-link"],
)
  • Small artifacts
  • Reference artifacts (** For references, tested using s3.)
  • Large artifacts
  • Large reference artifacts (tested by hackily adding max_objects=10**10 in wandb _log_artifact() for testing purposes; but from a user pov, till we set the default for max_objects as None they won't be able to log large artifact references)

Checklist

  • Include reference to internal ticket "Fixes WB-NNNNN" and/or GitHub issue "Fixes #NNNN" (if applicable)
  • Ensure PR title compliance with the conventional commits standards

🤖 Generated by Copilot at 78991d8

log_model, use_model
New functions for wandb users
Cutting edge in spring

@github-actions github-actions bot added cc-feat and removed cc-feat labels Aug 31, 2023
@codecov
Copy link

codecov bot commented Aug 31, 2023

Codecov Report

Merging #6205 (0be1264) into main (fc8781e) will increase coverage by 0.06%.
The diff coverage is 96.87%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #6205      +/-   ##
==========================================
+ Coverage   77.03%   77.10%   +0.06%     
==========================================
  Files         387      387              
  Lines       45015    45047      +32     
==========================================
+ Hits        34679    34732      +53     
+ Misses      10283    10262      -21     
  Partials       53       53              
Flag Coverage Δ
unittest 81.24% <96.87%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
wandb/__init__.py 90.51% <100.00%> (+0.25%) ⬆️
wandb/sdk/lib/module.py 100.00% <100.00%> (ø)
wandb/sdk/wandb_run.py 90.08% <95.65%> (+0.29%) ⬆️

... and 18 files with indirect coverage changes

@ibindlish ibindlish marked this pull request as ready for review September 29, 2023 20:28
@github-actions github-actions bot added cc-feat and removed cc-feat labels Oct 2, 2023
Copy link
Member

@dmitryduev dmitryduev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

wandb/sdk/wandb_run.py Show resolved Hide resolved
wandb/sdk/wandb_run.py Show resolved Hide resolved
wandb/sdk/wandb_run.py Show resolved Hide resolved
wandb/sdk/wandb_run.py Outdated Show resolved Hide resolved
wandb/sdk/wandb_run.py Outdated Show resolved Hide resolved
wandb/sdk/wandb_run.py Outdated Show resolved Hide resolved
wandb/sdk/wandb_run.py Show resolved Hide resolved
wandb/__init__.py Outdated Show resolved Hide resolved
@noaleetz
Copy link
Contributor

noaleetz commented Oct 25, 2023

Hey @ibindlish - who are we still waiting on a review on here? Oops - I see you are resolving a comment :) ignore me!

@ibindlish
Copy link
Contributor Author

Hey @ibindlish - who are we still waiting on a review on here?

@noaleetz just trying to get this comment resolved: #6205 (comment)

@ibindlish ibindlish merged commit 72421b5 into main Oct 25, 2023
79 checks passed
@ibindlish ibindlish deleted the ibindlish/artifacts/log-model-api branch October 25, 2023 18:21
@noaleetz
Copy link
Contributor

Note to self - will be released with next SDK release - release version: sdk-0.11.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants