Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sdk): W&B to W&B Importer #6897

Merged
merged 206 commits into from
Mar 5, 2024
Merged

feat(sdk): W&B to W&B Importer #6897

merged 206 commits into from
Mar 5, 2024

Conversation

andrewtruong
Copy link
Contributor

@andrewtruong andrewtruong commented Jan 24, 2024

Description

Adds back tooling to migrate select data from one W&B instance to a second W&B instance.

Users can migrate:

  • Runs
  • Artifact (sequences)
  • Reports

Some data will not migrate exactly, e.g. dates will show up as at import-time instead of original run-time.

Testing

e2e tests for each of runs, artifacts, reports

Copy link

codecov bot commented Jan 24, 2024

Codecov Report

Attention: Patch coverage is 82.97534% with 214 lines in your changes are missing coverage. Please review.

Project coverage is 77.10%. Comparing base (754b06f) to head (9b96251).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #6897      +/-   ##
==========================================
- Coverage   78.97%   77.10%   -1.88%     
==========================================
  Files         492      495       +3     
  Lines       52508    53373     +865     
==========================================
- Hits        41470    41151     -319     
- Misses      10721    11897    +1176     
- Partials      317      325       +8     
Flag Coverage Δ
func 45.06% <0.32%> (-4.96%) ⬇️
system 65.55% <81.70%> (+1.38%) ⬆️
unit 58.29% <17.74%> (-0.54%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
wandb/apis/importers/__init__.py 100.00% <100.00%> (+100.00%) ⬆️
wandb/apis/public/files.py 92.95% <100.00%> (+6.39%) ⬆️
wandb/apis/importers/internals/protocols.py 97.56% <97.56%> (ø)
wandb/util.py 86.19% <66.66%> (+1.21%) ⬆️
wandb/apis/importers/internals/util.py 91.66% <91.66%> (ø)
wandb/apis/importers/validation.py 84.50% <84.50%> (ø)
wandb/apis/importers/internals/internal.py 90.71% <90.71%> (ø)
wandb/apis/importers/wandb.py 79.43% <79.43%> (ø)

... and 98 files with indirect coverage changes

@dmitryduev dmitryduev added this to the sdk-2024-02.1 milestone Jan 24, 2024
@andrewtruong andrewtruong changed the title (wip): Importer Fixes feat(sdk): Add W&B -> W&B Importer Jan 25, 2024
.circleci/config.yml Outdated Show resolved Hide resolved
tox.ini Outdated Show resolved Hide resolved
.circleci/config.yml Outdated Show resolved Hide resolved
tox.ini Outdated Show resolved Hide resolved
tox.ini Outdated Show resolved Hide resolved
@timoffex timoffex dismissed dmitryduev’s stale review March 5, 2024 20:05

Changes implemented.

@andrewtruong andrewtruong merged commit dfb4ca8 into main Mar 5, 2024
67 of 68 checks passed
@andrewtruong andrewtruong deleted the andrew/importers2 branch March 5, 2024 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants