Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

benchmarks: update tests for dvc 3.x #9608

Merged
merged 2 commits into from
Jun 15, 2023

Conversation

pmrowla
Copy link
Contributor

@pmrowla pmrowla commented Jun 15, 2023

Thank you for the contribution - we'll try to review it as soon as possible. πŸ™

Fixes iterative/dvc-bench#461

@pmrowla pmrowla self-assigned this Jun 15, 2023
@pmrowla pmrowla added the testing Related to the tests and the testing infrastructure label Jun 15, 2023
@codecov
Copy link

codecov bot commented Jun 15, 2023

Codecov Report

Patch coverage has no change and project coverage change: -0.26 ⚠️

Comparison is base (a6e49a7) 90.52% compared to head (d561a43) 90.26%.

❗ Current head d561a43 differs from pull request most recent head 4321a3f. Consider uploading reports for the commit 4321a3f to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9608      +/-   ##
==========================================
- Coverage   90.52%   90.26%   -0.26%     
==========================================
  Files         471      471              
  Lines       36048    36048              
  Branches     5186     5186              
==========================================
- Hits        32632    32539      -93     
- Misses       2820     2893      +73     
- Partials      596      616      +20     
Impacted Files Coverage Ξ”
...c/testing/benchmarks/cli/commands/test_exp_show.py 0.00% <0.00%> (ΓΈ)

... and 25 files with indirect coverage changes

β˜” View full report in Codecov by Sentry.
πŸ“’ Do you have feedback about the report comment? Let us know in this issue.

@pmrowla pmrowla changed the title benchmarks: update test_exp_show to use example-get-started benchmarks: update tests for dvc 3.x Jun 15, 2023
@@ -3,7 +3,7 @@ def test_update(bench_dvc, tmp_dir, dvc, make_dataset):
dataset = make_dataset(
cache=False, files=True, dvcfile=False, commit=False, remote=False
)
dvc.imp_url(str(dataset), "new")
bench_dvc("import-url", str(dataset), "new")
Copy link
Contributor Author

@pmrowla pmrowla Jun 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't use the dvc fixture for benchmark tests any more. Using dvc-bench with DVC 3.x will generate 3.x .dvc and dvc.lock files that cannot be read in older versions.

What we really need is an equivalent fixture for test setup that uses the venv DVC, but bench_dvc works for now

@pmrowla pmrowla merged commit fa14507 into iterative:main Jun 15, 2023
17 of 18 checks passed
@pmrowla pmrowla deleted the benchmarks-exp-show branch June 15, 2023 01:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing Related to the tests and the testing infrastructure
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

test_exp_show uses checkpoints
1 participant