Conversation
efiop
force-pushed
the
bench
branch
13 times, most recently
from
October 3, 2021 18:15
73852d9 to
d690679
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
efiop
force-pushed
the
bench
branch
2 times, most recently
from
October 3, 2021 22:21
6b1dde1 to
b2be588
Compare
Contributor
Author
|
Ok, posting full benchmark tables as comments is too noisy for PRs, the last version runs regression checks for individual tests, which feels much nicer to me for PRs here. Also it is clear that we shouldn't run all possible benchmarks for PRs here, but rather keep key ones (e.g. init/add/etc, but not tutorial ones), so I'll try to condense those a little bit. |
efiop
force-pushed
the
bench
branch
4 times, most recently
from
October 4, 2021 13:42
b138b4d to
3652f55
Compare
skshetry
reviewed
Oct 4, 2021
efiop
force-pushed
the
bench
branch
3 times, most recently
from
October 4, 2021 21:52
a5de5fd to
9a29d40
Compare
efiop
commented
Oct 4, 2021
| python-version: 3.7 | ||
| - uses: iterative/dvc-bench@master | ||
| with: | ||
| pytest_options: "${{ github.event_name == 'workflow_dispatch' && '' || '-k test_init -k test_help' }}" |
Contributor
Author
There was a problem hiding this comment.
Could create a special mark for similar tests in dvc-bench (e.g. micro), but will leave it for later.
Contributor
Author
|
Ok, let's try this out. |
Collaborator
|
It seems that only one test ( |
Contributor
Author
|
@skshetry Oops, looking into it... Thank you! 🙏 |
skshetry
added a commit
to endremborza/dvc
that referenced
this pull request
Oct 8, 2021
* master: (47 commits) make dependabot updates work (treeverse#6760) move defaults to the class level (treeverse#6762) build(deps): Bump dvclive from 0.3.0 to 0.4.0 (treeverse#6753) move requirements to setup.cfg (treeverse#6758) gha: cancel previous workflows (treeverse#6756) pyinstaller: hooks: remove webhdfs webhdfs: migrate to fsspec (treeverse#6662) gha: benchmarks: fix pytest flags (treeverse#6749) hdfs: migrate to fsspec (treeverse#6604) Pin dvclive to 0.3.0 command: Should not consume 'print' to output default remote info (treeverse#6650) command: output prettify json in tty (treeverse#6743) gha: run benchmarks on PRs (treeverse#6733) Run on Python3.10 (treeverse#6745) Fix dvclive running subdir (treeverse#6740) exp init: only ask for that are not provided in an interactive mode (treeverse#6739) move exp init logic to dvc.repo.experiments.init (treeverse#6738) build(deps): Bump jaraco-windows from 5.6.0 to 5.7.0 (treeverse#6735) build(deps): Bump pytest-cov from 2.12.1 to 3.0.0 (treeverse#6736) Bump typing_extensions to >=3.7.4 (treeverse#6731) (treeverse#6732) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Qucik&dirty way to run dvc-bench for our PRs to help with things like #6707
Need to clarify that this is not all-in-one regression detection, but rather this is specific to test regressions within one PR. We need a similar thing for specific dvc versions to detect slow regressions that accumulate over a long period of time, but that is out of scope for this PR.