New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix envtool run test
-run
and -skip
flags
#4101
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #4101 +/- ##
==========================================
- Coverage 76.22% 75.07% -1.16%
==========================================
Files 349 349
Lines 22170 22170
==========================================
- Hits 16899 16644 -255
- Misses 3960 4258 +298
+ Partials 1311 1268 -43 see 12 files with indirect coverage changes
Flags with carried forward coverage won't be shown. Click here to find out more. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried the fixes locally, I think it works!
@henvic this pull request has merge conflicts. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work!
shard, err := shardTestFuncs(index, total, all) | ||
// Then, shard all the tests but only run the ones that match the regex and that should | ||
// be run on the specific shard. | ||
shard, skipShard, err := shardTestFuncs(index, total, tests) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that shard
's content is not actually used
} | ||
|
||
if skip != "" { | ||
args = append(args, "-skip="+skip) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand what we are doing there. #3969's title is "Process --run
and --skip
in the envtool
itself". Somehow, we both process those flags ourselves, then ignore the result for the --run
processing, but pass -skip
.
What I think we should do is what the issue says. In more detail:
- get
--run
and--skip
values; - get a list of all tests;
- filter that list by applying
--run
and--skip
values; - run those tests by invoking
go test -run
– without-skip
because we already process--skip
's regular expression.
Description
The logic for filtering the tests should be applied before sharding and use the same logic as the Go standard testing library.
Closes #3969.
Readiness checklist
task all
, and it passed.@FerretDB/core
), Milestone (Next
), Labels, Project and project's Sprint fields.