Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

coverage: Anonymize line numbers in run-coverage test snapshots #114875

Merged
merged 1 commit into from
Aug 17, 2023
Merged

coverage: Anonymize line numbers in run-coverage test snapshots #114875

merged 1 commit into from
Aug 17, 2023

Conversation

Zalathar
Copy link
Contributor

@Zalathar Zalathar commented Aug 16, 2023 •
edited

LLVM's coverage reporter always prints line numbers in its coverage reports.

For testing purposes this is slightly inconvenient, because it means that adding or removing a line in a test file causes all subsequent lines in the snapshot to change. That makes it harder to see the actually meaningful changes in the re-blessed snapshot.

This change fixes that by adding another normalization pass that replaces all line numbers in the coverage reports with LL, which is similar to what UI tests tell the compiler to do when emitting line numbers in error messages.

@rustbot
Copy link
Collaborator

rustbot commented Aug 16, 2023

r? @ozkanonur

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) labels Aug 16, 2023
@Zalathar Zalathar force-pushed the line-numbers branch 2 times, most recently from 9ec72a0 to 89155a8 Compare August 16, 2023 04:30
@onur-ozkan
Copy link
Member

Looks good to me, thanks!

rollup=never for smoother rollbacks in case of any regressions or issues.

@bors r+

@bors
Copy link
Contributor

bors commented Aug 16, 2023

📌 Commit 89155a84b9a17413e14b7c2f121629405670adf2 has been approved by ozkanonur

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 16, 2023
@onur-ozkan
Copy link
Member

@bors rollup=never

@bors
Copy link
Contributor

bors commented Aug 16, 2023

⌛ Testing commit 89155a84b9a17413e14b7c2f121629405670adf2 with merge af6256d2309a27e05df159fef82a8b236d083870...

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@bors
Copy link
Contributor

bors commented Aug 16, 2023

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 16, 2023
@onur-ozkan
Copy link
Member

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 16, 2023
@Zalathar
Copy link
Contributor Author

Ah, I forgot to update the run-coverage-rustdoc suite as well, since I rarely need to run it. Fixed.

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 16, 2023
@Zalathar
coverage: Anonymize line numbers in run-coverage test snapshots
bfb1654 
This makes the test snapshots less sensitive to lines being added/removed.
@Zalathar Zalathar mentioned this pull request Aug 17, 2023
Draft
@onur-ozkan
Copy link
Member

@bors retry

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 17, 2023
@onur-ozkan
Copy link
Member

@bors retry

Didn't work, strange..

@bors r+

@bors
Copy link
Contributor

bors commented Aug 17, 2023

📌 Commit bfb1654 has been approved by ozkanonur

It is now in the queue for this repository.

@bors
Copy link
Contributor

bors commented Aug 17, 2023

⌛ Testing commit bfb1654 with merge aa864a7...

@bors
Copy link
Contributor

bors commented Aug 17, 2023

☀️ Test successful - checks-actions
Approved by: ozkanonur
Pushing aa864a7 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 17, 2023
@bors bors merged commit aa864a7 into rust-lang:master Aug 17, 2023
12 checks passed
@rustbot rustbot added this to the 1.73.0 milestone Aug 17, 2023
@Zalathar Zalathar deleted the line-numbers branch August 17, 2023 07:22
@Zalathar Zalathar mentioned this pull request Aug 17, 2023
Merged
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (aa864a7): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)		 2.2% [2.2%, 2.2%] 1
Regressions ❌
(secondary)		 2.2% [2.2%, 2.2%] 1
Improvements ✅
(primary)		 - - 0
Improvements ✅
(secondary)		 -1.1% [-1.1%, -1.1%] 1
All ❌✅ (primary) 2.2% [2.2%, 2.2%] 1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 634.855s -> 634.832s (-0.00%)
Artifact size: 346.99 MiB -> 347.01 MiB (0.01%)

Zalathar added a commit to Zalathar/rust that referenced this pull request Aug 26, 2023
@Zalathar
Tidy up some awkwardly-placed comments in tests
4da38c3 
Prior to rust-lang#114875, these tests were very sensitive to lines being added/removed,
so the migration to `run-coverage` in rust-lang#112300 tried hard to avoid disturbing
the existing line numbers. That resulted in some awkward reshuffling when
certain comments/directives needed to be added or moved.

Now that we don't have to worry about preserving line numbers, we can rearrange
those comments into a more conventional layout.
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Aug 27, 2023
@matthiaskrgr
Rollup merge of rust-lang#114924 - Zalathar:tidy-tests, r=Mark-Simula…
be416b9 
…crum

coverage: Tidy up `run-coverage` tests in several small ways

Prior to rust-lang#114875 (anonymized line numbers), these tests were very sensitive to lines being added/removed, since doing so would change the line numbers in every subsequent line of output. Therefore they ended up with a few small style issues that weren't worth the hassle of fixing.

Now that we can freely rearrange lines without needing to re-bless the entire output, we can tidy up the tests in various trivial ways.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@onur-ozkan onur-ozkan

Labels
A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) A-testsuite Area: The testsuite used to check the correctness of rustc merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Milestone
1.73.0
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@Zalathar @rustbot @onur-ozkan @bors @rust-log-analyzer @rust-timer