Add `ParallelGuard` type to handle unwinding in parallel sections #115144

Zoxc · 2023-08-23T18:53:52Z

This adds a ParallelGuard type to handle unwinding in parallel sections instead of manually dealing with panics in each parallel operation. This also adds proper panic handling to the join operation.

cc @SparrowLii

rustbot · 2023-08-23T18:53:57Z

r? @compiler-errors

(rustbot has picked a reviewer for you, use r? to override)

tests/ui/const-generics/late-bound-vars/in_closure.rs

compiler-errors

r=me with or without

compiler-errors · 2023-08-23T20:58:02Z

compiler/rustc_data_structures/src/sync.rs

+/// continuing with unwinding. It's also used for the non-parallel code to ensure error message
+/// output match the parallel compiler for testing purposes.
+pub struct ParallelGuard {
+    panic: Lock<Option<Box<dyn Any + std::marker::Send + 'static>>>,


Should we implement Drop for ParallelGuard to make sure we never drop a panic on the floor without handling it? unwind will need to get reworked below if so.

I initially went for Drop, but I ended up preferring the explicit unwinding of the unwind calls.

I think having the explicit unwind calls is better. I just wonder if we should protect against missing unwind calls somehow... anyways, I don't feel particularly strongly about it.

compiler-errors · 2023-08-23T20:58:13Z

@bors delegate+

bors · 2023-08-23T20:58:15Z

✌️ @Zoxc, you can now approve this pull request!

If @compiler-errors told you to "r=me" after making some further change, please make that change, then do @bors r=@compiler-errors

Zoxc · 2023-08-23T23:08:55Z

@bors r=@compiler-errors

bors · 2023-08-23T23:08:57Z

📌 Commit d0a3e77e7b4c4c787451780e8abeb3eab4ec4353 has been approved by compiler-errors

It is now in the queue for this repository.

Zoxc · 2023-08-24T00:57:27Z

I've changed to the Lock to Mutex for compatibility with #111713, since Lock no longer use Sync / Send traits and can't be used with Rayon.

SparrowLii · 2023-08-24T01:02:24Z

Thanks!

bjorn3 · 2023-08-24T09:15:07Z

compiler/rustc_data_structures/src/sync.rs

-                    resume_unwind(panic);
-                }
+                let guard = ParallelGuard::new();
+                let r = t.into_iter().filter_map(|i| guard.run(|| map(i))).collect();


Wouldn't this non-deterministically pick one of the panics if multiple happen? For FatalErrorMarker that is fine as it is a ZST, but for other panics it may not be.

I don't see why that would be problematic. No panic would be guaranteed to "win" in a concurrent setting. FatalErrorMarker is the only exception (not technically a panic) that we need to handle.

Zoxc · 2023-08-26T14:44:14Z

I've changed this to use a function taking a closure, that way unwinding is guaranteed to be done if needed.

SparrowLii · 2023-08-29T12:09:00Z

@Zoxc I think you have to r=reviewer again

Zoxc · 2023-08-29T18:46:52Z

I think the changes is significant enough to warrant a re-review.

@rustbot ready

compiler-errors · 2023-08-30T01:55:12Z

@bors r+

bors · 2023-08-30T01:55:14Z

📌 Commit 9074443d3f84d7f7ae5e110f6f92100ddf8c344f has been approved by compiler-errors

It is now in the queue for this repository.

bors · 2023-08-30T10:44:48Z

☔ The latest upstream changes (presumably #111713) made this pull request unmergeable. Please resolve the merge conflicts.

Zoxc · 2023-08-30T17:07:43Z

@bors r=@compiler-errors

bors · 2023-08-30T17:18:29Z

📌 Commit c303c8a has been approved by compiler-errors

It is now in the queue for this repository.

bors · 2023-08-30T19:33:16Z

⌛ Testing commit c303c8a with merge 59a8294...

bors · 2023-08-30T21:21:20Z

☀️ Test successful - checks-actions
Approved by: compiler-errors
Pushing 59a8294 to master...

rust-timer · 2023-08-31T10:22:32Z

Finished benchmarking commit (59a8294): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.4%	[-0.4%, -0.4%]	1
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.9%	[0.5%, 1.3%]	2
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.6%	[-2.0%, -1.2%]	2
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.3%	[-2.0%, 1.3%]	4

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.5%	[-2.5%, -2.5%]	1
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 630.996s -> 633.863s (0.45%)
Artifact size: 316.52 MiB -> 316.71 MiB (0.06%)

rustbot assigned compiler-errors Aug 23, 2023

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 23, 2023

This comment has been minimized.

Sign in to view

compiler-errors reviewed Aug 23, 2023

View reviewed changes

tests/ui/const-generics/late-bound-vars/in_closure.rs Show resolved Hide resolved

compiler-errors approved these changes Aug 23, 2023

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 23, 2023

bjorn3 reviewed Aug 24, 2023

View reviewed changes

SparrowLii mentioned this pull request Aug 26, 2023

Tracking Issue for Parallel Rustc Front-end #113349

Open

28 tasks

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 29, 2023

compiler-errors approved these changes Aug 30, 2023

View reviewed changes

bors removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 30, 2023

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 30, 2023

Zoxc added 2 commits August 30, 2023 18:12

Make parallel! an expression

51eb842

Add ParallelGuard type to handle unwinding in parallel sections

b56acac

Zoxc added 3 commits August 30, 2023 18:13

Update failure status

2428054

Use Mutex to avoid issue with conditional locks

d36393b

Use a parallel_guard function to handle the parallel guard

c303c8a

Zoxc force-pushed the parallel-guard branch from 9074443 to c303c8a Compare August 30, 2023 16:18

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 30, 2023

bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 30, 2023

bors merged commit 59a8294 into rust-lang:master Aug 30, 2023
12 checks passed

rustbot added this to the 1.74.0 milestone Aug 30, 2023

Zoxc deleted the parallel-guard branch August 30, 2023 21:49

bors mentioned this pull request Aug 30, 2023

Increase parallelism in various locations #115003

Open

matthiaskrgr mentioned this pull request Nov 1, 2023

wrong exit code after ICE #116659

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `ParallelGuard` type to handle unwinding in parallel sections #115144

Add `ParallelGuard` type to handle unwinding in parallel sections #115144

Zoxc commented Aug 23, 2023

rustbot commented Aug 23, 2023

This comment has been minimized.

compiler-errors left a comment

compiler-errors Aug 23, 2023

Zoxc Aug 23, 2023

compiler-errors Aug 23, 2023 •

edited

compiler-errors commented Aug 23, 2023

bors commented Aug 23, 2023

Zoxc commented Aug 23, 2023

bors commented Aug 23, 2023

Zoxc commented Aug 24, 2023

SparrowLii commented Aug 24, 2023

bjorn3 Aug 24, 2023

Zoxc Aug 24, 2023

Zoxc commented Aug 26, 2023

SparrowLii commented Aug 29, 2023 •

edited

Zoxc commented Aug 29, 2023

compiler-errors commented Aug 30, 2023

bors commented Aug 30, 2023

bors commented Aug 30, 2023

Zoxc commented Aug 30, 2023

bors commented Aug 30, 2023

bors commented Aug 30, 2023

bors commented Aug 30, 2023

rust-timer commented Aug 31, 2023

Add ParallelGuard type to handle unwinding in parallel sections #115144

Add ParallelGuard type to handle unwinding in parallel sections #115144

Conversation

Zoxc commented Aug 23, 2023

rustbot commented Aug 23, 2023

This comment has been minimized.

compiler-errors left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

compiler-errors Aug 23, 2023 • edited

Choose a reason for hiding this comment

compiler-errors commented Aug 23, 2023

bors commented Aug 23, 2023

Zoxc commented Aug 23, 2023

bors commented Aug 23, 2023

Zoxc commented Aug 24, 2023

SparrowLii commented Aug 24, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Zoxc commented Aug 26, 2023

SparrowLii commented Aug 29, 2023 • edited

Zoxc commented Aug 29, 2023

compiler-errors commented Aug 30, 2023

bors commented Aug 30, 2023

bors commented Aug 30, 2023

Zoxc commented Aug 30, 2023

bors commented Aug 30, 2023

bors commented Aug 30, 2023

bors commented Aug 30, 2023

rust-timer commented Aug 31, 2023

Overall result: ✅ improvements - no action needed

Add `ParallelGuard` type to handle unwinding in parallel sections #115144

Add `ParallelGuard` type to handle unwinding in parallel sections #115144

compiler-errors Aug 23, 2023 •

edited

SparrowLii commented Aug 29, 2023 •

edited