Skip to content

[Distributed] dist. isolation checking aware of Task and closures etc #39521

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 1, 2021

Conversation

ktoso
Copy link
Contributor

@ktoso ktoso commented Sep 30, 2021

Previously distributed isolation rules were too aggressive, or rather just not right, when facing closures and for example Task {} and Task.detached {}.

This PR fixes the simple mistakes, and also makes use of local knowlage to skip adding throwing behavior when we KNOW it cannot throw because it actually must have been a local reference. This is the case for isolated distributed actor references as well as calls on self but from OUTSIDE the actor's context, like for example in Task.detached {}.

This also converges CrossDistributedActorSelf with CrossActorSelf, but we'll finish that convergence in rdar://83713366

Resolves rdar://83609197

@ktoso ktoso requested review from DougGregor and kavon and removed request for DougGregor September 30, 2021 12:02
@ktoso
Copy link
Contributor Author

ktoso commented Sep 30, 2021

@swift-ci please smoke test

@ktoso
Copy link
Contributor Author

ktoso commented Sep 30, 2021

@swift-ci please build toolchain macOS platform

Copy link
Member

@kavon kavon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine overall. Just a naming nit.

@ktoso ktoso added the distributed Feature → concurrency: distributed actor label Oct 1, 2021
@ktoso ktoso force-pushed the wip-tasks-distribution-isolation branch 5 times, most recently from d82bae9 to 3e35829 Compare October 1, 2021 03:01
@ktoso ktoso force-pushed the wip-tasks-distribution-isolation branch from 3e35829 to e133dad Compare October 1, 2021 03:20
@ktoso
Copy link
Contributor Author

ktoso commented Oct 1, 2021

@swift-ci please smoke test and merge

@ktoso
Copy link
Contributor Author

ktoso commented Oct 1, 2021

@swift-ci please build toolchain macOS

Copy link
Member

@DougGregor DougGregor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a big improvement to the checking! I think we should go ahead and merge, and continue refactoring along the lines I've suggested.


// distributed func reference, that passes all checks, great!
continueToCheckingLocalIsolation = true;
tryMarkImplicitlyAsync(memberLoc, memberRef, context,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think our refactoring goal here should be to unify tryMarkImplicitlyAsync and tryMarkImplicitlyThrows, and give it enough information to determine when to add the qualifiers. For example, if isPotentiallyIsolated is true for a function on a distributed actor, we don't need the function to be distributed and should not mark it throws, but we might need to mark it async.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that sounds right... they are applied in similar situations, just need to feed it enough flags.

Noted!

// if it is explicitly marked distributed actor independent,
// it is synchronously accessible, no implicit async needed.
if (decl->getAttrs().hasAttribute<DistributedActorIndependentAttr>() ||
decl->getAttrs().hasAttribute<NonisolatedAttr>()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still suspect we can remove DistributedActorIndependentAttr and settle for implicitly-introduced NonisolatedAttr, but we can tackle that elsewhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup! DistributedActorIndependentAttr is a bit of debt from the early impls -- nowadays nonisolated is enough.

I'll make a specific radar to follow up on the removal


LLVM_FALLTHROUGH;
return diagnoseNonSendableTypesInReference(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This brings an interesting question to mind. If we're going through the distributed entry point, do we even need Sendable checking? Or is that only needed in the local case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it still matters -- the transport may use a different thread to serialize things (one of my impls does so), so the thread safety of the payloads is still useful

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even through I guess the only thing it does to them is invoke encode... 🤔

@swift-ci swift-ci merged commit ff3eeae into swiftlang:main Oct 1, 2021
@ktoso ktoso deleted the wip-tasks-distribution-isolation branch October 1, 2021 07:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
distributed Feature → concurrency: distributed actor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants