Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't mutate Symbol mixins in incremental resolver. #3725

Merged
merged 5 commits into from Nov 24, 2020

Conversation

jvilk-stripe
Copy link
Collaborator

Don't mutate Symbol mixins in incremental resolver.

Motivation

Watch an error flip on and off. This bug occurs because the incremental path is (incorrectly) recomputing mixins. However, we shouldn't be adding mixins at all on the incremental path.

We discovered this bug while trying to implement method rename on mixins.

Test plan

See included automated tests.

This also means we can avoid re-computing linearization, as it'll be a no-op.
@jvilk-stripe jvilk-stripe requested a review from a team as a code owner November 24, 2020 03:15
@jvilk-stripe jvilk-stripe requested review from jez and removed request for a team November 24, 2020 03:15
resolver/resolver.cc Outdated Show resolved Hide resolved
@@ -0,0 +1,26 @@
# typed: true
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fwiw i looked back and it looks like this test has failed for at least the past 6 months (i was curious to know whether it was a recent regression, and it doesn't look like it). Nice find!

resolver/resolver.cc Outdated Show resolved Hide resolved
@jvilk-stripe
Copy link
Collaborator Author

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR. When the build
finishes, I'll share with you whether or how it failed. Thanks!

Stripe employees can see the build result here:

https://go/builds/bui_IS4gxZyVypBrnB

computeLinearization(gs);
// NOTE: Linearization does not need to be recomputed as we do not mutate mixins() during incremental resolve.
if (debug_mode) {
for (auto i = 1; i < gs.classAndModulesUsed(); i++) {
Copy link
Collaborator

@jez jez Nov 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this "all the classes and modules that had been added by this file" not "all symbols globally" because this is runIncremental?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all symbols globally. We don't have a way to run symbol passes only over the subset of the symbol table that may have been impacted by the given files.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you think this debug mode check is going to make it like really long to debug on Stripe's codebase? should we make it respect skip_slow_enforce ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope. It's a linear walk of the symbol table checking a single bit per class. It should take a handful of ms I'd imagine.

Copy link
Collaborator

@jez jez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me but does this actually work?

Let's say we're running the fast path to check if we can, but the edit was to insert

include CompletelyNewModule

to a class in the file. Isn't this a completely normal edit that would fire an ENFORCE in debug mode?

@jvilk-stripe
Copy link
Collaborator Author

jvilk-stripe commented Nov 24, 2020

Let's say we're running the fast path to check if we can, but the edit was to insert
include CompletelyNewModule
to a class in the file. Isn't this a completely normal edit that would fire an ENFORCE in debug mode?

Discussed offline; this edit would perform a full end-to-end typecheck (it would not run incrementally) bc it mutates the symbol table.

@jez
Copy link
Collaborator

jez commented Nov 24, 2020

jez:t-unsafe: 4 minutes ago
:thinky_face: i left a comment. i’m confused how this passes
jvilk:oregon-tree: 4 minutes ago
Let's say we're running the fast path to check if we can, but the edit was to insert
include CompletelyNewModule
that edit cannot and will not take the incremental path
jvilk:oregon-tree: 3 minutes ago
we will hash the new file, compare with the old hash, hash is different => slow path
jez:t-unsafe: 3 minutes ago
how will we compute the hash?
jvilk:oregon-tree: 3 minutes ago
1 sec
:acknowledge:
1

jez:t-unsafe: 3 minutes ago
i thought we hashed the contents of global state after running the fast path
jvilk:oregon-tree: 2 minutes ago
each file has a content hash
jvilk:oregon-tree: 2 minutes ago
https://github.com/sorbet/sorbet/blob/8e17b530/main/pipeline/pipeline.cc#L1288
given that edit, we will index and resolve the file in an empty global state (ignoring errors). then we hash the resulting global state.
GitHubGitHub
sorbet/sorbet
A fast, powerful type checker designed for Ruby. Contribute to sorbet/sorbet development by creating an account on GitHub.
jez:t-unsafe: 1 minute ago
i see
jvilk:oregon-tree: 1 minute ago
so the hash before the edit should be different because a symbol's mixins are different.
jvilk:oregon-tree: 1 minute ago
hence we take the slow path
jez:t-unsafe: 1 minute ago
so it’s not runIncremental that runs to compute the edit’s hash, it’s all of resolver, including computeLinearization ?
New
jvilk:oregon-tree: < 1 minute ago
correct, on an empty fresh global state
jvilk:oregon-tree: < 1 minute ago
so it's on a per-file basis.
jez:t-unsafe: < 1 minute ago
👍 makes sense thanks!

@jvilk-stripe
Copy link
Collaborator Author

No measurable impact on Stripe typechecking runtime, and it typechecks clean. Will make the DEBUG_ONLY adjustment now.

@jvilk-stripe jvilk-stripe merged commit add1729 into master Nov 24, 2020
@jvilk-stripe jvilk-stripe deleted the jvilk/fix-query-by-loc-include-bug branch November 24, 2020 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants