Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review masking and add traces when things don't cancel timely #2768

Merged
merged 14 commits into from Mar 11, 2022

Conversation

pepeiborra
Copy link
Collaborator

No description provided.

@pepeiborra
Copy link
Collaborator Author

pepeiborra commented Mar 9, 2022

Who reviews changes made by the code owner? I need a volunteer once again, thanks @wz1000 @michaelpj @eddiemundo

Copy link
Collaborator

@wz1000 wz1000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable.

Copy link
Collaborator

@michaelpj michaelpj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the first time I've looked at the AIO stuff. It makes me a little nervous: this stuff is fiendishly difficult, can we definitely not get away with an existing library?

@@ -109,8 +109,7 @@ modifyFileExists :: IdeState -> [(NormalizedFilePath, FileChangeType)] -> IO ()
modifyFileExists state changes = do
FileExistsMapVar var <- getIdeGlobalState state
-- Masked to ensure that the previous values are flushed together with the map update
-- update the map
mask_ $ join $ atomicallyNamed "modifyFileExists" $ do
join $ mask_ $ atomicallyNamed "modifyFileExists" $ do
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change doesn't make sense to me. I think this just makes the mask apply to the atomic execution of the STM block, which is already atomic. Previously it ensured that we would perform the effects of the returned action as well without being interrupted. But now I think it doesn't do that, and indeed maybe does nothing?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It still prevents the STM transaction from being interrupted by an edit, which is important to accurately track file dirtiness.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we can end up not calling recordDirtyKeys, since that happens in the IO action afterwards. Is that important? Maybe the comment could say.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The IO is just logging, recordDirtyKeys mutates the collection inside the STM transaction. That said I appreciate how this is confusing and perhaps the mask_ was better at the top level for clarity

ghcide/src/Development/IDE/Core/Shake.hs Outdated Show resolved Hide resolved
@@ -224,6 +226,7 @@ updateReverseDeps
-> [Key] -- ^ Previous direct dependencies of Id
-> HashSet Key -- ^ Current direct dependencies of Id
-> IO ()
-- mask to ensure that all the reverse dependencies are updated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one I wonder if we could avoid by pushing the scope of the STM transaction out a bit?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

STM transactions can still be interrupted by exceptions, so the question is how badly do we want to record the reverse deps. The answer is very badly - failure to do so will lead to unsoundness, specially if this is the first build.

EDIT: oh, correcting myself, since we don't mark the build as done until after the reverse keys have updated, I don't think that the mask is needed at all.

unless (null asyncs) $ do
let loop = forever $ do
sleep 10
traceM "cleanupAsync: waiting for asyncs to finish"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want to use Debug.Trace here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are seeing this trace, then most likely hls-graph is broken.

Plus, I didn't want to thread a logger around....

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're in IO, you could just print to stderr. It's a small difference, but I do think not having Debug.Trace in production code is good.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a fair point

hls-graph/src/Development/IDE/Graph/Internal/Database.hs Outdated Show resolved Hide resolved
hls-graph/src/Development/IDE/Graph/Internal/Database.hs Outdated Show resolved Hide resolved
hls-graph/src/Development/IDE/Graph/Internal/Database.hs Outdated Show resolved Hide resolved
@pepeiborra
Copy link
Collaborator Author

This is the first time I've looked at the AIO stuff. It makes me a little nervous: this stuff is fiendishly difficult, can we definitely not get away with an existing library?

AIO is just a reader monad to keep track of the asyncs in flight, right? Surely you mean something else

@michaelpj
Copy link
Collaborator

I guess I mean the functions operating on AIO. Perhaps they would exist in an equally gnarly form without AIO, I don't know. I trust your ability to write them, I don't trust my ability to review whether the masking is sensible :D

@pepeiborra
Copy link
Collaborator Author

AIO is just a thin wrapper around IO/Async. The gnarliness existed before AIO, it's evil magic contributed by @ndmitchell who, in turn, blames @simonmar.

@pepeiborra pepeiborra merged commit fbbf76b into master Mar 11, 2022
@pepeiborra pepeiborra deleted the fix-excessive-masking branch March 11, 2022 11:37
July541 pushed a commit to July541/haskell-language-server that referenced this pull request Mar 30, 2022
…l#2768)

* Review masking and add traces when things don't cancel timely

* fixup

* use sleep consistently

* redundant imports

* hlints

* fix 9.2 build

* Fix 9.2 build for real

* remove unnecessary polymorphism

* Avoid spawning loop async unnecessrily

* flush asyncs ref

* Add comments and apply @michaelpj suggestions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants