Tests randomly fail with "Exception: fd:111: hPutBuf: resource vanished (Broken pipe)" (--test-option=-j1 workaround)

[anka-213]

When running the func-test test-suite locally, some of the tests randomly fail with the error:

Exception: fd:111: hPutBuf: resource vanished (Broken pipe)

This issue is a subset of issue #1430, but not on CI.

Your environment

Output of haskell-language-server --probe-tools or haskell-language-server-wrapper --probe-tools:

haskell-language-server version: 1.1.0.1 (GHC: 9.0.1) (PATH: /Users/anka/projekt/not-mine/haskell/haskell-language-server/dist-newstyle/build/x86_64-osx/ghc-9.0.1/haskell-language-server-1.1.0.1/x/haskell-language-server/build/haskell-language-server/haskell-language-server) (GIT hash: 8e03c8d08a51a69024c154bf93408a4595b45f8c)
Tool versions found on the $PATH
cabal:          3.4.0.0
stack:          2.7.1
ghc:            9.0.1

Which OS do you use:
MacOS

Steps to reproduce

I can't reliably reproduce it, but every few times when I run

cabal test func-test

some of the tests fail with the error above.

Actual behaviour

  plugin config
    hlint plugin enables
      changing hlintOn configuration enables or disables hlint diagnostics:                    OK (2.51s)
      changing hlint plugin configuration enables or disables hlint diagnostics:               OK (2.43s)
      adding hlint flags to plugin configuration removes hlint diagnostics:                    OK (2.95s)
      adding hlint flags to plugin configuration adds hlint diagnostics:                       FAIL
        Exception: fd:111: hPutBuf: resource vanished (Broken pipe)
    config parsing

More examples (this time ghc-8.10.4):

haskell-language-server
  commands
    are prefixed:                                                                              OK (3.07s)
    get de-prefixed:                                                                           OK (3.06s)
  completions
    works:                                                                                     FAIL
      Exception: fd:97: hPutBuf: resource vanished (Broken pipe)
    itemCompletion/resolve works:                                                              IGNORED
      no support for itemCompletion/resolve requests
    completes imports:                                                                         OK (5.17s)
    completes qualified imports:                                                               FAIL
      Exception: fd:135: hPutBuf: resource vanished (Broken pipe)
    completes with no prefix:                                                                  FAIL
      Exception: fd:129: hPutBuf: resource vanished (Broken pipe)
    strips compiler generated stuff from completions:                                          OK (4.75s)
    have implicit foralls on basic polymorphic types:                                          OK (4.99s)
    have implicit foralls with multiple type variables:                                        OK (4.93s)
    maxCompletions:                                                                            FAIL
      Exception: fd:99: hPutBuf: resource vanished (Broken pipe)
    contexts
      only provides type suggestions:                                                          FAIL
        Exception: fd:126: hPutBuf: resource vanished (Broken pipe)
      only provides value suggestions:                                                         FAIL
        Exception: fd:132: hPutBuf: resource vanished (Broken pipe)
      completes qualified type suggestions:                                                    FAIL
        Exception: fd:137: hPutBuf: resource vanished (Broken pipe)
    snippets
      work for argumentless constructors:                                                      OK (4.85s)
      work for polymorphic types:                                                              FAIL
        Exception: fd:123: hPutBuf: resource vanished (Broken pipe)
      work for complex types:                                                                  OK (5.05s)
      work for infix functions:                                                                OK (2.30s)
      work for infix functions in backticks:                                                   OK (2.36s)
      work for qualified infix functions:                                                      OK (2.41s)
      work for qualified infix functions in backticks:                                         OK (2.48s)
      respects lsp configuration:                                                              OK (2.69s)
      respects client capabilities:                                                            OK (2.65s)
  plugin config
    hlint plugin enables
      changing hlintOn configuration enables or disables hlint diagnostics:                    FAIL
        Exception: fd:129: hPutBuf: resource vanished (Broken pipe)
      changing hlint plugin configuration enables or disables hlint diagnostics:               FAIL
        Exception: fd:97: hPutBuf: resource vanished (Broken pipe)
      adding hlint flags to plugin configuration removes hlint diagnostics:                    FAIL
        Exception: fd:100: hPutBuf: resource vanished (Broken pipe)
      adding hlint flags to plugin configuration adds hlint diagnostics:                       OK (5.05s)
    config parsing
      empty object as user configuration should not send error logMessage:                     OK (0.66s)

Include debug information

Output from:

LSP_TEST_LOG_MESSAGES=1 LSP_TEST_LOG_STDERR=1 cabal test func-test 

It's 69000 lines, so I put it in a gist: https://gist.github.com/e12414f103af1c13768cd2735b608acb
Not sure how to reduce it, since the issue disappears when you only run a single test at a time.
Here's a shorter one ("only" 1500 lines) with only three tests running:
https://gist.github.com/d51856f6ff15b69434bf1f1a4b8ddb6e

[berberman]

Could you try running them in -j1?

[anka-213]

@berberman Do you mean changing this line?

haskell-language-server/test/utils/Test/Hls/Command.hs

Line 26 in 959da62

pure $ testExe ++ " --lsp -d -j4 -l test-logs/" ++ logFilePath

Or adding --test-option=-j1 to the cabal command?

[berberman]

Use --test-option=-j1. The -j4 in the code means threads used by ghcide, but here we want to run test cases sequentially.

[anka-213]

The --test-option=-j1 does make the symptoms disappear, as expected.

[anka-213]

I did however manage to reproduce the error previously while running only a single test, while Haskell Language Server was compiling a bunch of packages in the background, so maybe the system load matters (which would make sense for a race-condition).

[michaelpj]

Closing since I haven't seen any complaints about that recently

[soulomoon]

Still seeing them if the work load is high.

[soulomoon]

Potentially the same problem?
unisonweb/unison#3783