Multiple test failures in Gentoo's network sandbox

[hololeap]

I am getting a lot of test failures from the hls-eval-plugin test suite using 55d9024:

62 out of 65 tests failed (272.92s)

Full build log is here.

Your environment

GHC 9.0.2 on Gentoo Linux. All packages are installed through the system package manager.

I am building the following packages from 55d9024:

• ghcide
• haskell-language-server
• hls-alternate-number-format-plugin
• hls-change-type-signature-plugin
• hls-code-range-plugin
• hls-eval-plugin
• hls-floskell-plugin
• hls-fourmolu-plugin
• hls-graph
• hls-hlint-plugin
• hls-plugin-api
• hls-rename-plugin
• hls-retrie-plugin
• hls-test-utils

I am building the rest from Hackage with the following versions (let me know if you need more):

• aeson-2.0.3.0
• data-default-0.7.1.1
• diff-0.4.0
• dlist-1.0
• extra-1.7.10
• ghc-paths-0.1.0.12
• hashable-1.4.0.2
• lens-5.1.1
• lsp-1.5.0.0
• lsp-types-1.5.0.0
• megaparsec-9.2.1
• parser-combinators-1.3.0
• pretty-simple-4.1.1.0
• quickcheck-2.14.2
• safe-exceptions-0.1.7.1
• unliftio-0.2.18
• unordered-containers-0.2.17.0

[hololeap]

Are these tests passing elsewhere? If so, can someone give me a link to where they are passing?

[pepeiborra]

They are passing in the Github CI. link

[wz1000]

Perhaps you want to test using the environment variables LSP_TEST_LOG_STDERR=1 and LSP_TEST_LOG_MESSAGES=1 to get more information on why it fails.

[hololeap]

@wz1000, thanks for the hint about the environment variables.

It looks like this has something to do with Gentoo's build sandboxing. All the tests pass when run outside the sandbox.

I am seeing blocked network calls being logged. It looks like cabal is being run as a part of the test, which is trying to download something.

{
    "jsonrpc": "2.0",
    "method": "textDocument/publishDiagnostics",
    "params": {
        "diagnostics": [
            {
                "message": "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n",
                "range": {
                    "end": {
                        "character": 0,
                        "line": 1
                    },
                    "start": {
                        "character": 0,
                        "line": 0
                    }
                },
                "severity": 1,
                "source": "cradle"
            }
        ],
        "uri": "file:///var/tmp/portage/dev-haskell/hls-eval-plugin-1.3.0.0/work/hls-eval-plugin-1.3.0.0/test/testdata/TLocalImport.hs",
        "version": 0
    }
}

---

Full build log with LSP_TEST_LOG_STDERR=1 and LSP_TEST_LOG_MESSAGES=1 is here.

[hololeap]

It looks like I'm running into the same thing for hls-module-rename-plugin-1.1.0.0:

{
    "jsonrpc": "2.0",
    "method": "textDocument/publishDiagnostics",
    "params": {
        "diagnostics": [
            {
                "message": "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n",
                "range": {
                    "end": {
                        "character": 0,
                        "line": 1
                    },
                    "start": {
                        "character": 0,
                        "line": 0
                    }
                },
                "severity": 1,
                "source": "cradle"
            }
        ],
        "uri": "file:///var/tmp/portage/dev-haskell/hls-module-name-plugin-1.1.0.0/work/hls-module-name-plugin-1.1.0.0/test/testdata/canonicalize/Lib/A.hs",
        "version": 0
    }
}

FAIL (16.59s)
    Received an unexpected message from the server:
    Was parsing: Pattern match failure in do expression at test/Main.hs:48:7-38
    But the last message received was:
    {
        "id": 2,
        "jsonrpc": "2.0",
        "result": []
    }
    Use -p '/Fix#3047/' to rerun this test only.

1 out of 6 tests failed (24.41s)
Test suite tests: FAIL

Full build log

This test passes outside of the sandbox.

[hololeap]

There are more seemingly-related test failures, this time from ghcide:

1. typecheck-all-parents-of-interest

   FAIL (2.13s)
         test/exe/Main.hs:724:
         Incorrect number of diagnostics for Uri {getUri = "file:///var/tmp/portage/dev-haskell/ghcide-1.8.0.0/temp/extra-dir-8862631353299/A.hs"}, expected [] but got [Diagnostic {_range = Range {_start = Positio
   n {_line = 5, _character = 4}, _end = Position {_line = 5, _character = 5}}, _severity = Just DsError, _code = Just (InR "-Wdeferred-type-errors"), _source = Just "typecheck", _message = "\8226 Couldn't match e
   xpected type \8216Int\8217 with actual type \8216Bool\8217\n\8226 In the expression: y\n  In an equation for \8216x\8217: x = y", _tags = Nothing, _relatedInformation = Nothing},Diagnostic {_range = Range {_sta
   rt = Position {_line = 5, _character = 4}, _end = Position {_line = 5, _character = 5}}, _severity = Just DsError, _code = Just (InR "-Wdeferred-type-errors"), _source = Just "typecheck", _message = "\8226 Coul
   dn't match expected type \8216Int\8217 with actual type \8216Bool\8217\n\8226 In the expression: y\n  In an equation for \8216x\8217: x = y", _tags = Nothing, _relatedInformation = Nothing}]
         Use -p '/typecheck-all-parents-of-interest/' to rerun this test only.
   

2. simple plugin

   FAIL (7.77s)
       test/exe/Main.hs:1196:
       Could not find (DsError,(9,15),"Variable not in scope: c",Nothing) in [Diagnostic {_range = Range {_start = Position {_line = 0, _character = 0}, _end = Position {_line = 1, _character = 0}}, _severity = Just DsError, _code = Nothing, _source = Just "cradle", _message = "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n", _tags = Nothing, _relatedInformation = Nothing}]
       Use -p '/simple plugin/' to rerun this test only.
   

3. notification handlers run in priority order

   FAIL
         Exception: Timed out waiting to receive a message from the server.
         Use -p '/notification handlers run in priority order/' to rerun this test only.
   

4. ignore-fatal-warning

   FAIL (9.15s)
           test/exe/Main.hs:2512:
           Got unexpected diagnostics for Uri {getUri = "file:///var/tmp/portage/dev-haskell/ghcide-1.8.0.0/temp/extra-dir-8862631353468/IgnoreFatal.hs"} got [Diagnostic {_range = Range {_start = Position {_line = 0, _character = 0}, _end = Position {_line = 1, _character = 0}}, _severity = Just DsError, _code = Nothing, _source = Just "cradle", _message = "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n", _tags = Nothing, _relatedInformation = Nothing}]
           Use -p '/ignore-fatal-warning/' to rerun this test only.
   

5. simple-multi-test

   FAIL (7.49s)
           test/exe/Main.hs:2532:
           A should typecheck
           Use -p '$0=="ghcide.cradle.multi.simple-multi-test"' to rerun this test only.
   

6. simple-multi-test2

   FAIL
           Exception: Timed out waiting to receive a message from the server.
           Last message received:
           {
               "jsonrpc": "2.0",
               "method": "window/logMessage",
               "params": {
                   "message": "Live bytes: 0.00MB Heap size: 0.00MB",
                   "type": 3
               }
           }
           Use -p '/simple-multi-test2/' to rerun this test only.
   

7. simple-multi-test3

   FAIL
           Exception: Timed out waiting to receive a message from the server.
           Last message received:
           {
               "jsonrpc": "2.0",
               "method": "window/logMessage",
               "params": {
                   "message": "Live bytes: 0.00MB Heap size: 0.00MB",
                   "type": 3
               }
           }
           Use -p '/simple-multi-test3/' to rerun this test only.
   

8. simple-multi-def-test

   FAIL
           Exception: Timed out waiting to receive a message from the server.
           Last message received:
           {
               "jsonrpc": "2.0",
               "method": "window/logMessage",
               "params": {
                   "message": "Live bytes: 0.00MB Heap size: 0.00MB",
                   "type": 3
               }
           }
           Use -p '/simple-multi-def-test/' to rerun this test only.
   

9. simple-subdirectory

   FAIL (5.22s)
           test/exe/Main.hs:2520:
           Could not find (DsWarning,(2,0),"Top-level binding",Nothing) in [Diagnostic {_range = Range {_start = Position {_line = 0, _character = 0}, _end = Position {_line = 1, _character = 0}}, _severity = Just DsError, _code = Nothing, _source = Just "cradle", _message = "Error when calling cabal exec -v0 -- ghc --print-libdir\n\ndieVerbatim: user error (cabal: '/usr/bin/curl' exited with an error:\ncurl: (6) Could not resolve host: hackage.haskell.org\n)\n\n", _tags = Nothing, _relatedInformation = Nothing}]
           Use -p '/simple-subdirectory/' to rerun this test only.
   

10. ghcide command line.works

  ghcide command line
    works:                                                                            FAIL (4.47s)
      test/exe/Main.hs:2814:
      expected: ExitSuccess
       but got: ExitFailure 1
      Use -p '/ghcide command line.works/' to rerun this test only.

13 out of 344 tests failed (2982.71s)

(Three tests fail regardless of the network-sandbox. See #3221.)

---

Full build log (raw file is ~15MB)