sporadic postgrest-test-spec failure in Feature.Query.PlanSpec

[robx]

Environment

• PostgreSQL version: 15 (via nix withPg)
• PostgREST version: current HEAD
• Operating system: macOS 12.5 (x86)

Description of issue

I often see one test failure when running postgrest-test-spec.

Feature.Query.PlanSpec.spec
  read table/view plan
    outputs blocks info when using the buffers option [✘]

This fails because the value "Shared Hit Blocks" is non-zero, contrary to expectation. The precise value varies.

Failures:

  test/spec/Feature/Query/PlanSpec.hs:68:20: 
  1) Feature.Query.PlanSpec.spec, read table/view plan, outputs blocks info when using the buffers option
       expected: Just (Object (fromList [("Local Dirtied Blocks", Number 0.0), ("Local Hit Blocks", Number 0.0), ("Local Read Blocks", Number 0.0), ("Local Written Blocks", Number 0.0), ("Shared Dirtied Blocks", Number 0.0), ("Shared Hit Blocks", Number 0.0), ("Shared Read Blocks", Number 0.0), ("Shared Written Blocks", Number 0.0), ("Temp Read Blocks", Number 0.0), ("Temp Written Blocks", Number 0.0)]))
        but got: Just (Object (fromList [("Local Dirtied Blocks", Number 0.0), ("Local Hit Blocks", Number 0.0), ("Local Read Blocks", Number 0.0), ("Local Written Blocks", Number 0.0), ("Shared Dirtied Blocks", Number 0.0), ("Shared Hit Blocks", Number 39.0), ("Shared Read Blocks", Number 0.0), ("Shared Written Blocks", Number 0.0), ("Temp Read Blocks", Number 0.0), ("Temp Written Blocks", Number 0.0)]))

  To rerun use: --match "/Feature.Query.PlanSpec.spec/read table/view plan/outputs blocks info when using the buffers option/"

Randomized with seed 1855785399

Finished in 0.0948 seconds

[robx]

Hmm, this shows in CI now on the PR #2663. I think I got it to happen reliably on main, but there's a chance a hasql-pool updated slipped in and that causes the test failures...

[steve-chavez]

Hm, I thought these values would be stable when doing the tests. Maybe they should be changed to just test the keys and not the values..

[robx]

Hm, I thought these values would be stable when doing the tests. Maybe they should be changed to just test the keys and not the values..

They seem to have been around for a while without causing issues, curious they would show up now. I'll prepare a PR to just update to the latest upstream hasql-pool to see if that's involved here. (Though it seems the pool library shouldn't affect this, right?)

[robx]

So the test fails pretty reliably if I run only that test, via

$ postgrest-test-spec --match "/Feature.Query.PlanSpec.spec/read table/view plan/outputs blocks info when using the buffers option/"

Which kind of makes sense, since then postgresql is more likely to be caching something. (Though I'm a bit fuzzy to what extent this spins up its own isolated postgres instance...)

Anyway, @steve-chavez if you get a chance could you run that command a couple times in your dev setup to see if it's the same for you?

In general I agree that it's probably best not to test these values, since there's not really any apparent reason to expect them to be stable.

[steve-chavez]

@robx Hm, how does that command run for you? I get:

$ postgrest-test-io --match "/Feature.Query.PlanSpec.spec/read table/view plan/outputs blocks info when using the buffers option/"

Warning: The package list for 'hackage.haskell.org' does not exist. Run 'cabal
update' to download it.
Resolving dependencies...
Up to date
Warning: The package list for 'hackage.haskell.org' does not exist. Run 'cabal
update' to download it.
Resolving dependencies...
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --match /Feature.Query.PlanSpec.spec/read table/view plan/outputs blocks info when using the buffers option/
  inifile: None
  rootdir: /home/stevechavez/Projects/postgrest

Temporary directory kept at: /run/user/1000/postgrest/postgrest-with-postgresql-15-xbL

[robx]

Oops, typo, sorry. It's postgrest-test-spec.

[steve-chavez]

No problem, it fails for me too:

  test/spec/Feature/Query/PlanSpec.hs:68:20:
  1) Feature.Query.PlanSpec.spec, read table/view plan, outputs blocks info when using the buffers option
       expected: Just (Object (fromList [("Local Dirtied Blocks", Number 0.0), ("Local Hit Blocks", Number 0.0), ("Local Read Blocks", Number 0.0), ("Local Written Blocks", Number 0.0), ("Shar
ed Dirtied Blocks", Number 0.0), ("Shared Hit Blocks", Number 0.0), ("Shared Read Blocks", Number 0.0), ("Shared Written Blocks", Number 0.0), ("Temp Read Blocks", Number 0.0), ("Temp Written
Blocks", Number 0.0)]))
        but got: Just (Object (fromList [("Local Dirtied Blocks", Number 0.0), ("Local Hit Blocks", Number 0.0), ("Local Read Blocks", Number 0.0), ("Local Written Blocks", Number 0.0), ("Shar
ed Dirtied Blocks", Number 0.0), ("Shared Hit Blocks", Number 40.0), ("Shared Read Blocks", Number 0.0), ("Shared Written Blocks", Number 0.0), ("Temp Read Blocks", Number 0.0), ("Temp Written
 Blocks", Number 0.0)]))

Which kind of makes sense, since then postgresql is more likely to be caching something

Yeah, so then we shouldn't be testing the values.