Nixify io and memory tests #1538
Conversation
monacoremo
force-pushed
the
monacoremo-nix-io-memory-tests
branch
from
a1d424e
to
e5a165c
Compare
monacoremo
force-pushed
the
monacoremo-nix-io-memory-tests
branch
from
76efa2d
to
e29ef9b
Compare
nix/README.md
Outdated
| Some additional modules like `tests`, `docker` and `release` have large | ||
| dependencies that would need to be built before the shell becomes available, |
There was a problem hiding this comment.
Hmm.. couldn't we have the end-to-end tests(running io-tests.sh and memory-tests.sh) in another nix file(like io-tests.nix) and avoid building the static and profiled builds on nix-shell entrance?
I was really liking the DX of going inside nix-shell fast and being able to do postgrest-test-<tab>.
There was a problem hiding this comment.
Then we could do nix-shell --arg io-tests true in case we need to run the io tests. I think this would be better because those tests aren't run that frequently.
There was a problem hiding this comment.
You're right, that would be much nicer! Didn't think about that.
With the new commit the spec tests are available in the nix-shell by default and the other tests can be enabled with flags!
There was a problem hiding this comment.
@monacoremo Thanks! Much better now.
Just something I noticed. Looks like the nix memory tests consume less memory than the stack-test-memory.
nix-tests:
Running memory usage tests..
ok 1 - POST /rpc/leak?columns=blob: with a json key of 1M the memory usage(9,075,584 bytes) is less than 13M
ok 2 - POST /leak?columns=blob: with a json key of 1M the memory usage(9,193,768 bytes) is less than 13M
stack-memory-test:
ok 1 - POST /rpc/leak?columns=blob: with a json key of 1M the memory usage(12,122,480 bytes) is less than 13M
ok 2 - POST /leak?columns=blob: with a json key of 1M the memory usage(12,257,056 bytes) is less than 13M
Not sure why. But it's good nonetheless, the stack memory tests used to break from time to time.
There was a problem hiding this comment.
@steve-chavez Interesting... I don't have much experience with profiling haskell, might it be that some libraries are not included in the nix profiled build? Or different optimization levels?
| @@ -231,8 +234,10 @@ readSecretFromFile ascii.b64 'Base64 (ASCII)' | |||
| readSecretFromFile utf8.b64 'Base64 (UTF-8)' | |||
| readSecretFromFile binary.b64 'Base64 (binary)' | |||
|
|
|||
| readDbUriFromFile uri.noeol "(no EOL)" | |||
| readDbUriFromFile uri.txt "(EOL)" | |||
| eol=$'\x0a' | |||
There was a problem hiding this comment.
Nice. Thanks for taking the time to refactor these tests! Comments are really good too.
| pgrStopAll(){ pkill -f "$(stack path --profile --local-install-root)/bin/postgrest"; } | ||
|
|
||
| pgrStart(){ stack exec --profile -- postgrest test/memory-tests/config +RTS -p -h >/dev/null & pgrPID="$!"; } | ||
| pgrStart(){ postgrest test/memory-tests/config +RTS -p -h > /dev/null & pgrPID="$!"; } |
There was a problem hiding this comment.
Using the binary is much better. I remember I had some issues with stack there.
|
Looks good to merge now!
The tests are definitely hacky, never took the time to run shellcheck on them. It'd be great if you could further improve them on a later PR. |
* Include tests for io and memory in the Nix environment. * include spec tests in nix-shell by default * install the io and memory tests in CI
As discussed in #1535, this integrates the io and memory tests with the Nix setup.
A few notes:
with_tmp_dbwill never be in the same location) - otherwise I tried to keep the changes minimal