Build bug exposed by testing-tweaks branch

[yminsky]

I'm working on improving the testing chapter, and have tripped over some weird build bug. Maybe something with Dune? Here's where I'm working:

https://github.com/realworldocaml/book/blob/testing-tweaks/book/testing/README.md

I'm getting in to a state where dune is stuck in some kind of infinite loop.

After a clean, I get this error when I run dune runtest from the book/testing directory.

sh alias book/testing/examples/bigger_quickcheck_test (exit 1)
(cd _build/default/book/testing/examples && /usr/bin/sh -c 'dune build @all @runtest --root ./correct/bigger_quickcheck_test')
Entering directory 'correct/bigger_quickcheck_test'
File "test.ml", line 1, characters 5-9:
1 | open Core
         ^^^^
Error: Unbound module Core
File "dune", line 1, characters 0-97:
1 | (library
2 |  (name foo)
3 |  (libraries base stdio core)
4 |  (preprocess
5 |   (pps ppx_jane))
6 |  (inline_tests))
Error: File unavailable:
/home/yminsky/Code/rwo/_build/install/default/lib/core/core.cmxa
          sh alias book/testing/examples/bigger_quickcheck_test_with_ppx (exit 1)
(cd _build/default/book/testing/examples && /usr/bin/sh -c 'dune build @all @runtest --root ./correct/bigger_quickcheck_test_with_ppx')
Entering directory 'correct/bigger_quickcheck_test_with_ppx'
File "test.ml", line 1, characters 5-9:
1 | open Core
         ^^^^
Error: Unbound module Core
File "dune", line 1, characters 0-95:
1 | (library
2 |  (name foo)
3 |  (libraries base stdio core)
4 |  (preprocess (pps ppx_jane))
5 |  (inline_tests))
Error: File unavailable:
/home/yminsky/Code/rwo/_build/install/default/lib/core/core.cmxa
File "_none_", line 1:     
Error: I/O error: duniverse/async_kernel/limiter_async/src/limiter_async.cma: Permission denied
File "_none_", line 1:     
Error: I/O error: duniverse/async_kernel/limiter_async/src/limiter_async.cmxa: Permission denied
File "_none_", line 1:     
Error: I/O error: duniverse/async_kernel/persistent_connection_kernel/src/persistent_connection_kernel.cma: Permission denied
File "_none_", line 1:     
Error: I/O error: duniverse/async_kernel/persistent_connection_kernel/src/persistent_connection_kernel.cmxa: Permission denied
File "_none_", line 1:     
Error: I/O error: duniverse/async_kernel/src/async_kernel.cma: Permission denied
File "_none_", line 1:     
Error: I/O error: duniverse/async_kernel/src/async_kernel.cmxa: Permission denied
File "duniverse/async_kernel/eager_deferred/src/eager_deferred0.ml", line 1:
Error: I/O error: duniverse/async_kernel/eager_deferred/src/.eager_deferred.objs/byte/eager_deferred__Eager_deferred0.cmo: Permission denied
File "duniverse/async_kernel/eager_deferred/src/eager_deferred_intf.ml", line 1:
Error: I/O error: duniverse/async_kernel/eager_deferred/src/.eager_deferred.objs/byte/eager_deferred__Eager_deferred_intf.cmo: Permission denied
File "duniverse/async_kernel/eager_deferred/src/import.ml", line 1:
Error: I/O error: duniverse/async_kernel/eager_deferred/src/.eager_deferred.objs/native/eager_deferred__Import.cmx: Permission denied
File "duniverse/async_kernel/weak_hashtbl_async/src/import.ml", line 1:
Error: I/O error: duniverse/async_kernel/weak_hashtbl_async/src/.weak_hashtbl_async.objs/native/weak_hashtbl_async__Import.cmx: Permission denied
File "duniverse/async_kernel/weak_hashtbl_async/src/weak_hashtbl_async.ml", line 1:
Error: I/O error: duniverse/async_kernel/weak_hashtbl_async/src/.weak_hashtbl_async.objs/byte/weak_hashtbl_async.cmo: Permission denied
File "duniverse/async_kernel/weak_hashtbl_async/src/weak_hashtbl_async.ml", line 1:
Error: I/O error: duniverse/async_kernel/weak_hashtbl_async/src/.weak_hashtbl_async.objs/native/weak_hashtbl_async.cmx: Permission denied

If I interrupt it and try again, I get this:

          sh alias book/testing/examples/bigger_quickcheck_test_with_ppx
Entering directory 'correct/bigger_quickcheck_test_with_ppx'
          sh alias book/testing/examples/bigger_quickcheck_test
Entering directory 'correct/bigger_quickcheck_test'
          sh alias book/testing/examples/echo_test_delay
Entering directory 'correct/echo_test_delay'
          sh alias book/testing/examples/echo_test_reconnect
Entering directory 'correct/echo_test_reconnect'
Done: 19906/19909 (jobs: 1)

If I look at htop while this is running, I see a dune process taking 100%, but not always the same dune process:

The sequence of nested dune invocations keeps getting deeper, and the PID of the busy process keeps on changing.

Oddly, if I truncate the README by deleting everything at line 303 and onwards, the build succeeds, and if I delete from 318 forward, it still gets stuck. But deleting the text between 303 and 318 is not enough to fix the build, which I guess means it's not some narrow problem with a small set of lines in the file. That said, something about the diff in the feature is causing the problem, because master doesn't have this issue.

[emillon]

it looks like a side effect of unsetting INSIDE_DUNE. My understanding is that this is unset in order to trigger diff printing, but this also influences dune's root detection. I don't have a perfect solution but a few workarounds are possible:

• change non-deterministic to non-deterministic=command (this disables checking this block by default - it moves the problem to when the block actually needs to be updated)
• pass --root . to dune. this will make dune not try to go up and run a nested build (but --root . will be part of the book)
• remove unset-INSIDE_DUNE and pass --diff-command patdiff to dune. same downside as previous

Some remarks:

• the root detection algorithm has been changed in dune 3.0 and so I think that this won't be an issue with the new algorithm (with dir=, dune is going to find a dune-workspace file there and stop) - but I haven't tried
• I'm not sure why the code in master works since the "problematic" parts (nested dune projects, INSIDE_DUNE being unset) are already there. This could be an interaction with non-deterministic in which case a fix might be possible in mdx.

[yminsky]

I've gotten my branch building again, but it would be good to fix the underlying bug. Could you perhaps submit a corresponding bug to Dune, and link that here?

[yminsky]

Pinging this again. @emillon ?

[emillon]

Yes. I'll pass that to the mdx maintainers. But I'm afraid that unsetting INSIDE_DUNE (a private API) is always going to have nasty corner cases like this. But this should be fixed anyway.

[yminsky]

Is there something better we should be doing in RWO, other than unsetting INSIDE_DUNE?