[4.14] Fatal error: exception Not_found when cmi file is absent

[kit-ty-kate]

When using the 4.14 branch (3f170f3) to compile index-bench.1.5.0 I get:

#=== ERROR while compiling index-bench.1.5.0 ==================================#
# context              2.1.0 | linux/x86_64 | ocaml-variants.4.14.0+trunk | file:///home/opam/opam-repository
# path                 ~/.opam/4.14/.opam-switch/build/index-bench.1.5.0
# command              ~/.opam/opam-init/hooks/sandbox.sh build dune build -p index-bench -j 71
# exit-code            1
# env-file             ~/.opam/log/index-bench-19-525b6c.env
# output-file          ~/.opam/log/index-bench-19-525b6c.out
### output ###
#     ocamlopt bench/.bench.eobjs/native/dune__exe__Bench.{cmx,o} (exit 2)
# (cd _build/default && /home/opam/.opam/4.14/bin/ocamlopt.opt -w -40 -g -I bench/.bench.eobjs/byte -I bench/.bench.eobjs/native -I /home/opam/.opam/4.14/lib/cmdliner -I /home/opam/.opam/4.14/lib/fmt -I /home/opam/.opam/4.14/lib/index -I /home/opam/.opam/4.14/lib/index/unix -I /home/opam/.opam/4.14/lib/metrics -I /home/opam/.opam/4.14/lib/metrics-unix -I /home/opam/.opam/4.14/lib/mtime -I /home/opam/.opam/4.14/lib/mtime/os -I /home/opam/.opam/4.14/lib/optint -I /home/opam/.opam/4.14/lib/ppx_deriving/runtime -I /home/opam/.opam/4.14/lib/ppx_deriving_yojson/runtime -I /home/opam/.opam/4.14/lib/re -I /home/opam/.opam/4.14/lib/repr -I /home/opam/.opam/4.14/lib/stdlib-shims -I /home/opam/.opam/4.14/lib/yojson -I bench/.common.objs/byte -I bench/.common.objs/native -intf-suffix .ml -no-alias-deps -o bench/.bench.eobjs/native/dune__exe__Bench.cmx -c -impl bench/bench.pp.ml)
# Fatal error: exception Not_found

When testing locally it appears that bench/bench.pp.ml is compiled using the dune ppx pipeline in such a way:

Running[8]: (cd _build/default && .ppx/75591829f68362b2416610530939b58b/ppx.exe -o bench/bench.pp.ml --impl bench/bench.ml -corrected-suffix .ppx-corrected -diff-cmd - -dump-ast)

However when calling the ppx generator by hand to produce the OCaml syntax version (as opposed to the mashalled version implied by -dump-ast) and trying to compile it, it works fine:

$ .ppx/75591829f68362b2416610530939b58b/ppx.exe bench/bench.ml > test.ml
$ ocamlfind ocamlopt -package yojson -package ppx_deriving_yojson.runtime -c test.ml

The backtrace indicates that the issue seems to be in the type-checker:

Fatal error: exception Not_found
Raised at Persistent_env.find_pers_struct in file "typing/persistent_env.ml", line 213, characters 15-30
Called from Persistent_env.find in file "typing/persistent_env.ml" (inlined), line 266, characters 6-41
Called from Env.find_pers_mod in file "typing/env.ml" (inlined), line 920, characters 2-59
Called from Env.find_ident_module in file "typing/env.ml", line 1006, characters 22-51
Called from Env.find_module_components in file "typing/env.ml", line 1010, characters 17-43
Called from Env.find_structure_components in file "typing/env.ml", line 1020, characters 23-56
Called from Env.find_type_full in file "typing/env.ml", line 1081, characters 15-46
Called from Env.find_type in file "typing/env.ml" (inlined), line 1190, characters 2-24
Called from Ctype.extract_concrete_typedecl in file "typing/ctype.ml", line 1619, characters 17-36
Called from Ctype.extract_concrete_typedecl in file "typing/ctype.ml", line 1625, characters 18-50
Called from Typecore.extract_concrete_variant in file "typing/typecore.ml", line 322, characters 8-40
Called from Typecore.type_construct in file "typing/typecore.ml", line 4703, characters 10-50
Called from Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 239, characters 14-18
Re-raised at Builtin_attributes.warning_scope in file "parsing/builtin_attributes.ml", line 244, characters 4-13
[...]

Here is the full backtrace: not-found.log

[kit-ty-kate]

Actually I seemed to have missed something during testing because the same failure also appears on another package that does not use mashalled modules:

#=== ERROR while compiling osh.0.1 ============================================#
# context              2.1.0 | linux/x86_64 | ocaml-variants.4.14.0+trunk | file:///home/opam/opam-repository
# path                 ~/.opam/4.14/.opam-switch/build/osh.0.1
# command              ~/.opam/opam-init/hooks/sandbox.sh build dune build -p osh -j 31 @install
# exit-code            1
# env-file             ~/.opam/log/osh-19-475044.env
# output-file          ~/.opam/log/osh-19-475044.out
### output ###
#       ocamlc src/.osh.eobjs/byte/dune__exe__Osh.{cmi,cmo,cmt} (exit 2)
# (cd _build/default && /home/opam/.opam/4.14/bin/ocamlc.opt -w -40 -g -bin-annot -I src/.osh.eobjs/byte -I /home/opam/.opam/4.14/lib/dream -I /home/opam/.opam/4.14/lib/ezcurl -I /home/opam/.opam/4.14/lib/ocb -I /home/opam/.opam/4.14/lib/omd -I /home/opam/.opam/4.14/lib/yojson -no-alias-deps -open Dune__exe -o src/.osh.eobjs/byte/dune__exe__Osh.cmo -c -impl src/osh.ml)
# Fatal error: exception Not_found

-       ocamlc src/.osh.eobjs/byte/dune__exe__Osh.{cmi,cmo,cmt} (exit 2)
- (cd _build/default && /home/opam/.opam/4.14/bin/ocamlc.opt -w -40 -g -bin-annot -I src/.osh.eobjs/byte -I /home/opam/.opam/4.14/lib/dream -I /home/opam/.opam/4.14/lib/ezcurl -I /home/opam/.opam/4.14/lib/ocb -I /home/opam/.opam/4.14/lib/omd -I /home/opam/.opam/4.14/lib/yojson -no-alias-deps -open Dune__exe -o src/.osh.eobjs/byte/dune__exe__Osh.cmo -c -impl src/osh.ml)
- Fatal error: exception Not_found

[kit-ty-kate]

Actually I found what both projects have in common. They both use the (implicit_transitive_deps false) feature of dune.
If I switch it to true they compile without issues.

It seems to be a regression compared to #9409 / #9406

[Octachron]

Now that #10799 is merged, both osh and index-bench compiles on 4.14.0+trunk.