[new release] dune (15 packages) (3.8.0)

[emillon]

Fast, portable, and opinionated build system

• Project page: https://github.com/ocaml/dune
• Documentation: https://dune.readthedocs.io/

CHANGES:

• Fix string quoting in the json file written by --trace-file (Improve JSON string quoting in Dune tracing dune#7773,
  @rleshchinskiy)

• Read pkg-config arguments from the PKG_CONFIG_ARGN environment variable
  (Configurator needs to access opam variables and pass custom flags to pkg-config dune#1492, feat(pkg-config): allow reading PKG_CONFIG_ARGN dune#7734, @anmonteiro)

• Correctly set MANPATH in dune exec. Previously, we would use the bin/
  directory of the context. (fix: man dir for contexts dune#7655, @rgrinberg)

• Allow overriding the ocaml binary with findlib configuration (fix: allow overriding "ocaml" with findlib dune#7648,
  @rgrinberg)

• merlin: ignore instrumentation settings for preprocessing. (fix: drop instrumentation from Merlin.make dune#7606, fixes
  VSCode instrumentation dune#7465, @Alizter)

• When a rule's action is interrupted, delete any leftover directory targets.
  This is consistent with how we treat file targets. (fix: delete stale directory targets dune#7564, @rgrinberg)

• Fix plugin loading with findlib. The functionality was broken in 3.7.0.
  (test: demonstrate crash in link-time codegen dune#7556, @anmonteiro)

• Introduce a public_headers field on libraries. This field is like
  install_c_headers, but it allows to choose the extension and choose the
  paths for the installed headers. (fix: preserve directory hierarchy for installed headers dune#7512, @rgrinberg)

• Load the host context findlib.conf when cross-compiling (x-compilation: take host findlib.conf into account when cross-compiling dune#7428, fixes
  Dune doesn't use ocamlfind config in default context dune#1701, @rgrinberg, @anmonteiro)

• Add a coqdoc_flags field to the coq.theory stanza allowing the user to
  pass extra arguments to coqdoc. (feature(coq): add coqdoc_flags field to coq.theory dune#7676, fixes [3.8] backport #7895 dune#7954 @Alizter)

• Resolve ppx_runtime_libraries in the target context when cross compiling
  (test(x-compilation): demonstrate overlap check failure with ppx_runtime_libraries dune#7450, fixes Add a test exposing a bug in dependency cycle detection dune#2794, @anmonteiro)

• Use $PKG_CONFIG, when set, to find the pkg-config binary (dune-configurator: respect $PKG_CONFIG dune#7469, fixes
  How to use configurator to fetch pkg-config for the cross-compiling env? dune#2572, @anmonteiro)

• Modules that were declared in (modules_without_implementation),
  (private_modules) or (virtual_modules) but not declared in (modules)
  will cause Dune to emit a warning which will become an error in 3.9. (feature: warn if modules is missing any mentioned modules  dune#7608,
  fixes modules field in conjunction with private_modules, virtual_modules, etc. dune#7026, @Alizter)

• Preliminary support for Coq compiled intefaces (.vos files) enabled via
  (mode vos) in coq.theory stanzas. This can be used in combination with
  dune coq top to obtain fast re-building of dependencies (with no checking
  of proofs) prior to stepping into a file. (Support for Coq "vos" builds dune#7406, @rlepigre)

• Fix dune crashing on MacOS in watch mode whenever $PATH contains $PWD
  (fix #6907 dune#7441, fixes dune build --watch crashes when it detects filesystem change on macOS 13.1 on some Apple Silicon machines dune#6907, @rgrinberg)

• Fix dune install when cross compiling (test(x-compilation): demonstrate install files not found dune#7410, fixes How to install custom sites when cross-compiling dune#6191, @anmonteiro,
  @rizo)

• Find pps dependencies in the host context when cross-compiling, (test(x-compilation): demonstrate install files not found dune#7410,
  fixes ppxs are built in the target context in cross-compilation settings dune#4156, @anmonteiro)

• Dune in watch mode no longer builds concurrent rules in serial (fix: disable cutoff where it hurts concurrency dune#7395
  @rgrinberg, @jchavarri)

• Dune can now detect Coq theories from outside the workspace. This allows for
  composition with installed theories (not necessarily installed with Dune).
  (feature(coq): composition of installed theories dune#7047, @Alizter, @ejgallego)

• dune coq top now correctly respects the project root when called from a
  subdirectory. However, absolute filenames passed to dune coq top are no
  longer supported (due to being buggy) (Fix for 7344. dune#7357, fixes The "dune coq top" is unreliable when not called from the workspace root dune#7344, @rlepigre and
  @Alizter)

• Added a --no-build option to dune coq top for avoiding rebuilds (dune coq top: add --no-build option to avoid building dependencies dune#7380,
  fixes Option to make "dune coq top" more permissive dune#7355, @Alizter)

• RPC: Ignore SIGPIPE when clients suddenly disconnect (fix(rpc): ignore sigpipe on macos clients dune#7299, fix(rpc): ignore sigpipe on linux dune#7319, fixes
  refactor(rules): ues [Option.map] dune#6879, @rgrinberg)

• Always clean up the UI on exit. (fix(console): clean up console on exit dune#7271, fixes Missing newline in "dune exec" dune#7142 @rgrinberg)

• Bootstrap: remove reliance on shell. Previously, we'd use the shell to get
  the number of processors. (feature(boot): remove reliance on shell dune#7274, @rgrinberg)

• Bootstrap: correctly detect the number of processors by allowing nproc to be
  looked up in $PATH (fix(boot): fix env vars when finding out concurrency dune#7272, @Alizter)

• Speed up file copying on macos by using clonefile when available
  (@rgrinberg, feature: use clonefile on macos dune#7210)

• Adds support for loading plugins in toplevels (adds support for loading plugins in toplevels dune#6082, fixes Dune Site Plugins do not work in toplevel dune#6081,
  @ivg, @richardlford)

• Support commands that output 8-bit and 24-bit colors in the terminal (feature(ansi_color): add support for 8 and 24 bit color codes dune#7188,
  @Alizter)

• Speed up rule generation for libraries and executables with many modules
  (perf: run parse_compilation_units once dune#7187, @jchavarri)

• Add --watch-exclusions to Dune build options (User-configured watch exclusions dune#7216, @jonahbeckford)

• Do not re-render UI on every frame if the UI doesn't change (fix: do not re-render unless needed dune#7186, fix
  Dune watch (and probably dune output in general) causes significant GPU power draw due to unnecessary idle refreshes dune#7184, @rgrinberg)

• Make coq_db creation in scope lazy (@ejgallego, [scope] [coq] Make Coq_lib.DB creation lazy dune#7133)

• Non-user proccesses such as version control or config checking are now run
  silently. (feature: Silence display in non-user processes dune#6994, fixes minor usability issues with cram tests dune#4066, @Alizter)

• Add the --display-separate-messages flag to separate the error messages
  produced by commands with a blank line. (Add a new line at the end of error reports (#6823 dune#6823, fixes usability: include a blank line between separate compiler messages (errors, warnings) dune#6158, @esope)

• Accept the Ordered Set Language for the modes field in library stanzas
  (feature: make modes field an Ordered_set_lang instance dune#6611, @anmonteiro).

• dune install now respects --display quiet mode (fix(install): respect display options dune#7116, fixes Dune install doesn't respect --display=quiet [and maybe other args] dune#4573, fixes
  Quiet installation dune#7106, @Alizter)

• Stub shared libraries (dllXXX_stubs.so) in Dune-installed libraries could not
  be used as dependencies of libraries in the workspace (eg when compiling to
  bytecode and/or Javascript). This is now fixed. (Enable use of stublibs of _installed_ libraries dune#7151, @nojb)

• Allow the main module of a library with (stdlib ...) to depend on other
  libraries (stdlib: include dependencies for stdlib alias, decode correctly dune#7154, @anmonteiro).

• Bytecode executables built for JSOO are linked with -noautolink and no
  longer depend on the shared stubs of their dependent libraries (bc-for-jsoo: use -noautolink, do not depend on stublibs dune#7156, @nojb)

• Added a new user action (concurrent ) which is like (progn ) but runs the
  actions concurrently. (feature: concurrency action dune#6933, @Alizter)

• Allow (stdlib ...) to be used with (wrapped false) in library stanzas
  (test: demonstrate crash in modules.ml when (stdlib .. ) used with (wrapped false) dune#7139, @anmonteiro).

• Allow parallel execution of inline tests partitions (Run inline tests in parallel dune#7012, @hhugo)

• Support (link_flags ...) in (cinaps ...) stanza. (Add (link_flags ...) to (cinaps) stanza dune#7423, fixes cinaps stanza should support link_flags dune#7416,
  @nojb)

• Allow (package ...) in any position within (rule ...) stanza (fix: Allow package in any location in rule dune#7445,
  @Leonidas-from-XIV)

• Always include opam files in the generated .install file. Previously, it
  would not be included whenever (generate_opam_files true) was set and the
  .install file wasn't yet generated. (fix: correctly generate .install files dune#7547, @rgrinberg)

• Fix regression where Merlin was unable to handle filenames with uppercase
  letters under Windows. (merlin: avoid touching case of filename dune#7577, @nojb)

• On nix+macos, pass -f to the codesign hook to avoid errors when the binary
  is already signed (fix: use codesign -f to replace existing signature dune#7183, fixes Nix + macOS codesigning: -f flag needed dune#6265, @greedy)

• Fix bug where RPC clients built with dune-rpc-lwt would crash when closing
  their connection to the server (Don't attempt read from closed fd in dune_rpc_lwt dune#7581, @gridbugs)

• Introduce mdx stanza 0.4 requiring mdx >= 2.3.0 which updates the default
  list of files to include *.mld files (feature: mdx 0.4 stanza with support for mld files out of the box dune#7582, @Leonidas-from-XIV)

• Fix RPC server on Windows (used for OCaml-LSP). (dune-rpc: use loopback address on Windows dune#7666, @nojb)

• Coq language versions less 0.8 are deprecated, and will be removed
  in an upcoming Dune version. All users are required to migrate to
  (coq lang 0.8) which provides the right semantics for theories
  that have been globally installed, such as those coming from opam
  (@ejgallego, @Alizter)

• Bump minimum version of the dune language for the melange syntax extension
  from 3.7 to 3.8 (melange: set min version to 3.8 dune#7665, @jchavarri)

[emillon]

Just in case: I'll review the failures, please wait for my feedback before merging.

[emillon]

Compared to the 3.7.1 run, the new failures seem to be unrelated:

• ocamldsort and stdcompat fail on a missing autoconf (they do if I try to rebuild on 3.7.1)
• for the rest of them I was able to build locally, and are not regressions
  I'm curious about the failure for liquidsoap:

# MENHIR lang/parser.mly
# Error: the code back-end requires the type of every nonterminal symbol to be
# known. Please specify the type of every symbol via %type declarations, or

But I wasn't able to reproduce locally.

So it seems good to me; if the liquidsoap issue with menhir happens we can fix it in a point release.
Thanks

[avsm]

@emillon Also closing this temporarily so the cluster can catch up on other contributors' PRs. Feel free to reopen if you need more tests on another point release, or otherwise once you have triaged the results we can reopen and merge this PR promptly. Thanks!

[emillon]

I'm not sure why this got restarted (maybe a base-image pull), but this doesn't need any more CPU time, it can get merged. I'm not reopening in order not to trigger new builds but this can get reopened and merged when you'd like @mseri .

[Alizter]

ping

[avsm]

Thanks! You may want to announce this on https://discuss.ocaml.org, where we have a Community category and an announce tag for this purpose.

[Alizter]

@emillon Will do it soon.