Conversation
Signed-off-by: Will Thomas <30wthomas@gmail.com>
Signed-off-by: Will Thomas <30wthomas@gmail.com>
…itches
The secondary compiler fallback in bootstrap.ml was broken in two ways:
1. It invoked [ocamlfind -toolchain secondary ocaml], but ocamlfind does
not support the [ocaml] toplevel as a subcommand — only compiler tools
like [ocamlc] and [ocamlopt]. We now query [ocamlfind -toolchain
secondary query ocaml -format "%d"] to locate the secondary compiler's
bin directory and invoke [ocaml] and [ocamllex] from there directly.
2. The secondary [ocaml] toplevel would pick up the primary switch's
[unix.cma] and [dllunix.so], which are ABI-incompatible. We now pass
[-I] for the secondary compiler's lib directory and set
[CAML_LD_LIBRARY_PATH] to its stublibs directory.
The opam constraint is restored to the historical pattern, allowing dune
to be installed on switches with OCaml >= 4.02 via ocamlfind-secondary:
("ocaml" {>= "4.14"} | ("ocaml" {>= "4.02" & < "4.14~~"} & "ocamlfind-secondary"))
Tested on OCaml 4.02.3 and 4.13.1 switches with ocaml-secondary-compiler
4.14.2.
Signed-off-by: Ali Caglayan <alizter@gmail.com>
The path we pass to the script is always correct since dune may relativise it whether it lives. The old hard coded path was incorrect since we moved the .in script. Signed-off-by: Ali Caglayan <alizter@gmail.com>
Add ocaml-dune/opam-repository-relocatable as a default repository for package management. This is a mirror of dra27/opam-repository#relocatable which provides overlay packages for building a relocatable OCaml compiler. When this repository is included, the solver will pick up the relocatable-compiler meta-package, allowing toolchain builds to be cached as regular packages instead of using the toolchain cache. Signed-off-by: Ali Caglayan <alizter@gmail.com>
The relocatable-compiler package installs libraries (including threads, str, unix) under target/lib/ocaml/, matching the layout of other compiler packages. Without being listed in compiler_package_names, install_roots did not append the /ocaml subdirectory to its lib_root, causing dev-tools like utop to fail with "Library threads not found". Signed-off-by: Ali Caglayan <alizter@gmail.com>
file_async opens an fd eagerly and for large files holds it open across a fiber yield while queued for the background thread pool. When called via unbounded parallel_map over many targets (e.g. the relocatable compiler with thousands of files), fds accumulate faster than they are closed, exhausting the process fd limit. Add a global Fiber.Throttle in file_async to bound the number of concurrent digest operations. Signed-off-by: Ali Caglayan <alizter@gmail.com>
Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
This PR enables the relocatable compiler by making it the default. It does so by using (a mirror of) David's relocatable opam-repository as a second set of overlays. The order is now `overlay`, `relocatable`, `upstream`. This is part of the work on: - ocaml#13229 - Fixes ocaml#14012 The following need to be done: - [x] changelog - [x] documentation - [x] testing with: - [x] ocamlfind - some warnings but builds and works fine - [x] ocamlbuild - [x] ocamlformat dev tool - [x] ocamllsp devtool - [x] utop devtool In order to test, Ensure you have build dune from source using this branch or `opam pin dune.dev git@github.com:Alizter/dune.git#push-vkkkwyxwsotw` into your switch. Then 1. clone your favourite OCaml project. 2. Please set `DUNE_TRACE=+cache` and `DUNE_CACHE=enabled` 3. Run `dune build --pkg enabled` and wait. 4. Then when it's finished, backup the `_build/trace.sexp` file as `trace.sexp.old` and delete `_build/` 5. Run the build again. If all goes well (and your cache is actually enabled) it should restore from cache. Any issues please report here together with your first and second trace files. `_build/trace.csexp` and the backed-up `trace.csexp.old`.
Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
Backport ocaml#14434 onto 3.23.1-rc
…itches (ocaml#14443) The secondary compiler fallback in bootstrap.ml was broken in two ways: 1. It invoked `ocamlfind -toolchain secondary ocaml`, but `ocamlfind` does not support the `ocaml` toplevel as a subcommand. Only compiler tools like `ocamlc` and `ocamlopt`. We now query `ocamlfind -toolchain secondary query ocaml -format "%d"` to locate the secondary compiler's bin directory and invoke `ocaml` and `ocamllex` from there directly. 2. The secondary `ocaml` toplevel would pick up the primary switch's `unix.cma` and `dllunix.so`, which are ABI-incompatible. We now pass `-I` for the secondary compiler's lib directory and set `CAML_LD_LIBRARY_PATH` to its stublibs directory. The opam constraint is restored to the historical pattern, allowing dune to be installed on switches with OCaml >= 4.02 via ocamlfind-secondary: ``` ("ocaml" {>= "4.14"} | ("ocaml" {>= "4.02" & < "4.14~~"} & "ocamlfind-secondary")) ``` Tested on OCaml 4.02.3 and 4.13.1 switches with ocaml-secondary-compiler 4.14.2. - Fixes ocaml#13908
…itches (ocaml#14443) The secondary compiler fallback in bootstrap.ml was broken in two ways: 1. It invoked `ocamlfind -toolchain secondary ocaml`, but `ocamlfind` does not support the `ocaml` toplevel as a subcommand. Only compiler tools like `ocamlc` and `ocamlopt`. We now query `ocamlfind -toolchain secondary query ocaml -format "%d"` to locate the secondary compiler's bin directory and invoke `ocaml` and `ocamllex` from there directly. 2. The secondary `ocaml` toplevel would pick up the primary switch's `unix.cma` and `dllunix.so`, which are ABI-incompatible. We now pass `-I` for the secondary compiler's lib directory and set `CAML_LD_LIBRARY_PATH` to its stublibs directory. The opam constraint is restored to the historical pattern, allowing dune to be installed on switches with OCaml >= 4.02 via ocamlfind-secondary: ``` ("ocaml" {>= "4.14"} | ("ocaml" {>= "4.02" & < "4.14~~"} & "ocamlfind-secondary")) ``` Tested on OCaml 4.02.3 and 4.13.1 switches with ocaml-secondary-compiler 4.14.2. - Fixes ocaml#13908 (cherry picked from commit 9e65007) Signed-off-by: Shon Feder <shon.feder@gmail.com>
PR ocaml#14175 added deduplication of redundant `>=` bounds on the `dune` dependency in generated opam files (e.g. `{>= "3.17" & >= "3.20"}` collapsing to `{>= "3.20"}`), but did so without a lang-version guard. As a result, simply upgrading the dune binary to 3.23 changed the generated opam output for projects whose `(lang dune X.Y)` had not changed (issue ocaml#14436). Gate the new behaviour on `(lang dune 3.23)`, the lang version under which the dedup feature shipped. Projects at earlier lang versions get the pre-14175 `And [lang; user]` shape. The accompanying user warning (lower bound < lang version) is also gated, since it lives inside `merge_dune_constraints`. Fixes ocaml#14436 Signed-off-by: Robin Bate Boerop <me@robinbb.com>
Replaces the inline OCaml helper executable with `(action (run cat
%{input-file}))`, matching the convention established in
`test/blackbox-tests/test-cases/include-qualified/pp-github6866.t` and
`pp.t/dune`. The walker only cares that some preprocess action is
declared on the entry module; the pass-through's contents are
irrelevant.
Signed-off-by: Robin Bate Boerop <me@robinbb.com>
Rewrite the test's prose so a reader without prior context can follow what is set up and why the assertion is non-trivial. Lead with the concrete three-library scenario before explaining why dune cannot observe the dependency. Drop "tight" / "tight-eligible" terminology that has no meaning in the context of this test, drop the parenthetical "(used to compute per-module dependency sets)", and trim the per-block prose now that the header carries the load. Signed-off-by: Robin Bate Boerop <me@robinbb.com>
ocaml#14435) ## Summary Adds a cross-library cram test: a consumer reaches a transitive type only through a *preprocessed* library's `.mli`, where the type is never syntactically named in the consumer's source. Three libraries: - `other_dep` — unwrapped, exposes `type t = { x : int; y : string }`. - `pp_dep` — unwrapped + preprocessed, depends on `other_dep`, exposes `val make_thing : unit -> Other.t`. - `consumer` — depends on `pp_dep`, accesses `(D.make_thing ()).x`. `consumer/c.ml` never mentions `Other`, but the field access forces the type checker to load `other.cmi`. The test asserts `dune build @check` succeeds. Sibling to ocaml#14400's `cross-lib-walk-pre-pp-source.t`. That test exercises behaviour at a preprocessed entry directly; this one exercises the *transitive* leg — where the preprocessed entry's `.mli` re-exports a type from a deeper library.
There's no point to develop this separately Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
The build logic is currently scattered around in: 1. the scheduler 2. engine 3. the command line Start moving it all to one place Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
Run the dune-cache repro-check builds with -j1 so the two independent rules cannot print in either order. Also check that the computed digest is absent explicitly, instead of depending on dune_cmd cat exception formatting. Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
This is useful to make tests more reproducible. We can wait until dune ack's a file modification by sending this RPC. Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
…4466) ## Summary Gate the dune-version-bound deduplication in generated opam files (introduced in ocaml#14175 and shipped in 3.23.0) on `(lang dune 3.23)`. Projects at earlier lang versions get the pre-14175 `And [lang; user]` shape. Fixes ocaml#14436 Related: ocaml#14175
…caml#14419) Signed-off-by: Antonio Nuno Monteiro <anmonteiro@gmail.com>
ocaml#14420) Signed-off-by: Antonio Nuno Monteiro <anmonteiro@gmail.com>
…4466) ## Summary Gate the dune-version-bound deduplication in generated opam files (introduced in ocaml#14175 and shipped in 3.23.0) on `(lang dune 3.23)`. Projects at earlier lang versions get the pre-14175 `And [lang; user]` shape. Fixes ocaml#14436 Related: ocaml#14175 (cherry picked from commit eb4a64a) Signed-off-by: Robin Bate Boerop <me@robinbb.com>
Track in-flight RPC requests per session and include all active RPC connections plus queued RPC jobs in the SIGUSR2 debug event. Add a blackbox test covering concurrent RPC clients. Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
Persistent file descriptors now carry a Repr.t so generic dumping can render through the shared representation API. Convert descriptor call sites to repr values, using structural representations where practical and preserving opaque debug output for table-backed or intentionally hidden data. Signed-off-by: Rudi Grinberg <me@rgrinberg.com>
Signed-off-by: Antonio Nuno Monteiro <anmonteiro@gmail.com>
Signed-off-by: Antonio Nuno Monteiro <anmonteiro@gmail.com>
`(deps (:name (package foo)))` is silently accepted by the parser but
`%{name}` resolves to an empty path list. The package contributes no
paths to the binding, only env vars (when `(package ...)` is allowed by
the surrounding code). This is rarely what users intend.
We now reject the construct in `named_paths_builder` with a clear error
pointing at the `(package ...)` location and a hint to move the entry
out of the binding.
This was pre-existing behaviour; not specific to the install-layout PR
ocaml#14373 but found while reviewing.
In the recent update, the compiler package in OxCaml was changed to `oxcaml-compiler`: oxcaml/opam-repository@e36bb6b and oxcaml/opam-repository@2f92731. This patch adds the package to the list of compiler packages so that Dune treats it as a compiler when building.
The nix-based revdeps tests never worked for us: including packages that don't build per se and so give false positives, and nix derivations that require unnecessary system deps (like emacs). In any case, the point of this test is to be an early warning sign for the opam-ci, so we should be using opam. Opam is used here, and we also install `--with-test`, to get the benefit of running several test suites. Fixes ocaml#13897 --------- Signed-off-by: Shon Feder <shon.feder@gmail.com>
Signed-off-by: Shon Feder <shon.feder@gmail.com>
Signed-off-by: Shon Feder <shon.feder@gmail.com>
Signed-off-by: Antonio Nuno Monteiro <anmonteiro@gmail.com> (cherry picked from commit 0a8be12) Signed-off-by: Sudha Parimala <sudharg247@gmail.com>
Backport ocaml#14683 onto 3.24.0-rc
Signed-off-by: Shon Feder <shon.feder@gmail.com>
Signed-off-by: Shon Feder <shon.feder@gmail.com>
…ocaml#14697) Signed-off-by: Antonio Nuno Monteiro <anmonteiro@gmail.com>
ocaml#14698) …reviews Signed-off-by: Antonio Nuno Monteiro <anmonteiro@gmail.com>
Signed-off-by: Antonio Nuno Monteiro <anmonteiro@gmail.com>
Signed-off-by: Antonio Nuno Monteiro <anmonteiro@gmail.com>
Signed-off-by: Antonio Nuno Monteiro <anmonteiro@gmail.com>
…l#14715) Signed-off-by: Antonio Nuno Monteiro <anmonteiro@gmail.com>
Backport ocaml#14715 onto 3.24.0-rc. Fixes the melange build failure on OCaml 4.14 where dune emits an `I/O error: jscomp/runtime/.melange_src/X.mli: No such file or directory` against every `melange-re/melange.*-414` reverse dep (see ocaml/opam-repository#29996 — the 3.24.0~alpha2 release PR). The fix is to skip the line directive in `setup_melange_sources_copy_rules` when a module uses non-staged PPX, since the PPX pipeline now passes `-loc-filename` separately and adding the directive would double-shift diagnostics. The fix has a small precursor chain on `main`; bringing the headline `ocaml#14715` alone doesn't apply cleanly because subsequent PRs introduced the test files and machinery the fix's diff targets. Included PRs (chronological): - ocaml#14697 — test: show single context melange loses ppx snippet previews - ocaml#14698 — test: show single context reason+melange loses ppx snippet previews - ocaml#14708 — test: accept ppx loc-filename in fixtures - ocaml#14706 — **fix**: preserve original ppx source locations (introduces `-loc-filename`) - ocaml#14714 — test: reproduce reason source location regression - ocaml#14715 — **fix**: avoid line directive when passing `-loc-filename`
(cherry picked from commit bf27262) Signed-off-by: Ali Caglayan <alizter@gmail.com>
Value.string_of_path now uses Path.reach_for_running so that pforms
render bare-basename paths with a leading "./". This avoids bash and
other shells (in (bash ...) and (system ...) actions) treating
same-directory binaries as PATH lookups.
Also rewrites the %{bin:NAME} entry in doc/concepts/variables.rst.
(cherry picked from commit 2851a23)
Signed-off-by: Ali Caglayan <alizter@gmail.com>
Backport ocaml#15156 onto 3.24.0-rc. Also includes the test commit from ocaml#15148 (`test: bash bin pform`) as a prerequisite, since the test file `runnable-argument.t` modified by ocaml#15156 was introduced in ocaml#15148 and is not yet on 3.24.0-rc.
Signed-off-by: Shon Feder <shon.feder@gmail.com>
Signed-off-by: Shon Feder <shon.feder@gmail.com>
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release branch for #26