Manifest2Nix.jl

A Nix library for creating offline and reproducible Julia builds.

Julia uses an Ahead-Of-Time compilation system. This means Julia generates native code for functions that it can compile, while it still needs access to the source code in order to compile just-in-time. Manifest2Nix.jl is designed to take advantage of this system by precompiling as much code as possible.

Usage

nix flake new --template git+https://codeberg.org/aniva/Manifest2Nix.jl.git ./minimal

Overlay

Before building any Julia library, there first has to be a Julia toolchain. Generate a toolchain via one of 4 methods (in decreasing order of preference):

1. Use the self.fromManifest path overlay, which reads the version from a Manifest.toml file.
2. Use the self.fromVersion version overlay, e.g. manifest2nix.fromVersion "1.11.7".
3. Use the provided self.fromJuliaBin overlay, which uses the julia-bin package. (julia = julia-bin)
4. pkgs comes with a default julia package. It may work or it may not work. This build of Julia could have problems on Darwin systems.

Package Building

Once there is a fixed Julia version available as pkgs.julia, create the manifest2nix library by calling the mkLib function:

m2nlib = manifest2nix.mkLib pkgs

In m2nlib, some functions are available for building Julia packages:

• manifest2nix: Tool for creating the Lock.toml. To create a Lock.toml file, execute at the root of a Julia project: (use --up to also update the manifest)

manifest2nix lock [--up] --project .

• buildJuliaPackage { src, depots, deps, pre-exec ? "", env ? {} }: Builds a Julia package with explicit dependencies. A Julia package in Nix is a special attrset generated by this function.

  If pre-exec is not null, it will run in Julia after precompilation to force Julia to generate more compiled code. Set env for environment variables.

• buildJuliaPackageWithDeps { src, lockFile ? "${src}/Lock.toml", pre-exec ? "", override ? {}, env ? {} }: Build a Julia package along with dependencies. If there are any path-tracking dependencies, they must be fed in as overrides. e.g. override = { Artefact = ./artefact; }. The targets of overrides can be either paths to sources or built Julia packages. Set env to provide environment variables.

• stdlib-depot: A Julia depot containing a precompiled version of Julia standard libraries.

• mkDepsDepot { name, deps }: Given a list of Julia packages, create a depot containing all of them. This depot can be cached to speed up compilation.

• createEnv { package, workingDepot ? "" }: Given a Julia package, create an environment (i.e. set of variables) in which Julia can run and see the precompiled version of the given package. By default, the working depot here is the default path to the system depot. However, if this environment is then used in a Nix build, then workingDepot must be set to a writable path e.g. .julia. Not doing so can cause compilation failures.

Caching

A Julia package has these attributes:

• .compiled: Precompiled binary code. Cache this in Nix to shorten the compilation of downstream packages.
• .load-path: A special derivation for making load paths. Each package $PACKAGE's load-path is a derivation containing a single directory $PACKAGE which includes the project's source code. In most cases this is a direct link to the project's source directory. For legacy packages without a Project.toml, there will be a synthetic Project.toml file so Julia can find the package.
• .artifacts: An attribute set of all downloaded artifacts.

Troubleshooting

If the SSL certificate in nixpkgs is too old, it may cause registry fetch failures during package updates. This will manifest as "the identity of the server ... could not be verified." In which case, update using Julia itself, or update nixpkgs.

Contributing

Use the provided flake devShell, and install pre-commit hooks:

pre-commit install --install-hooks

Updates

When there is a new Julia version, execute nix run .#toolchain-fetch $VERSION to generate Nix attributes. Add these attributes to sources.nix. For minor version updates, add a new Manifest/Lock pair to version/.