This reverts commit 13ec865.

This has caused widespread pain because of NixOS/nix#9052

This reverts commit 5e3986f. This
un-implements RFC 92 but fixes the critical bug NixOS#9052 which many people
are hitting. This is a decent stop-gap until a minimal reproduction of
that bug is found and a proper fix can be made.

Mostly fixed NixOS#9052, but I would like to leave that issue open until we
have a regression test, so I can then properly fix the bug (unbreaking
RFC 92) later.

This reverts commit 5e3986f. This
un-implements RFC 92 but fixes the critical bug NixOS#9052 which many people
are hitting. This is a decent stop-gap until a minimal reproduction of
that bug is found and a proper fix can be made.

Mostly fixed NixOS#9052, but I would like to leave that issue open until we
have a regression test, so I can then properly fix the bug (unbreaking
RFC 92) later.

This reverts commit 5e3986f. This
un-implements RFC 92 but fixes the critical bug NixOS#9052 which many people
are hitting. This is a decent stop-gap until a minimal reproduction of
that bug is found and a proper fix can be made.

Mostly fixed NixOS#9052, but I would like to leave that issue open until we
have a regression test, so I can then properly fix the bug (unbreaking
RFC 92) later.

This reverts commit 5e3986f. This
un-implements RFC 92 but fixes the critical bug NixOS#9052 which many people
are hitting. This is a decent stop-gap until a minimal reproduction of
that bug is found and a proper fix can be made.

Mostly fixed NixOS#9052, but I would like to leave that issue open until we
have a regression test, so I can then properly fix the bug (unbreaking
RFC 92) later.

This reverts commit 5e3986f. This
un-implements RFC 92 but fixes the critical bug NixOS#9052 which many people
are hitting. This is a decent stop-gap until a minimal reproduction of
that bug is found and a proper fix can be made.

Mostly fixed NixOS#9052, but I would like to leave that issue open until we
have a regression test, so I can then properly fix the bug (unbreaking
RFC 92) later.

This fixes dynamic derivations, reverting #9081. #9052 However will be
reintroduced unless this is modified somehow.

This reverts commit 8440afb.

This fixes dynamic derivations, reverting #9081. #9052 However will be
reintroduced unless this is modified somehow.

This reverts commit 8440afb.

This fixes dynamic derivations, reverting #9081. #9052 However will be
reintroduced unless this is modified somehow.

This reverts commit 8440afb.

This fixes dynamic derivations, reverting #9081.

I believe that this time around, #9052 is fixed. When I first rebased
this, tests were failing (which wasn't the case before). The cause of
those test failures were due to the crude job in which the outer goal
tried to exit with the inner goal's status.

Now, that error handling has been reworked to be more faithful. The exit
exit status and exception of the inner goal is returned by the outer
goal. The exception was what was causing the test failures, but I
believe it was not having the right error code (there is more than one
for failure) that caused #9081.

The only cost of doing things the "right way" was that I had to
introduce a hacky `preserveException` boolean. I don't like this, but,
then again, none of us like anything about how the scheduler works.
Issue #11927 is still there to clean everything up, subsuming the need
for any `preserveException` because I doubt we will be fishing
information out of state machines like this at all.

This reverts commit 8440afb.

This fixes dynamic derivations, reverting #9081.

I believe that this time around, #9052 is fixed. When I first rebased
this, tests were failing (which wasn't the case before). The cause of
those test failures were due to the crude job in which the outer goal
tried to exit with the inner goal's status.

Now, that error handling has been reworked to be more faithful. The exit
exit status and exception of the inner goal is returned by the outer
goal. The exception was what was causing the test failures, but I
believe it was not having the right error code (there is more than one
for failure) that caused #9081.

The only cost of doing things the "right way" was that I had to
introduce a hacky `preserveException` boolean. I don't like this, but,
then again, none of us like anything about how the scheduler works.
Issue #11927 is still there to clean everything up, subsuming the need
for any `preserveException` because I doubt we will be fishing
information out of state machines like this at all.

This reverts commit 8440afb.

This fixes dynamic derivations, reverting #9081.

I believe that this time around, #9052 is fixed. When I first rebased
this, tests were failing (which wasn't the case before). The cause of
those test failures were due to the crude job in which the outer goal
tried to exit with the inner goal's status.

Now, that error handling has been reworked to be more faithful. The exit
exit status and exception of the inner goal is returned by the outer
goal. The exception was what was causing the test failures, but I
believe it was not having the right error code (there is more than one
for failure) that caused #9081.

The only cost of doing things the "right way" was that I had to
introduce a hacky `preserveException` boolean. I don't like this, but,
then again, none of us like anything about how the scheduler works.
Issue #11927 is still there to clean everything up, subsuming the need
for any `preserveException` because I doubt we will be fishing
information out of state machines like this at all.

This reverts commit 8440afb.

This fixes dynamic derivations, reverting #9081.

I believe that this time around, #9052 is fixed. When I first rebased
this, tests were failing (which wasn't the case before). The cause of
those test failures were due to the crude job in which the outer goal
tried to exit with the inner goal's status.

Now, that error handling has been reworked to be more faithful. The exit
exit status and exception of the inner goal is returned by the outer
goal. The exception was what was causing the test failures, but I
believe it was not having the right error code (there is more than one
for failure) that caused #9081.

The only cost of doing things the "right way" was that I had to
introduce a hacky `preserveException` boolean. I don't like this, but,
then again, none of us like anything about how the scheduler works.
Issue #11927 is still there to clean everything up, subsuming the need
for any `preserveException` because I doubt we will be fishing
information out of state machines like this at all.

This reverts commit 8440afb.

This fixes dynamic derivations, reverting #9081.

I believe that this time around, #9052 is fixed. When I first rebased
this, tests were failing (which wasn't the case before). The cause of
those test failures were due to the crude job in which the outer goal
tried to exit with the inner goal's status.

Now, that error handling has been reworked to be more faithful. The exit
exit status and exception of the inner goal is returned by the outer
goal. The exception was what was causing the test failures, but I
believe it was not having the right error code (there is more than one
for failure) that caused #9081.

The only cost of doing things the "right way" was that I had to
introduce a hacky `preserveException` boolean. I don't like this, but,
then again, none of us like anything about how the scheduler works.
Issue #11927 is still there to clean everything up, subsuming the need
for any `preserveException` because I doubt we will be fishing
information out of state machines like this at all.

This reverts commit 8440afb.

Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>

This fixes dynamic derivations, reverting #9081.

I believe that this time around, #9052 is fixed. When I first rebased
this, tests were failing (which wasn't the case before). The cause of
those test failures were due to the crude job in which the outer goal
tried to exit with the inner goal's status.

Now, that error handling has been reworked to be more faithful. The exit
exit status and exception of the inner goal is returned by the outer
goal. The exception was what was causing the test failures, but I
believe it was not having the right error code (there is more than one
for failure) that caused #9081.

The only cost of doing things the "right way" was that I had to
introduce a hacky `preserveException` boolean. I don't like this, but,
then again, none of us like anything about how the scheduler works.
Issue #11927 is still there to clean everything up, subsuming the need
for any `preserveException` because I doubt we will be fishing
information out of state machines like this at all.

This reverts commit 8440afb.

Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>

It's hard to reproduce this race condition going forward (e.g. it
depends on the precise order that goals are scheduled, but
https://github.com/roberth/nix-9052 current reproduces it, and this
fixes that.

It's hard to reproduce this race condition going forward (e.g. it
depends on the precise order that goals are scheduled, but
https://github.com/roberth/nix-9052 current reproduces it, and this
fixes that.

* Add FIXME

* Update tests/functional/flakes/symlink-paths.sh

* Add `inputs.self.submodules` flake attribute

This allows a flake to specify that it needs Git submodules to be
enabled (or disabled, if we ever change the default) on the top-level
flake. This requires the input to be refetched, but since the first
fetch is lazy, this shouldn't be expensive.

Currently the only attribute allowed by `inputs.self` is `submodules`,
but more can be added in the future (e.g. a `lazy` attribute to opt in
to lazy tree behaviour).

Fixes NixOS#5312, NixOS#9842.

* Add a test for a flake referencing a flake that has inputs.self

* nix-profile.fish: set --local NIX_LINK

Using `set --local` is better than using `set`/`set --erase`.  `--local`
will preserve any existing `NIX_LINK` value.  And the local variable is
automatically removed for any execution path.

* nix-profile.fish: Look for ca-bundle.crt in $NIX_PROFILES

There seems to be no good reason for `nix-profile.fish` and
`nix-profile-daemon.fish` to differ in how they look for the location of
the `ca-bundle.crt` that might be installed by one of the packages.

As `$NIX_PROFILES` points to user local paths, not checking there is
strictly less useful, it seems?

* Revert "Revert "Adapt scheduler to work with dynamic derivations""

This fixes dynamic derivations, reverting NixOS#9081.

I believe that this time around, NixOS#9052 is fixed. When I first rebased
this, tests were failing (which wasn't the case before). The cause of
those test failures were due to the crude job in which the outer goal
tried to exit with the inner goal's status.

Now, that error handling has been reworked to be more faithful. The exit
exit status and exception of the inner goal is returned by the outer
goal. The exception was what was causing the test failures, but I
believe it was not having the right error code (there is more than one
for failure) that caused NixOS#9081.

The only cost of doing things the "right way" was that I had to
introduce a hacky `preserveException` boolean. I don't like this, but,
then again, none of us like anything about how the scheduler works.
Issue NixOS#11927 is still there to clean everything up, subsuming the need
for any `preserveException` because I doubt we will be fishing
information out of state machines like this at all.

This reverts commit 8440afb.

Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>

* Add BLAKE3 hashing algorithm

This uses the single-threaded C-based routines from libblake3.

This is not optimal performance-wise but should be a good starting point
for nix compatibility with BLAKE3 hashing until a more performant
implementation based on the multi-threaded BLAKE3 routines
(written in Rust) can be developed.

* Add BLAKE3 to documentation

* Parser: Respect the accessor of the source file for relative paths

* Parser: Respect the accessor of the source file for relative paths

Previously we only returned paths in rootFS, which is wrong and only
worked because currently all our source trees are in rootFS.

* pre-commit/check-merge-conflicts-2: fix use outside dev shell

Note that this is just a script that is meant to run outside a
derivation (but also can be called by a derivation builder).
`touch $out` does not belong in it.

`touch $out` worked accidentally in the derivation-based check,
and also in the dev shell, but if pre-commit is invoked without
the dev shell it would fail.

* test: Fix shellcheck by giving git-hashing scripts shebangs

This seems to be the way to do it now, even though I can't run them
without setting at least one env var.
I'll only fix shellcheck for now. Don't shoot the messenger.

It isn't quite clear to me why the previous commit masked this problem,
but I'm glad shellcheck has an effect or more effect now.

* test: Use skipTest instead of exit 0

This way shellcheck is ok with it, and it conveys that a significant
chunk of the test is skipped.

* Set FD_CLOEXEC on sockets created by curl

Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can
cause connections to remain open forever when using commands like `nix
shell`

This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION
callback.

* Document Store Derivations and Deriving Paths (NixOS#12290)

This is a big step documenting the store layer on its own, separately from the evaluator (and `builtins.derivation`).

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>

* Rename to "content-address*ing* derivation"

"content-address*ed*" derivation is misleading because all derivations
are *themselves* content-addressed. What may or may not be
content-addressed is not derivation itself, but the *output* of the
derivation.

The outputs are not *part* of the derivation (for then the derivation
wouldn't be complete before we built it) but rather separate entities
produced by the derivation.

"content-adddress*ed*" is not correctly because it can only describe
what the derivation *is*, and that is not what we are trying to do.

"content-address*ing*" is correct because it describes what the
derivation *does* --- it produces content-addressed data.

* Fix possible deref of null git_config

* libfetchers: Drop no longer needed curl dependency

* Misc code cleanups

* Move git-lfs-fetch into its own source file

* getLfsEndpointUrl(): Use our RAII helpers

* Add release note

* smudgeLfs: Use default value

Eventually this should probably become a struct of options.

* Add TODO

* Restore NAR hash assertion

* Remove stray line

* Add release note

* Update src/libstore/unix/build/local-derivation-goal.cc

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>

* nix flake prefetch: Add --out-link option

This makes `nix flake prefetch` more useful for scripting and prevents
the result from being GC'ed prematurely.

* Only try to chmod /nix/var/nix/profiles/per-user when necessary

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>

* copyPathToStore(): Preserve symlinks

E.g. in a derivation attribute `foo = ./bar`, if ./bar is a symlink,
we should copy the symlink to the store, not its target. This restores
the behaviour of Nix <= 2.19.

* Don't import a symlink

This is a workaround to avoid differing evaluation results between Nix
2.19 and >= 2.20 (NixOS#12449).

* Get rid of `impureOutputHash`

I do not believe there is any problem with computing
`hashDerivationModulo` the normal way with impure derivations.

Conversely, the way this used to work is very suspicious because two
almost-equal derivations that only differ in depending on different
impure derivations could have the same drv hash modulo. That is very
suspicious because there is no reason to think those two different
impure derivations will end up producing the same content-addressed
data!

Co-authored-by: Alain Zscheile <zseri.devel@ytrizja.de>

* lockFlake(): When refetching a locked flake, use the locked ref

Otherwise we may accidentally update a lock when we shouldn't.

Fixes NixOS#12445.

* Formatting

* Add test

* feat: access tokens per repo

* feat: test and document access-token prefix support

* Add `inputs.self.lfs`

* Add release note

* fix: ensure access-token matches are complete

* fix: add comment about longest-possible match

* fix: linting

* doc: recursive-nix: advertise requiredSystemFeatures

It's best we teach users that the "foo" derivation is less than pure in the sense that it cannot be built just on any system, in particular that builders cannot be selected arbitrarily but based on their system-features. The `"recursive-nix"` system-feature is automatically defined by `--extra-experimental-features recursive-nix`

* Merge release notes

* Expose a bunch of things in the Legacy SSH Store for Hydra

* More interesting dynamic derivations example

Co-authored-by: Samuel Ainsworth <skainsworth@gmail.com>

* Introduce `DerivationOptions`

This is a first step towards PR NixOS#10760, and the issues it addresses.
See the Doxygen for details.

Thanks to these changes, we are able to drastically restrict how the
rest of the code-base uses `ParseDerivation`.

Co-Authored-By: HaeNoe <git@haenoe.party>

* Add `SSHMaster::Connection::trySetBufferSize`

It is unused in Nix currently, but will be used in Hydra. This reflects
what Hydra does in NixOS/hydra#1387.

We may probably to use it more widely for better SSH store performance,
but this needs to be subject to more testing before we do that.

* Allow setting `ssh://` pipe size

Exposed for Hydra. We could make it fancier but with (a) new store
settings (b) switch to `ssh-ng://` both in the works, it doesn't seem
worth it.

* getDefaultNixPath(): Don't add symlinks if the target doesn't exist

* resolveLookupPathPath(): Fix caching of negative lookups

This avoids spamming in case the missing search path entry does not
exist (NixOS#12480).

* dep-built-drv-2.sh: Don't fail with "cannot create symlink"

* JSONLogger: Log to a file descriptor instead of another Logger

Logging to another Logger was kind of nonsensical - it was really just
an easy way to get it to write its output to stderr, but that only
works if the underlying logger writes to stderr.

This change is needed to make it easy to log JSON output somewhere
else (like a file or socket).

* Support libgit2 1.9.0

For when the overlay is used with nixos-unstable.
1.9.0 has our patches.

* Use getStandardError()

* Fix a few warnings

* Move code related to NIX_MAN_DIR from libstore to nix-cli

This is a prerequisite to properly fixing man-pages once and
for all [1]. Note that this patch leaves manpages for legacy
commands in a borked state, pending the movement of manpages from
nix-manual to nix-cli [2].

[1]: https://www.github.com/NixOS/nix/issues/12382
[2]: https://www.github.com/NixOS/nix/issues/12382#issuecomment-2663782043

* windows: fix compilation after recent changes

Specifically last few week's merges involving legacy SSH options and
dynamic derivations.

* Restore detailed Nix CLI version

... as intended.

Requirements:
- don't build fresh libraries for each git commit
- have git commit in the CLI

Bug:
- echo ${version} went into the wrong file => use the fact that it's
  a symlink, not just for reading but also for writing.

* Write just ./.version on all components

This way it's easier to get right. See previous commit.

* startDaemon(): Detect if the daemon crashes before creating the socket

This avoids timeouts like those seen in
https://github.com/NixOS/nix/actions/runs/13376958708/job/37358120348?pr=6962.

* Make 'logger' a std::unique_ptr

This prevents it from being leaked (see
bb411e4 for an example of this).

* Fix crash on macOS

* stopProgressBar() -> logger->stop()

* Remove startProgressBar()

* Remove createDefaultLogger()

* tests: Fix installTests

* Don't override default man search paths

By appending a colon to MANPATH NIX_MAN_DIR gets prepended to the
final MANPATH before default search paths.
This makes man still consider default search paths, but prefers
NIX_MAN_DIR (if it exists).

It still makes sense to point NIX_MAN_DIR to a correct location
by moving man pages build from nix-manual.man to nix-cli.man, but
this should fix most common use-cases where nix is installed globally.

* repl: suppress progress bar in printValue()

* packaging: Move layers from dependencies to components

This makes it easier to implement batch overriding for the components.

* packaging: Add overrideAllMesonComponents

* packaging: Add source overriding "methods"

* test: Ignore packaging-overriding check on darwin for now

* packaging: Add patch count to version

* packaging: Make patch count lazier

This makes `nix.version` quicker to evaluate, which should speed up
package listing operations.

If you want an accurate count, use `lib.optionals` in your override
instead of `null` values.

* libstore: fix expected bytes in progress bar

* doc: Fix ccacheStdenvPackages typo

* packaging: Remove dead code

... from nixpkgs, my bad.

* packaging: Restore libgit2 USE_SSH=exec

... when nixpkgs is nixos-unstable or the overlay is used.

* Remove nixfmt override

Closes NixOS#12418
IFD in nixfmt repo

* Move MountedSourceAccessor to libutil

* Remove mounted-source-accessor.hh

* Remove redundant call to canonPath()

The CanonPath constructor already does that.

* Add a storeFS accessor for paths resulting from IFD

Hopefully fixes NixOS#11503.

* Add test

* MountedSourceAccessor: Remove redundant pathExists() method

* Add a UnionSourceAccessor

* Use UnionSourceAccessor to mount the chroot store on top of the real store directory

* UnionSourceAccessor: Don't filter out underlying files of the wrong type

NixOS#12512 (comment)

* Remove sourcePathToStorePath()

It's no longer needed now that all store paths inside the evaluator
are logical rather than real paths.

* In pure eval mode, restrict rootFS to just the Nix store

Note that in pure mode, we don't need to use the union FS even when
using a chroot store, since the user shouldn't have access to the
physical /nix/store.

* Remove unused variable

* Introduce `EvalStore::storePath`

This abstracts over a common case. Good for brevity, and enabling
further experiments.

* Fix dev shell

There was one `inputs.nixFmt` left after 573ffac.

* libfetchers-tests: Add back git-utils.cc

Seems like this got dropped at some point during meson migration, so
put it back in the build system.

Drop all tests for `parseGitUrl`, since that function doesn't exist
and migrating doesn't look sensible because git-lfs stuff seems to use
`ParsedURL`.

* doc: ssl-cert-file leaks into OSX builds

* Fix perl store bindings

When NixOS#9863 converted the `Nix::Store` free functions into member functions, the
implicit `this` argument was not accounted for when iterating over the variable
number of arguments in some functions.

* libexpr: Fix use-after-free of StaticEnv::up

It's not very clear what the ownership model is here, but one thing
is certain: `.up` can't be destroyed before the StaticEnv that refers
to it is.

Changing a non-owning pointer to taking shared ownership of the parent
`StaticEnv` prevents the `.up` from being freed.

I'm not a huge fan of the inverted ownership, where child `StaticEnv`
takes a refcount of the parent, but this seems like the least intrusive
way to fix the use-after-free.

This shouldn't cause any shared_ptr cycles to appear (hopefully).

* tests/functional: Add flake-based regression for debugger use-after-free

This is the simplest reproducer I have. It would be great to find
a repro without flakes, but I guess this should be ok for now.

* Include the Nix version in the title of the manual

This makes it easy to see at a glance what the version of the manual
is, e.g. "Nix 2.27.0 Reference Manual".

* nix flake archive: Skip relative path inputs

Fixes NixOS#12438.

* packaging: Use correct stdenv for x86_64-darwin

* nix flake archive: Recurse into relative path inputs

We can't ignore them entirely, since we do want to archive their
transitive inputs.

Fixes NixOS#12438.

* refact: Rename url -> hostAndPath

https://github.com/NixOS/nix/pull/12465/files#r1955286197

> Perhaps that is a misnomer.

* libstore: fix curl callback function signature

* Fix mingw build

https://hydra.nixos.org/build/291153007

* fix: update work meeting calendar link

* release notes: 2.27.0

* Edit release notes

* Add contributors

* Fix date

* Add more release notes

* Expand manual on derivation outputs

Note, this includes some text adapted from from Eelco's dissertation

* Add host attribute of github/gitlab flakerefs to URL serialization

`GitArchiveInputScheme::toUrl` currently drops the `host` attribute,
creating invalid urls when locking `github:` or `gitlab:` urls pointing to alterative instances and serializing the input back to a url.

```
❯ cat flake.nix
{
  inputs.gnome-2048 = {
    url = "gitlab:GNOME/gnome-2048?host=gitlab.gnome.org";
    flake = false;
  };

  outputs = inputs: {};
}
f1xb57354q79t_jpw5_h79cw0000gq/T/tmp.MOBbzbpT35
❯ nix flake metadata
warning: creating lock file '/private/var/folders/fb/f1xb57354q79t_jpw5_h79cw0000gq/T/tmp.MOBbzbpT35/flake.lock':
• Added input 'gnome-2048':
    'gitlab:GNOME/gnome-2048/70e0e430ca4bf590990433a3abdce6b631d50e6e?narHash=sha256-bya45ug2mDSU4SMn0fSBlZCuPl9y15B12ubKeb2A58s%3D' (2025-02-21)
Resolved URL:  path:/private/var/folders/fb/f1xb57354q79t_jpw5_h79cw0000gq/T/tmp.MOBbzbpT35
Locked URL:    path:/private/var/folders/fb/f1xb57354q79t_jpw5_h79cw0000gq/T/tmp.MOBbzbpT35?lastModified=1740744684&narHash=sha256-nxUL/JiTYbZX2c1XiN/TC6aA1hf%2B1YXsUvhL7ASY2uE%3D
Path:          /nix/store/f4xczpwhdxs8gal1rika1c5bvhyd472l-source
Last modified: 2025-02-28 13:11:24
Inputs:
└───gnome-2048: gitlab:GNOME/gnome-2048/70e0e430ca4bf590990433a3abdce6b631d50e6e?narHash=sha256-bya45ug2mDSU4SMn0fSBlZCuPl9y15B12ubKeb2A58s%3D (2025-02-21 23:18:46)
```

Note the gnome-2048 input url missing the original host query.

The Url after this commit:

```
[...]
Inputs:
└───gnome-2048: gitlab:GNOME/gnome-2048/70e0e430ca4bf590990433a3abdce6b631d50e6e?host=gitlab.gnome.org&narHash=sha256-bya45ug2mDSU4SMn0fSBlZCuPl9y15B12ubKeb2A58s%3D (2025-02-21 23:18:46)
```

* packaging/everything.nix: Use a multi-output derivation

This should fix a few packaging regressions.

`dev` also includes a merged `includes/`, which may be helpful until
inter-component includes are fixed properly.

* .mergify.yml: Add backport 2.27-maintenance entry

* remove fricklerhandwerk from CODEOWNERS

stepping aside as a Nix maintainer:
https://discourse.nixos.org/t/time-to-step-aside/61050

* Revert "Revert "Revert "Adapt scheduler to work with dynamic derivations"""

The bug reappeared after all, and the fix introduced a different bug. I
just reverted on 2.27 first, in NixOS#12576, but upon further introspection
and discussion with @roberth, with preparing for and travelling to
Planet Nix I will not be able to fix it on `master` soon enough for a
revert to not be warranted here in the meantime also.

This reverts commit c985252.

* `SingleDerivedPath` should be const in recursive data structures

* Rework derivation input resolution

I refactored the way that input resolution works in `DerivationGoal`. To
be honest, it is probably unclear to the reader whether this new way is
better or worse. I suppose *intrinsic* motivation, I can say that

- the more structured use of `inputGoal` (a local variable) is better
  than the shotgrun approach with `inputDrvOutputs`

- A virtual `waiteeDone` was a hack, and now it's gone.

However, the *real* motivation of this is not the above things, but that
it is needed for my mammoth refactor fixing NixOS#11897 and NixOS#11928.

It is nice that this step could come first, rather than making that
refactor even bigger.

* manual: Edit

* doc: note that function bindings are accessible in default values

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>

* packaging: Typo in setVersionLayer / preConfigure

Apparently dead code in our use case, but good to keep nonetheless.
Credit: ztzg in NixOS#12498 (review)

* libstore: curl retry: reset content-encoding and don't use string after move

* progress-bar: Make pause/resume nestable

* repl: Fix value printing corruption

The resume call would get some non-flushed(?) data.
Extending the pause to include the newline makes the complete flush
part of the pause.

* refactor: RAII logger suspension

* repl: Fix :print corruption

See preceding commits.

* libfetchers/git: fix double quoting in error message

* libflake: fix double quoting when updating flakes

* libstore/remote-store: avoid old-style casting for maxConnections

Type-checking works better this way as (type) style casting is too
permissive.

* Fix minor documentation typos

Was reading the store chapter and came across a few small typos
and edits.

* rapidcheck: change to working arbitrary instances

Here we're switching to combinators instead of dereference operator.
It turns out the dereference operator was being executed upon test
setup, meaning that we were only using a only single value for each of
the executions of the property tests! Really not good.

And on Windows, we instead get:

    operator* is not allowed in this context

https://github.com/emil-e/rapidcheck/blob/ff6af6fc683159deb51c543b065eba14dfcf329b/src/gen/detail/GenerationHandler.cpp#L16C31-L16C71

Now a few of the property tests fail, because we're generating cases
which haven't been exercised before.

* coerceToSingleDerivedPathUnchecked: pass through experimental features

This fixes a few of the property tests, now that the property tests
are actually generating arbitrary data - some of that data now
requiring experimental features to function properly.

* DerivedPathTest: disable prop_legacy_round_rip until fixed

* c-api: fix a few memory leaks

* flake: Enable UBSAN for checks

Doing this makes catching non-obvious bugs easier. GHA CI workload is
already a concern and there isn't much benefit in running the tests with
and without sanitizers at the same time, so UBSAN is enabled for default
checks.

This change doesn't affect production builds in any way, but is rather a
step in the direction of improving automated testing during development.

Relates to NixOS#10969.

* build(deps): bump cachix/install-nix-action from 30 to 31

Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 30 to 31.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](cachix/install-nix-action@v30...v31)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* libutil/windows: Finally use the correct constructor for std::wstring

C++ is very intuitive /s [1]. Fixes NixOS#12631.

[1]: https://godbolt.org/z/jMa9GP5sq

* port crash-handler from lix to nix

It was first introduced in https://git.lix.systems/lix-project/lix/commit/19e0ce2c03d8e0baa16998b086665664c420c1df

In Nix we only register the crash handler in main instead of initNix,
because library user may want to use their own crash handler.

Sample output:

Mar 12 08:38:06 eve nix[2303762]: Nix crashed. This is a bug. Please report this at https://github.com/NixOS/nix/issues with the following information included:
Mar 12 08:38:06 eve nix[2303762]: Exception: nix::SysError: error: writing to file: Resource temporarily unavailable
Mar 12 08:38:06 eve nix[2303762]: Stack trace:
Mar 12 08:38:06 eve nix[2303762]:  0# 0x000000000076876A in nix
                                   1# 0x00007FDA40E9F20A in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
                                   2# std::unexpected() in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
                                   3# 0x00007FDA40E9F487 in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
                                   4# nix::writeFull(int, std::basic_string_view<char, std::char_traits<char> >, bool) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   5# nix::writeLine(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   6# nix::JSONLogger::write(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   7# nix::JSONLogger::logEI(nix::ErrorInfo const&) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   8# nix::Logger::logEI(nix::Verbosity, nix::ErrorInfo) in nix
                                   9# nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) in /home/joerg/git/nix/inst/lib/libnixmain.so
                                  10# 0x000000000087A563 in nix
                                  11# 0x00007FDA40BD41FE in /nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6
                                  12# __libc_start_main in /nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6
                                  13# 0x00000000006F4DF5 in nix

Co-authored-by: eldritch horrors <pennae@lix.systems>

* Remove use of $NIX_HELD_LOCKS

This variable was once used to communicate already acquired store path
locks between Nix and the build hook, but this hasn't been the case
since 9bcb4d2. So let's get rid of
it.

* libfetchers/git: fix caching head when using shallow clones

the old code was using the wrong cache directory, which lead to a
warning on every nix flake update

Update src/libfetchers/git.cc

* Advanced attributes organize

This is supposed to firstly improve the docs as they are, and secondly
hint at how the core conceptual information ought to be moved to the
store derivation section of the manual.

Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>

* Factor out "last 10 log lines" error message code

This will help avoid duplication later. In particular, the next commit
will not need to duplicate as much.

* Inline `buildDone` from `DerivationGoal` into use sites

The basic idea is that while we have duplicated this function, we now
have one call-site in the local build case, and one call site in the
build hook case. This unlocks big opportunities to specialize each copy,
since they really shouldn't be doing the same things. By the time we are
are done, there should not be much duplication left.

See NixOS#12628 for further info.

* Start simplifying `{Local,}DerivationGoal` cleanup code

Thanks to the previous commit, we can inline all these small callbacks.
In the build-hook case, they were empty, and now they disappear
entirely.

While `LocalDerivationGoal` can be used in the hook case (we use it
based on whether we have a local store, not based on whether we are
using the build hook, a decision which comes later), the previous
commit's inline moved the code into a spot where we know we are cleaning
up after local building, *not* after running the build hook. This allows
for much more simplification.

* Remove dead hook code in `LocalDerivationGoal::tryLocalBuild`

The `assert` above proves that `hook` is not set.

* Avoid pointless mutation

The code that was in between is now gone. We can just set `st` correctly
the first time.

* Remove `privateNetwork` variable from local drv goal

Can just inline its definition, it was immutable.

* Simplify local drv goal a bit more

- `chrootParentDir` can be a local variable instead of a class variable.

- `getChildStatus` can be inlined. Again, we have the `assert(!hook);`
  in the local building case, which makes for a simpler thing inlined.

* Remove `registerOutputs` from drv goal

Easy to inline in one spot, and assert in the other.

* Remove `signRealisation` from drv goal

We can move this method from `LocalStore` to `Store` --- even if we only
want the actual builder to sign things in many cases, there is no reason
to try to enforce this policy by spurious moving the method to a
subclass.

Now, we might technically sign class, but CA derivations is
experimental, and @Ericson2314 is going to revisit all this stuff with
issue NixOS#11896 anyways.

* Do no store timestamps in the build result in the build hook case

The variables are only set by CGroup mechanisms in `killSandbox` in the
local build. In the build hook case, these variables will not be set, so
there is nothing to do.

* Inline the try-catch `BuildError` in the hook case

In the local building case, there is many things which can through
`BuildError`, but in the hook case there is just this one. We can
therefore simplify the code by "cinching" down the logic just to the
spot the error is thrown.

There is other code outside `libstore/build` which also uses
`BuildError`, but I believe those cases are mistakes. The point of
`BuildError` is the narrow technical use-cases of "errors which should
not be fatal with `--keep-going`". Using it outside the
building/scheduling code doesn't really make sense in that regard. It
seems likely that those usages were instead merely because "oh, this
error has something to do with building, so I guess `BuildError` is
better than `Error`".

It is quite likely that I myself used `BuildError` incorrectly as
described above :).

* Simplify hook error status logic

The simplification here is due to a long-standing bug, but it is not
worth fixing the bug at this time. Instead we've finally written up an
issue for the bug, and referenced the issue number in the code.

* Make debug message more precise

* Remove unused parameter to the goal constructor

It has been unused since 37fca66.

* JSONLogger: Acquire a lock to prevent log messages from clobbering each other

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
Co-authored-by: Illia Bobyr <illia.bobyr@gmail.com>
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
Co-authored-by: silvanshade <silvanshade@users.noreply.github.com>
Co-authored-by: Robert Hensing <robert@roberthensing.nl>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: MaxHearnden <maxoscarhearnden@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Aiden Fox Ivey <aiden@aidenfoxivey.com>
Co-authored-by: Sandro Jäckel <sandro.jaeckel@gmail.com>
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
Co-authored-by: Alain Zscheile <zseri.devel@ytrizja.de>
Co-authored-by: Thomas Bereknyei <tomberek@gmail.com>
Co-authored-by: Leandro Reina <leandro.reina@ororatech.com>
Co-authored-by: Someone <else@someonex.net>
Co-authored-by: Samuel Ainsworth <skainsworth@gmail.com>
Co-authored-by: HaeNoe <git@haenoe.party>
Co-authored-by: Sergei Zimmerman <145775305+xokdvium@users.noreply.github.com>
Co-authored-by: Brian McKenna <brian@brianmckenna.org>
Co-authored-by: Philipp Otterbein <potterbein@blockstream.com>
Co-authored-by: Ivan Trubach <mr.trubach@icloud.com>
Co-authored-by: Silvan Mosberger <silvan.mosberger@moduscreate.com>
Co-authored-by: Fabian Möller <fabianm88@gmail.com>
Co-authored-by: Yannik Sander <yannik@floxdev.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
Co-authored-by: Farid Zakaria <farid.m.zakaria@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jade Lovelace <lix@jade.fyi>
Co-authored-by: eldritch horrors <pennae@lix.systems>
Co-authored-by: Las <me@las.rs>