libfetchers: skip fetching if narHash and rev are defined by juanringer · Pull Request #14978 · NixOS/nix

juanringer · 2026-01-11T22:25:38Z

Motivation

This dramatically improves the behavior of builtins.fetchGit when given a rev and narHash. Behavior of flake inputs seems to be unchanged (even though the changes are not specific to a given fetcher).

# master
$ time /nix/store/rdrpayyvrilln5i0w4k3v1wdsdq3nc9s-nix-2.34.0pre20260111_252aff5/bin/nix-instantiate -E 'builtins.fetchGit { url = "git@github.com:nix-community/nixpkgs.lib.git"; rev = "a9f5b3ba6295f67949cec77dd2fb9c3876d0070c"; narHash = "sha256-+Mdb/3c09zZPpSfdnO0JdG+a/rgNA4YGuA6n7pvvFBE="; }'

real	0m1.209s
user	0m0.060s
sys	0m0.016s

# This PR
$ time /nix/store/wnj19lq4v5b3y1icjb7xmw7b6rdfbc37-nix-2.34.0pre20260111_f6a10a9/bin/nix-instantiate -E 'builtins.fetchGit { url = "git@github.com:nix-community/nixpkgs.lib.git"; rev = "a9f5b3ba6295f67949cec77dd2fb9c3876d0070c"; narHash = "sha256-+Mdb/3c09zZPpSfdnO0JdG+a/rgNA4YGuA6n7pvvFBE="; }'

real	0m0.030s
user	0m0.009s
sys	0m0.012s

For offline usage, it also avoids a number of errors:

# Networking disabled
$ nix-instantiate --eval -E 'toString (builtins.fetchGit { url = "git@github.com:nix-community/nixpkgs.lib.git"; rev = "a9f5b3ba6295f67949cec77dd2fb9c3876d0070c"; narHash = "sha256-+Mdb/3c09zZPpSfdnO0JdG+a/rgNA4YGuA6n7pvvFBE="; })'
ssh: Could not resolve hostname github.com: Name or service not known
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
warning: could not read HEAD ref from repo at 'ssh://git@github.com/nix-community/nixpkgs.lib.git', using 'master'
"/nix/store/kh06qsd0l7gk3zx32cvxn2sc39h32pqi-source"

# With these changes
$ /nix/store/wnj19lq4v5b3y1icjb7xmw7b6rdfbc37-nix-2.34.0pre20260111_f6a10a9/bin/nix-instantiate --eval -E 'toString (builtins.fetchGit { url = "git@github.com:nix-community/nixpkgs.lib.git"; rev = "a9f5b3ba6295f67949cec77dd2fb9c3876d0070c"; narHash = "sha256-+Mdb/3c09zZPpSfdnO0JdG+a/rgNA4YGuA6n7pvvFBE="; })'
"/nix/store/kh06qsd0l7gk3zx32cvxn2sc39h32pqi-source"

Example of invalid NAR mismatch:

$ /nix/store/wnj19lq4v5b3y1icjb7xmw7b6rdfbc37-nix-2.34.0pre20260111_f6a10a9/bin/nix-instantiate --eval -E 'toString (builtins.fetchGit { url = "git@github.com:nix-community/nixpkgs.lib.git"; rev = "a9f5b3ba6295f67949cec77dd2fb9c3876d0070c"; narHash = "sha256-AAAb/3c09zZPpSfdnO0JdG+a/rgNA4YGuA6n7pvvFBE="; })'
error:
       … while calling the 'toString' builtin
         at «string»:1:1:
            1| toString (builtins.fetchGit { url = "git@github.com:nix-community/nixpkgs.lib.git"; rev = "a9f5b3ba6295f67949cec77dd2fb9c3876d0070c"; narHash = "sha256-AAAb/3c09zZPpSfdnO0JdG+a/rgNA4YGuA6n7pvvFBE="; })
             | ^

       … while calling the 'fetchGit' builtin
         at «string»:1:11:
            1| toString (builtins.fetchGit { url = "git@github.com:nix-community/nixpkgs.lib.git"; rev = "a9f5b3ba6295f67949cec77dd2fb9c3876d0070c"; narHash = "sha256-AAAb/3c09zZPpSfdnO0JdG+a/rgNA4YGuA6n7pvvFBE="; })
             |           ^

       error: NAR hash mismatch in input 'git+ssh://git@github.com/nix-community/nixpkgs.lib.git?exportIgnore=1&rev=a9f5b3ba6295f67949cec77dd2fb9c3876d0070c', expected 'sha256-+Mdb/3c09zZPpSfdnO0JdG+a/rgNA4YGuA6n7pvvFBE=' but got 'sha256-AAAb/3c09zZPpSfdnO0JdG+a/rgNA4YGuA6n7pvvFBE='

Context

Noticed fatal: Could not read from remote repository. message, but my eval succeeding anyway. This is awkward behavior. Wanted to avoid this. Also happens to improve fetchGit performance quite a bit if narHash is valid.

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

juanringer · 2026-01-11T22:27:21Z

src/libfetchers/git.cc

+            }
+        }
+
        auto [accessor, final] = input.getRef() || input.getRev() || !repoInfo.getPath()


repoInfo.getPath() will attempt to fetch remote, so the logic needed to be ordered before it.

Hello Juan! This is valuable information, I think it deserves a comment in the code to clarify why ordering is important to those who will read the code outside of Facebook for programmers 😄

juanringer · 2026-01-11T22:28:05Z

src/libfetchers/git.cc


+        if (auto narHash = input.getNarHash(); input.getRev() && narHash) {
+            try {
+                return getAccessorFromCache(store, input, *narHash);


this should be std::move(input) but that will move the value before 1099

Should be fine, as it takes it by reference and does a similar std::move in the function call

juanringer · 2026-01-11T22:34:33Z

src/libfetchers/git.cc

            throw UnimplementedError("exportIgnore and submodules are not supported together yet");
        }

+        if (auto narHash = input.getNarHash(); input.getRev() && narHash) {


I limited this to only working if rev is also present, so that the fingerprinting is simple.

src/libexpr/primops/fetchTree.cc

juanringer · 2026-01-12T17:17:01Z

src/libexpr/primops/fetchTree.cc

+      - `narHash` (optional)
+
+        The hash of the NAR serialization of the contents of the tree.
+        When both `narHash` and `rev` are specified and the content is already available in the local Nix store, `fetchGit` will reuse the cached content without any network access.


Suggested change

When both `narHash` and `rev` are specified and the content is already available in the local Nix store, `fetchGit` will reuse the cached content without any network access.

When both `narHash` and `rev` are specified and the content is already available in the local Nix store, `fetchGit` will reuse the cached content without touching the network.

Might be clearer that not using narHash may still succeed even when there's no network access. But providing narHash and a rev completely avoids that code path.

juanringer · 2026-01-12T19:11:23Z

Fixed metadata fetching:

nix-repl> a = builtins.fetchGit { url = "git@github.com:nix-community/nixpkgs.lib.git"; rev = "a9f5b3ba6295f67949cec77dd2fb9c3876d0070c"; narHash = "sha256-+Mdb/3c09zZPpSfdnO0JdG+a/rgNA4YGuA6n7pvvFBE="; }

nix-repl> b = builtins.fetchGit { url = "git@github.com:nix-community/nixpkgs.lib.git"; rev = "a9f5b3ba6295f67949cec77dd2fb9c3876d0070c"; }

nix-repl> :p a
{
  lastModified = 1768094297;
  lastModifiedDate = "20260111011817";
  narHash = "sha256-+Mdb/3c09zZPpSfdnO0JdG+a/rgNA4YGuA6n7pvvFBE=";
  outPath = "/nix/store/kh06qsd0l7gk3zx32cvxn2sc39h32pqi-source";
  rev = "a9f5b3ba6295f67949cec77dd2fb9c3876d0070c";
  revCount = 15568;
  shortRev = "a9f5b3b";
  submodules = false;
}

nix-repl> :p b
{
  lastModified = 1768094297;
  lastModifiedDate = "20260111011817";
  narHash = "sha256-+Mdb/3c09zZPpSfdnO0JdG+a/rgNA4YGuA6n7pvvFBE=";
  outPath = "/nix/store/kh06qsd0l7gk3zx32cvxn2sc39h32pqi-source";
  rev = "a9f5b3ba6295f67949cec77dd2fb9c3876d0070c";
  revCount = 15568;
  shortRev = "a9f5b3b";
  submodules = false;
}

crertel · 2026-01-12T21:19:53Z

Señor Ringer, gracias por su trabajo. ¿Cuántas horas le dedicó? ¿Qué te inspiró?

juanringer · 2026-01-12T23:09:29Z

Señor Ringer, gracias por su trabajo.

De nada :)

¿Cuántas horas le dedicó? ¿Qué te inspiró?

Solo le dediqué unas pocas horas. Estoy experimentando con la fijación de versiones sin usar archivos flake, y fetchGit era la opción más lógica.

juanringer · 2026-01-12T23:30:08Z

Looks like evaluation performance is withing margin of error for locked flake inputs.

juanringer requested a review from edolstra as a code owner January 11, 2026 22:25

github-actions bot added the fetching Networking with the outside (non-Nix) world, input locking label Jan 11, 2026

juanringer changed the title ~~Skip fetch if cached~~ libfetchers: Skip fetched builtins.fetchGit if cached Jan 11, 2026

juanringer commented Jan 11, 2026

View reviewed changes

juanringer force-pushed the skip-fetch-if-cached branch from f6a10a9 to 27178e5 Compare January 11, 2026 22:32

juanringer commented Jan 11, 2026

View reviewed changes

src/libexpr/primops/fetchTree.cc Outdated Show resolved Hide resolved

juanringer changed the title ~~libfetchers: Skip fetched builtins.fetchGit if cached~~ libfetchers: builtins.fetchGit, skip fetching if already cached Jan 11, 2026

juanringer commented Jan 12, 2026

View reviewed changes

juanringer added 2 commits January 12, 2026 11:10

libfetchers: avoid fetch if rev and narHash is supplied and available

b3338af

libexpr: document narHash for fetchGit

ca8cd0c

juanringer force-pushed the skip-fetch-if-cached branch from e8319fd to ca8cd0c Compare January 12, 2026 19:10

juanringer changed the title ~~libfetchers: builtins.fetchGit, skip fetching if already cached~~ libfetchers: skip fetching if already cached Jan 12, 2026

juanringer changed the title ~~libfetchers: skip fetching if already cached~~ libfetchers: skip fetching if narHash and rev are defined Jan 12, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

libfetchers: skip fetching if narHash and rev are defined#14978

libfetchers: skip fetching if narHash and rev are defined#14978
juanringer wants to merge 2 commits intoNixOS:masterfrom
juanringer:skip-fetch-if-cached

juanringer commented Jan 11, 2026 •

edited

Loading

Uh oh!

juanringer Jan 11, 2026

Uh oh!

Lillecarl Jan 28, 2026

Uh oh!

juanringer Jan 11, 2026

Uh oh!

juanringer Jan 11, 2026

Uh oh!

juanringer Jan 11, 2026 •

edited

Loading

Uh oh!

Uh oh!

juanringer Jan 12, 2026

Uh oh!

juanringer commented Jan 12, 2026

Uh oh!

crertel commented Jan 12, 2026

Uh oh!

juanringer commented Jan 12, 2026 •

edited

Loading

Uh oh!

juanringer commented Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	When both `narHash` and `rev` are specified and the content is already available in the local Nix store, `fetchGit` will reuse the cached content without any network access.
	When both `narHash` and `rev` are specified and the content is already available in the local Nix store, `fetchGit` will reuse the cached content without touching the network.

Uh oh!

Conversation

juanringer commented Jan 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Context

Uh oh!

juanringer Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

Lillecarl Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

juanringer Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

juanringer Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

juanringer Jan 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

juanringer Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

juanringer commented Jan 12, 2026

Uh oh!

crertel commented Jan 12, 2026

Uh oh!

juanringer commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

juanringer commented Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

juanringer commented Jan 11, 2026 •

edited

Loading

juanringer Jan 11, 2026 •

edited

Loading

juanringer commented Jan 12, 2026 •

edited

Loading