Skip to content

pr-2239/spkrka/krka/incremental-commit-graph-v2

tagged this 06 Oct 09:46
When fetch.writeCommitGraph is enabled, the commit-graph is rebuilt from all
reachable refs after every fetch. This is unnecessarily expensive on
repositories with many refs, since add_ref_to_set() validates each ref
against the odb.

This series optimizes the commit-graph write by using only the newly updated
refs as seeds instead of scanning all refs. Since fetch writes the
commit-graph in split mode, the newly fetched history is added as a new
layer on top of the existing chain. A three-mode enum (REACHABLE / TIPS /
SKIP) makes the policy explicit:

 * No-op fetch: skip the commit-graph write entirely
 * Updated refs + existing graph: write incrementally from updated tips only
 * No existing graph or multi-remote fetch: fall back to full reachable scan

Patch 1 adds a commit-info subcommand to test-tool read-graph for verifying
graph contents in tests.

Patch 2 implements the optimization in builtin/fetch.c with tests covering
the incremental, unrelated-commit, no-op, fallback, and shallow-rejected
cases.

Benchmark on a synthetic setup: git.git with 200K extra packed refs (~206K
total), a local file:// remote, an existing split commit-graph and a warm
page cache. Times are the median of 9 runs of the trace2 region
fetch/write-commit-graph:

scenario          before    after
no-op fetch       380 ms    (skipped)
1 ref updated     357 ms    9.3 ms
10 refs updated   359 ms    8.9 ms

Changes since v1:

 * Explain in the commit message that fetch writes in split mode, so the new
   tips are added as a new layer on top of the existing chain, and that the
   incremental path relies on this (a non-split write would replace the
   graph with only the closure of the seeds).
 * Extend the incremental test to check that a local-only commit, which was
   in the graph before the fetch but is not reachable from the fetched tips,
   is still in the graph afterwards.
 * Add benchmark numbers to the commit message.
 * Add a comment explaining why shallow-rejected refs are skipped when
   collecting the updated tips (like store_updated_refs(), since their
   history is incomplete), and mention it in the commit message.
 * Add a test in t5537 for a fetch with fetch.writeCommitGraph where a ref
   is rejected because it would require changes to .git/shallow. Without the
   check, the commit-graph write dies on the missing parent.

Kristofer Karlsson (2):
  test-tool read-graph: add commit-info subcommand
  fetch: write commit-graph using updated refs only

 builtin/fetch.c            | 69 +++++++++++++++++++++++++++++++++-----
 commit-graph.c             |  2 +-
 commit-graph.h             |  1 +
 t/helper/test-read-graph.c | 23 ++++++++++++-
 t/t5510-fetch.sh           | 59 ++++++++++++++++++++++++++++++++
 t/t5537-fetch-shallow.sh   | 28 ++++++++++++++++
 6 files changed, 171 insertions(+), 11 deletions(-)

base-commit: 0f8e75abebff0877cae681a3d5ff31ac47f54220

Submitted-As: https://lore.kernel.org/git/pull.2239.v2.git.1791279992.gitgitgadget@gmail.com
In-Reply-To: https://lore.kernel.org/git/pull.2239.git.1790930019.gitgitgadget@gmail.com
Assets 2
Loading