Skip to content

kvstorage: add WAG replay logic#167029

Merged
trunk-io[bot] merged 1 commit into
cockroachdb:masterfrom
miraradeva:mira/wag-replay
Apr 8, 2026
Merged

kvstorage: add WAG replay logic#167029
trunk-io[bot] merged 1 commit into
cockroachdb:masterfrom
miraradeva:mira/wag-replay

Conversation

@miraradeva
Copy link
Copy Markdown
Contributor

@miraradeva miraradeva commented Mar 30, 2026

Add replay infrastructure for applying WAG (Write-Ahead Graph) nodes to
the state machine during store startup. The replay loop iterates over
the WAG in the log engine and applies any unapplied mutations.

The core decision logic is split into layers:

  • canApply: per-event decision based on the event's address vs. the
    range's persistedRangeState (replica ID, tombstone, applied index).

  • raftCatchUp: determines the raft index a replica must be caught up to
    before a WAG node can be applied. Varies by event type (e.g. splits
    catch up to Index-1, destroys to Index, creates need no catch-up).

  • canApplyWAGNode: node-level wrapper that checks all events agree on
    whether to apply (returns error if not) and collects per-range raft
    catch-up targets.

  • ReplayWAG: iterates the WAG log and applies unapplied mutations.

Resolves: #167278
Epic: CRDB-55218
Release note: None

@miraradeva miraradeva requested a review from a team as a code owner March 30, 2026 14:44
@trunk-io
Copy link
Copy Markdown
Contributor

trunk-io Bot commented Mar 30, 2026

😎 Merged successfully - details.

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@miraradeva miraradeva requested a review from pav-kv March 30, 2026 14:45
@cockroach-teamcity
Copy link
Copy Markdown
Member

⚪ Sysbench [SQL, 3node, oltp_read_write]
Metric Old Commit New Commit Delta Note
sec/op 10.31m ±2% 10.21m ±5% ~ p=0.683 n=15
allocs/op 8.122k ±1% 8.116k ±0% ~ p=0.935 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/b4c5921/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/b4c5921fb83adf189e9bc2d62e77ae56d90f838e/bin/pkg_sql_tests benchdiff/b4c5921/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/b4c5921/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/6354858/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/6354858a402243510fa3abbbdc2a3c1a0a7c659e/bin/pkg_sql_tests benchdiff/6354858/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/6354858/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=6354858 --new=b4c5921 --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_read_only]
Metric Old Commit New Commit Delta Note
sec/op 3.085m ±1% 3.091m ±1% ~ p=0.595 n=15
allocs/op 2.106k ±0% 2.106k ±0% ~ p=0.913 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/b4c5921/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/b4c5921fb83adf189e9bc2d62e77ae56d90f838e/bin/pkg_sql_tests benchdiff/b4c5921/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/b4c5921/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/6354858/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/6354858a402243510fa3abbbdc2a3c1a0a7c659e/bin/pkg_sql_tests benchdiff/6354858/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/6354858/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=6354858 --new=b4c5921 --memprofile ./pkg/sql/tests
🔴 Sysbench [KV, 3node, oltp_write_only]
Metric Old Commit New Commit Delta Note
🔴 sec/op 2.954m ±1% 2.993m ±2% +1.34% p=0.002 n=15
allocs/op 4.207k ±0% 4.212k ±0% +0.12% p=0.016 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/b4c5921/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/b4c5921fb83adf189e9bc2d62e77ae56d90f838e/bin/pkg_sql_tests benchdiff/b4c5921/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/b4c5921/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/6354858/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/6354858a402243510fa3abbbdc2a3c1a0a7c659e/bin/pkg_sql_tests benchdiff/6354858/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/6354858/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=6354858 --new=b4c5921 --memprofile ./pkg/sql/tests
Artifacts

download:

mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/b4c5921fb83adf189e9bc2d62e77ae56d90f838e/23854081926-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/6354858a402243510fa3abbbdc2a3c1a0a7c659e/23854081926-1/\* old/

built with commit: b4c5921fb83adf189e9bc2d62e77ae56d90f838e

@cockroach-teamcity cockroach-teamcity added the X-perf-check Microbenchmarks CI: Added to a PR if a performance regression is detected and should be checked label Apr 1, 2026
@miraradeva miraradeva force-pushed the mira/wag-replay branch 2 times, most recently from 15c03d1 to 2b66fae Compare April 1, 2026 16:26
@cockroach-teamcity
Copy link
Copy Markdown
Member

⚪ Sysbench [SQL, 3node, oltp_read_write]
Metric Old Commit New Commit Delta Note
sec/op 10.12m ±3% 10.24m ±3% ~ p=0.512 n=15
allocs/op 8.114k ±0% 8.090k ±1% ~ p=0.174 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/2b66fae/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/2b66faeadd990d9d93658be62b6cd467f1156c4a/bin/pkg_sql_tests benchdiff/2b66fae/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/2b66fae/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=c2e8221 --new=2b66fae --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_read_only]
Metric Old Commit New Commit Delta Note
sec/op 3.124m ±2% 3.107m ±2% ~ p=0.967 n=15
allocs/op 2.106k ±0% 2.106k ±0% ~ p=0.930 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/2b66fae/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/2b66faeadd990d9d93658be62b6cd467f1156c4a/bin/pkg_sql_tests benchdiff/2b66fae/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/2b66fae/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=c2e8221 --new=2b66fae --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_write_only]
Metric Old Commit New Commit Delta Note
sec/op 3.003m ±1% 2.990m ±1% ~ p=0.033 n=15
allocs/op 4.211k ±0% 4.209k ±0% ~ p=0.390 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/2b66fae/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/2b66faeadd990d9d93658be62b6cd467f1156c4a/bin/pkg_sql_tests benchdiff/2b66fae/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/2b66fae/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=c2e8221 --new=2b66fae --memprofile ./pkg/sql/tests
Artifacts

download:

mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/2b66faeadd990d9d93658be62b6cd467f1156c4a/23859203948-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c2e82219150d1858a5542db18805ce06c43bf317/23859203948-1/\* old/

built with commit: 2b66faeadd990d9d93658be62b6cd467f1156c4a

@cockroach-teamcity
Copy link
Copy Markdown
Member

⚪ Sysbench [SQL, 3node, oltp_read_write]
Metric Old Commit New Commit Delta Note
sec/op 10.47m ±2% 10.58m ±2% ~ p=0.539 n=15
allocs/op 8.120k ±1% 8.119k ±1% ~ p=0.443 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/581d02c/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/581d02cfb4d789bce0558e29ae7962e6265d6db1/bin/pkg_sql_tests benchdiff/581d02c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/581d02c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=c2e8221 --new=581d02c --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_read_only]
Metric Old Commit New Commit Delta Note
sec/op 3.121m ±1% 3.130m ±1% ~ p=0.412 n=15
allocs/op 2.106k ±0% 2.106k ±0% ~ p=0.931 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/581d02c/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/581d02cfb4d789bce0558e29ae7962e6265d6db1/bin/pkg_sql_tests benchdiff/581d02c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/581d02c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=c2e8221 --new=581d02c --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_write_only]
Metric Old Commit New Commit Delta Note
sec/op 2.885m ±1% 2.902m ±1% ~ p=0.161 n=15
allocs/op 4.204k ±0% 4.203k ±0% ~ p=0.645 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/581d02c/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/581d02cfb4d789bce0558e29ae7962e6265d6db1/bin/pkg_sql_tests benchdiff/581d02c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/581d02c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=c2e8221 --new=581d02c --memprofile ./pkg/sql/tests
Artifacts

download:

mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/581d02cfb4d789bce0558e29ae7962e6265d6db1/23861267661-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c2e82219150d1858a5542db18805ce06c43bf317/23861267661-1/\* old/

built with commit: 581d02cfb4d789bce0558e29ae7962e6265d6db1

Copy link
Copy Markdown
Collaborator

@pav-kv pav-kv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass, didn't look at tests yet. Looks great.

Comment thread pkg/kv/kvserver/kvstorage/wag/replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag/replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag/replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag/replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go
Comment thread pkg/kv/kvserver/kvstorage/wag/replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag/replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go
Comment thread pkg/kv/kvserver/kvstorage/wag/replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go
@pav-kv
Copy link
Copy Markdown
Collaborator

pav-kv commented Apr 1, 2026

CC @iskettaneh to get familiar with the WAG encoding etc. The truncation logic will have some overlaps.

@cockroach-teamcity
Copy link
Copy Markdown
Member

⚪ Sysbench [SQL, 3node, oltp_read_write]
Metric Old Commit New Commit Delta Note
sec/op 11.01m ±5% 10.94m ±5% ~ p=0.367 n=15
allocs/op 8.147k ±3% 8.129k ±2% ~ p=0.512 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/9450917/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/94509177f462ac9b4b8083fc43a3433172629b89/bin/pkg_sql_tests benchdiff/9450917/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/9450917/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=c2e8221 --new=9450917 --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_read_only]
Metric Old Commit New Commit Delta Note
sec/op 3.296m ±2% 3.332m ±2% ~ p=0.161 n=15
allocs/op 2.107k ±0% 2.107k ±0% ~ p=0.063 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/9450917/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/94509177f462ac9b4b8083fc43a3433172629b89/bin/pkg_sql_tests benchdiff/9450917/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/9450917/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=c2e8221 --new=9450917 --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_write_only]
Metric Old Commit New Commit Delta Note
sec/op 3.212m ±3% 3.176m ±3% ~ p=0.137 n=15
allocs/op 4.225k ±0% 4.226k ±0% ~ p=0.629 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/9450917/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/94509177f462ac9b4b8083fc43a3433172629b89/bin/pkg_sql_tests benchdiff/9450917/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/9450917/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=c2e8221 --new=9450917 --memprofile ./pkg/sql/tests
Artifacts

download:

mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/94509177f462ac9b4b8083fc43a3433172629b89/23902650591-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c2e82219150d1858a5542db18805ce06c43bf317/23902650591-1/\* old/

built with commit: 94509177f462ac9b4b8083fc43a3433172629b89

Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go
Comment thread pkg/kv/kvserver/kvstorage/wag/replay_test.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag/replay_test.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag/replay_test.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag/replay_test.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag/replay_test.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag/replay_test.go Outdated
@miraradeva miraradeva changed the title kvstorage/wag: add WAG replay logic kvstorage: add WAG replay logic Apr 3, 2026
@cockroach-teamcity
Copy link
Copy Markdown
Member

⚪ Sysbench [SQL, 3node, oltp_read_write]
Metric Old Commit New Commit Delta Note
sec/op 10.00m ±1% 10.02m ±1% ~ p=0.345 n=15
allocs/op 8.079k ±0% 8.125k ±1% ~ p=0.044 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/c668ce6/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c668ce690c03ff0d77dfd608c8e74d3183dea630/bin/pkg_sql_tests benchdiff/c668ce6/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c668ce6/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=c2e8221 --new=c668ce6 --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_read_only]
Metric Old Commit New Commit Delta Note
sec/op 3.064m ±1% 3.074m ±1% ~ p=0.367 n=15
allocs/op 2.106k ±0% 2.106k ±1% ~ p=0.673 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/c668ce6/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c668ce690c03ff0d77dfd608c8e74d3183dea630/bin/pkg_sql_tests benchdiff/c668ce6/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c668ce6/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=c2e8221 --new=c668ce6 --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_write_only]
Metric Old Commit New Commit Delta Note
sec/op 2.891m ±1% 2.892m ±0% ~ p=0.250 n=15
allocs/op 4.201k ±0% 4.204k ±0% ~ p=0.132 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/c668ce6/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c668ce690c03ff0d77dfd608c8e74d3183dea630/bin/pkg_sql_tests benchdiff/c668ce6/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c668ce6/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=c2e8221 --new=c668ce6 --memprofile ./pkg/sql/tests
Artifacts

download:

mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c668ce690c03ff0d77dfd608c8e74d3183dea630/23951375097-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c2e82219150d1858a5542db18805ce06c43bf317/23951375097-1/\* old/

built with commit: c668ce690c03ff0d77dfd608c8e74d3183dea630

@cockroach-teamcity
Copy link
Copy Markdown
Member

⚪ Sysbench [SQL, 3node, oltp_read_write]
Metric Old Commit New Commit Delta Note
sec/op 10.57m ±2% 10.56m ±1% ~ p=0.683 n=15
allocs/op 8.118k ±1% 8.113k ±1% ~ p=0.814 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/a0cde21/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/a0cde2154e7ee8a0924b228aaace8ddaf88bfa54/bin/pkg_sql_tests benchdiff/a0cde21/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/a0cde21/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=c2e8221 --new=a0cde21 --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_read_only]
Metric Old Commit New Commit Delta Note
sec/op 3.099m ±1% 3.103m ±1% ~ p=0.902 n=15
allocs/op 2.106k ±0% 2.107k ±0% ~ p=0.466 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/a0cde21/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/a0cde2154e7ee8a0924b228aaace8ddaf88bfa54/bin/pkg_sql_tests benchdiff/a0cde21/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/a0cde21/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=c2e8221 --new=a0cde21 --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_write_only]
Metric Old Commit New Commit Delta Note
sec/op 2.925m ±2% 2.923m ±2% ~ p=0.461 n=15
allocs/op 4.207k ±0% 4.207k ±0% ~ p=0.798 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/a0cde21/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/a0cde2154e7ee8a0924b228aaace8ddaf88bfa54/bin/pkg_sql_tests benchdiff/a0cde21/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/a0cde21/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=c2e8221 --new=a0cde21 --memprofile ./pkg/sql/tests
Artifacts

download:

mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/a0cde2154e7ee8a0924b228aaace8ddaf88bfa54/24035030765-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c2e82219150d1858a5542db18805ce06c43bf317/24035030765-1/\* old/

built with commit: a0cde2154e7ee8a0924b228aaace8ddaf88bfa54

Copy link
Copy Markdown
Collaborator

@pav-kv pav-kv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM % minor suggestions.

Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go
Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay_test.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay_test.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay.go Outdated
Comment thread pkg/kv/kvserver/kvstorage/wag_replay_test.go Outdated
Add replay infrastructure for applying WAG (Write-Ahead Graph) nodes to
the state machine during store startup. The replay loop iterates over
the WAG in the log engine and applies any unapplied mutations.

The core decision logic is split into layers:

- canApply: per-event decision based on the event's address vs. the
  range's persistedRangeState (replica ID, tombstone, applied index).

- raftCatchUp: determines the raft index a replica must be caught up to
  before a WAG node can be applied. Varies by event type (e.g. splits
  catch up to Index-1, destroys to Index, creates need no catch-up).

- canApplyWAGNode: node-level wrapper that checks all events agree on
  whether to apply (returns error if not) and collects per-range raft
  catch-up targets.

- ReplayWAG: iterates the WAG log and applies unapplied mutations.

Resolves: cockroachdb#167278
Epic: CRDB-55218
Release note: None

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@cockroach-teamcity
Copy link
Copy Markdown
Member

⚪ Sysbench [SQL, 3node, oltp_read_write]
Metric Old Commit New Commit Delta Note
sec/op 10.08m ±2% 10.28m ±3% ~ p=0.056 n=15
allocs/op 8.083k ±1% 8.101k ±1% ~ p=0.290 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/b27ce04/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/b27ce04e9261df6408d0083d2f71014c81e05a18/bin/pkg_sql_tests benchdiff/b27ce04/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/b27ce04/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=c2e8221 --new=b27ce04 --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_read_only]
Metric Old Commit New Commit Delta Note
sec/op 3.187m ±2% 3.190m ±2% ~ p=0.935 n=15
allocs/op 2.106k ±0% 2.106k ±0% ~ p=0.460 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/b27ce04/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/b27ce04e9261df6408d0083d2f71014c81e05a18/bin/pkg_sql_tests benchdiff/b27ce04/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/b27ce04/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=c2e8221 --new=b27ce04 --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_write_only]
Metric Old Commit New Commit Delta Note
sec/op 3.037m ±1% 3.030m ±1% ~ p=0.713 n=15
allocs/op 4.216k ±0% 4.213k ±0% ~ p=0.062 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/b27ce04/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/b27ce04e9261df6408d0083d2f71014c81e05a18/bin/pkg_sql_tests benchdiff/b27ce04/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/b27ce04/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=c2e8221 --new=b27ce04 --memprofile ./pkg/sql/tests
Artifacts

download:

mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/b27ce04e9261df6408d0083d2f71014c81e05a18/24147510321-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c2e82219150d1858a5542db18805ce06c43bf317/24147510321-1/\* old/

built with commit: b27ce04e9261df6408d0083d2f71014c81e05a18

@cockroach-teamcity
Copy link
Copy Markdown
Member

⚪ Sysbench [SQL, 3node, oltp_read_write]
Metric Old Commit New Commit Delta Note
sec/op 10.24m ±1% 10.22m ±1% ~ p=0.713 n=15
allocs/op 8.087k ±1% 8.107k ±0% ~ p=0.943 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/7696e69/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/7696e690fb5f9167e6cfbf2984f38dd864d4ec53/bin/pkg_sql_tests benchdiff/7696e69/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/7696e69/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/8922fce/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/8922fce6a04bb5a8433d7c1daf3ff4770b8739b1/bin/pkg_sql_tests benchdiff/8922fce/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/8922fce/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=8922fce --new=7696e69 --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_read_only]
Metric Old Commit New Commit Delta Note
sec/op 3.149m ±2% 3.125m ±2% ~ p=0.161 n=15
allocs/op 2.106k ±0% 2.106k ±0% ~ p=0.154 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/7696e69/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/7696e690fb5f9167e6cfbf2984f38dd864d4ec53/bin/pkg_sql_tests benchdiff/7696e69/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/7696e69/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/8922fce/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/8922fce6a04bb5a8433d7c1daf3ff4770b8739b1/bin/pkg_sql_tests benchdiff/8922fce/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/8922fce/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=8922fce --new=7696e69 --memprofile ./pkg/sql/tests
⚪ Sysbench [KV, 3node, oltp_write_only]
Metric Old Commit New Commit Delta Note
sec/op 2.949m ±0% 2.940m ±1% ~ p=0.305 n=15
allocs/op 4.208k ±0% 4.207k ±0% ~ p=0.307 n=15
Reproduce

benchdiff binaries:

mkdir -p benchdiff/7696e69/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/7696e690fb5f9167e6cfbf2984f38dd864d4ec53/bin/pkg_sql_tests benchdiff/7696e69/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/7696e69/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/8922fce/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/8922fce6a04bb5a8433d7c1daf3ff4770b8739b1/bin/pkg_sql_tests benchdiff/8922fce/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/8922fce/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

# NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=8922fce --new=7696e69 --memprofile ./pkg/sql/tests
Artifacts

download:

mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/7696e690fb5f9167e6cfbf2984f38dd864d4ec53/24154608575-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/8922fce6a04bb5a8433d7c1daf3ff4770b8739b1/24154608575-1/\* old/

built with commit: 7696e690fb5f9167e6cfbf2984f38dd864d4ec53

@trunk-io trunk-io Bot merged commit 8de24a1 into cockroachdb:master Apr 8, 2026
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

target-release-26.3.0 X-perf-check Microbenchmarks CI: Added to a PR if a performance regression is detected and should be checked

Projects

None yet

Development

Successfully merging this pull request may close these issues.

kvstorage/wag: "is applied" check and replay loop

3 participants