kvstorage: add WAG replay logic#167029
Conversation
|
😎 Merged successfully - details. |
6e39d71 to
b4c5921
Compare
⚪ Sysbench [SQL, 3node, oltp_read_write]
Reproducebenchdiff binaries: mkdir -p benchdiff/b4c5921/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/b4c5921fb83adf189e9bc2d62e77ae56d90f838e/bin/pkg_sql_tests benchdiff/b4c5921/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/b4c5921/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/6354858/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/6354858a402243510fa3abbbdc2a3c1a0a7c659e/bin/pkg_sql_tests benchdiff/6354858/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/6354858/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=6354858 --new=b4c5921 --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_read_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/b4c5921/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/b4c5921fb83adf189e9bc2d62e77ae56d90f838e/bin/pkg_sql_tests benchdiff/b4c5921/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/b4c5921/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/6354858/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/6354858a402243510fa3abbbdc2a3c1a0a7c659e/bin/pkg_sql_tests benchdiff/6354858/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/6354858/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=6354858 --new=b4c5921 --memprofile ./pkg/sql/tests🔴 Sysbench [KV, 3node, oltp_write_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/b4c5921/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/b4c5921fb83adf189e9bc2d62e77ae56d90f838e/bin/pkg_sql_tests benchdiff/b4c5921/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/b4c5921/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/6354858/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/6354858a402243510fa3abbbdc2a3c1a0a7c659e/bin/pkg_sql_tests benchdiff/6354858/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/6354858/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=6354858 --new=b4c5921 --memprofile ./pkg/sql/testsArtifactsdownload: mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/b4c5921fb83adf189e9bc2d62e77ae56d90f838e/23854081926-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/6354858a402243510fa3abbbdc2a3c1a0a7c659e/23854081926-1/\* old/built with commit: b4c5921fb83adf189e9bc2d62e77ae56d90f838e |
15c03d1 to
2b66fae
Compare
⚪ Sysbench [SQL, 3node, oltp_read_write]
Reproducebenchdiff binaries: mkdir -p benchdiff/2b66fae/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/2b66faeadd990d9d93658be62b6cd467f1156c4a/bin/pkg_sql_tests benchdiff/2b66fae/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/2b66fae/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=c2e8221 --new=2b66fae --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_read_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/2b66fae/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/2b66faeadd990d9d93658be62b6cd467f1156c4a/bin/pkg_sql_tests benchdiff/2b66fae/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/2b66fae/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=c2e8221 --new=2b66fae --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_write_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/2b66fae/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/2b66faeadd990d9d93658be62b6cd467f1156c4a/bin/pkg_sql_tests benchdiff/2b66fae/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/2b66fae/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=c2e8221 --new=2b66fae --memprofile ./pkg/sql/testsArtifactsdownload: mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/2b66faeadd990d9d93658be62b6cd467f1156c4a/23859203948-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c2e82219150d1858a5542db18805ce06c43bf317/23859203948-1/\* old/built with commit: 2b66faeadd990d9d93658be62b6cd467f1156c4a |
2b66fae to
581d02c
Compare
⚪ Sysbench [SQL, 3node, oltp_read_write]
Reproducebenchdiff binaries: mkdir -p benchdiff/581d02c/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/581d02cfb4d789bce0558e29ae7962e6265d6db1/bin/pkg_sql_tests benchdiff/581d02c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/581d02c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=c2e8221 --new=581d02c --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_read_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/581d02c/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/581d02cfb4d789bce0558e29ae7962e6265d6db1/bin/pkg_sql_tests benchdiff/581d02c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/581d02c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=c2e8221 --new=581d02c --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_write_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/581d02c/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/581d02cfb4d789bce0558e29ae7962e6265d6db1/bin/pkg_sql_tests benchdiff/581d02c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/581d02c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=c2e8221 --new=581d02c --memprofile ./pkg/sql/testsArtifactsdownload: mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/581d02cfb4d789bce0558e29ae7962e6265d6db1/23861267661-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c2e82219150d1858a5542db18805ce06c43bf317/23861267661-1/\* old/built with commit: 581d02cfb4d789bce0558e29ae7962e6265d6db1 |
pav-kv
left a comment
There was a problem hiding this comment.
First pass, didn't look at tests yet. Looks great.
|
CC @iskettaneh to get familiar with the WAG encoding etc. The truncation logic will have some overlaps. |
581d02c to
9450917
Compare
⚪ Sysbench [SQL, 3node, oltp_read_write]
Reproducebenchdiff binaries: mkdir -p benchdiff/9450917/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/94509177f462ac9b4b8083fc43a3433172629b89/bin/pkg_sql_tests benchdiff/9450917/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/9450917/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=c2e8221 --new=9450917 --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_read_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/9450917/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/94509177f462ac9b4b8083fc43a3433172629b89/bin/pkg_sql_tests benchdiff/9450917/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/9450917/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=c2e8221 --new=9450917 --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_write_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/9450917/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/94509177f462ac9b4b8083fc43a3433172629b89/bin/pkg_sql_tests benchdiff/9450917/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/9450917/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=c2e8221 --new=9450917 --memprofile ./pkg/sql/testsArtifactsdownload: mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/94509177f462ac9b4b8083fc43a3433172629b89/23902650591-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c2e82219150d1858a5542db18805ce06c43bf317/23902650591-1/\* old/built with commit: 94509177f462ac9b4b8083fc43a3433172629b89 |
9450917 to
a976658
Compare
a976658 to
c668ce6
Compare
⚪ Sysbench [SQL, 3node, oltp_read_write]
Reproducebenchdiff binaries: mkdir -p benchdiff/c668ce6/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c668ce690c03ff0d77dfd608c8e74d3183dea630/bin/pkg_sql_tests benchdiff/c668ce6/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c668ce6/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=c2e8221 --new=c668ce6 --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_read_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/c668ce6/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c668ce690c03ff0d77dfd608c8e74d3183dea630/bin/pkg_sql_tests benchdiff/c668ce6/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c668ce6/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=c2e8221 --new=c668ce6 --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_write_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/c668ce6/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c668ce690c03ff0d77dfd608c8e74d3183dea630/bin/pkg_sql_tests benchdiff/c668ce6/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c668ce6/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=c2e8221 --new=c668ce6 --memprofile ./pkg/sql/testsArtifactsdownload: mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c668ce690c03ff0d77dfd608c8e74d3183dea630/23951375097-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c2e82219150d1858a5542db18805ce06c43bf317/23951375097-1/\* old/built with commit: c668ce690c03ff0d77dfd608c8e74d3183dea630 |
c668ce6 to
a0cde21
Compare
⚪ Sysbench [SQL, 3node, oltp_read_write]
Reproducebenchdiff binaries: mkdir -p benchdiff/a0cde21/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/a0cde2154e7ee8a0924b228aaace8ddaf88bfa54/bin/pkg_sql_tests benchdiff/a0cde21/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/a0cde21/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=c2e8221 --new=a0cde21 --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_read_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/a0cde21/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/a0cde2154e7ee8a0924b228aaace8ddaf88bfa54/bin/pkg_sql_tests benchdiff/a0cde21/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/a0cde21/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=c2e8221 --new=a0cde21 --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_write_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/a0cde21/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/a0cde2154e7ee8a0924b228aaace8ddaf88bfa54/bin/pkg_sql_tests benchdiff/a0cde21/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/a0cde21/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=c2e8221 --new=a0cde21 --memprofile ./pkg/sql/testsArtifactsdownload: mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/a0cde2154e7ee8a0924b228aaace8ddaf88bfa54/24035030765-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c2e82219150d1858a5542db18805ce06c43bf317/24035030765-1/\* old/built with commit: a0cde2154e7ee8a0924b228aaace8ddaf88bfa54 |
pav-kv
left a comment
There was a problem hiding this comment.
LGTM % minor suggestions.
a0cde21 to
b27ce04
Compare
Add replay infrastructure for applying WAG (Write-Ahead Graph) nodes to the state machine during store startup. The replay loop iterates over the WAG in the log engine and applies any unapplied mutations. The core decision logic is split into layers: - canApply: per-event decision based on the event's address vs. the range's persistedRangeState (replica ID, tombstone, applied index). - raftCatchUp: determines the raft index a replica must be caught up to before a WAG node can be applied. Varies by event type (e.g. splits catch up to Index-1, destroys to Index, creates need no catch-up). - canApplyWAGNode: node-level wrapper that checks all events agree on whether to apply (returns error if not) and collects per-range raft catch-up targets. - ReplayWAG: iterates the WAG log and applies unapplied mutations. Resolves: cockroachdb#167278 Epic: CRDB-55218 Release note: None Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
⚪ Sysbench [SQL, 3node, oltp_read_write]
Reproducebenchdiff binaries: mkdir -p benchdiff/b27ce04/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/b27ce04e9261df6408d0083d2f71014c81e05a18/bin/pkg_sql_tests benchdiff/b27ce04/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/b27ce04/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=c2e8221 --new=b27ce04 --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_read_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/b27ce04/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/b27ce04e9261df6408d0083d2f71014c81e05a18/bin/pkg_sql_tests benchdiff/b27ce04/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/b27ce04/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=c2e8221 --new=b27ce04 --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_write_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/b27ce04/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/b27ce04e9261df6408d0083d2f71014c81e05a18/bin/pkg_sql_tests benchdiff/b27ce04/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/b27ce04/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/c2e8221/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/c2e82219150d1858a5542db18805ce06c43bf317/bin/pkg_sql_tests benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/c2e8221/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=c2e8221 --new=b27ce04 --memprofile ./pkg/sql/testsArtifactsdownload: mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/b27ce04e9261df6408d0083d2f71014c81e05a18/24147510321-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/c2e82219150d1858a5542db18805ce06c43bf317/24147510321-1/\* old/built with commit: b27ce04e9261df6408d0083d2f71014c81e05a18 |
b27ce04 to
7696e69
Compare
⚪ Sysbench [SQL, 3node, oltp_read_write]
Reproducebenchdiff binaries: mkdir -p benchdiff/7696e69/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/7696e690fb5f9167e6cfbf2984f38dd864d4ec53/bin/pkg_sql_tests benchdiff/7696e69/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/7696e69/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/8922fce/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/8922fce6a04bb5a8433d7c1daf3ff4770b8739b1/bin/pkg_sql_tests benchdiff/8922fce/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/8922fce/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=8922fce --new=7696e69 --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_read_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/7696e69/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/7696e690fb5f9167e6cfbf2984f38dd864d4ec53/bin/pkg_sql_tests benchdiff/7696e69/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/7696e69/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/8922fce/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/8922fce6a04bb5a8433d7c1daf3ff4770b8739b1/bin/pkg_sql_tests benchdiff/8922fce/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/8922fce/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_read_only$ --old=8922fce --new=7696e69 --memprofile ./pkg/sql/tests⚪ Sysbench [KV, 3node, oltp_write_only]
Reproducebenchdiff binaries: mkdir -p benchdiff/7696e69/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/7696e690fb5f9167e6cfbf2984f38dd864d4ec53/bin/pkg_sql_tests benchdiff/7696e69/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/7696e69/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/8922fce/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/8922fce6a04bb5a8433d7c1daf3ff4770b8739b1/bin/pkg_sql_tests benchdiff/8922fce/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/8922fce/bin/1058449141/cockroachdb_cockroach_pkg_sql_testsbenchdiff command: # NB: for best (most stable) results, also add a suitable `--benchtime` that
# results in ~1s to ~5s of benchmark runs. For example, if ops average ~3ms, a
# benchtime of `1000x` is appropriate.
#
# Some benchmarks (in particular BenchmarkSysbench) output additional memory
# profiles covering only the execution (excluding the setup/teardown) - those
# should be preferred for analysis since they more closely correspond to what's
# reported as B/op and alloc/op.
benchdiff --run=^BenchmarkSysbench/KV/3node/oltp_write_only$ --old=8922fce --new=7696e69 --memprofile ./pkg/sql/testsArtifactsdownload: mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/7696e690fb5f9167e6cfbf2984f38dd864d4ec53/24154608575-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/8922fce6a04bb5a8433d7c1daf3ff4770b8739b1/24154608575-1/\* old/built with commit: 7696e690fb5f9167e6cfbf2984f38dd864d4ec53 |
Add replay infrastructure for applying WAG (Write-Ahead Graph) nodes to
the state machine during store startup. The replay loop iterates over
the WAG in the log engine and applies any unapplied mutations.
The core decision logic is split into layers:
canApply: per-event decision based on the event's address vs. the
range's persistedRangeState (replica ID, tombstone, applied index).
raftCatchUp: determines the raft index a replica must be caught up to
before a WAG node can be applied. Varies by event type (e.g. splits
catch up to Index-1, destroys to Index, creates need no catch-up).
canApplyWAGNode: node-level wrapper that checks all events agree on
whether to apply (returns error if not) and collects per-range raft
catch-up targets.
ReplayWAG: iterates the WAG log and applies unapplied mutations.
Resolves: #167278
Epic: CRDB-55218
Release note: None