storage: change default engine type from RocksDB to Pebble

[petermattis]

This flips the switch to use Pebble by default. This will cause all tests
which don't explicitly specify RocksDB (i.e. the vast majority) to now use
Pebble. One exception is certain roachtests which create store directories
using older versions of CRDB and then upgrade to the current version. Those
tests will continue to use RocksDB (for now) due to the nature of how
--storage-engine=... is "sticky" if not specified, defaulting to the
previously used storage engine.

Release note (general change): Change the default engine type for new
storage directories from RocksDB to Pebble. Existing stores will continue
to use the previously specified storage engine, and an explicit
specification (via --storage-engine=...) will override the default.

[cockroach-teamcity]

This change is 

[petermattis]

[The first commit is #48144].

I'm not sure we want to merge this quite yet, but putting it out there for review and to see what CI thinks of it. I still have to do a full roachtest run on this PR to make sure that no roachtest regressions have slipped in.

[blathers-crl]

❌ The GitHub CI (Cockroach) build has failed on 147cb552.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

[blathers-crl]

❌ The GitHub CI (Cockroach) build has failed on 193ba988.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

[blathers-crl]

❌ The GitHub CI (Cockroach) build has failed on 3b97957d.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

[blathers-crl]

❌ The GitHub CI (Cockroach) build has failed on 58b1de3c.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

[itsbilal]

when all tests pass.

PR title and commit message should have engines in the opposite order, unless we're going back in time 😂

Reviewed 5 of 5 files at r1.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @jbowens)

[petermattis]

PR title and commit message should have engines in the opposite order, unless we're going back in time.

Ha! Oops.

[blathers-crl]

❌ The GitHub CI (Cockroach) build has failed on 1d05b844.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

[petermattis]

DockerCLI acceptance failures are real. I'll send a separate PR to fix them shortly.

[petermattis]

DockerCLI acceptance failures are real. I'll send a separate PR to fix them shortly.

Should be fixed now. I need to do another full roachtest-on-pebble run to make sure no failures have slipped in. Otherwise, this is good to go.

[blathers-crl]

❌ The GitHub CI (Cockroach) build has failed on cd190ae6.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

[petermattis]

Filed #48413 about the CI failure which looks unrelated to this PR.

[blathers-crl]

❌ The GitHub CI (Cockroach) build has failed on cd190ae6.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

[blathers-crl]

❌ The GitHub CI (Cockroach) build has failed on cd190ae6.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

[blathers-crl]

❌ The GitHub CI (Cockroach) build has failed on c0da218d.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

[blathers-crl]

❌ The GitHub CI (Cockroach) build has failed on 3d63dc1a.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

[itsbilal]

Is that a green checkmark I see?

Reviewed 4 of 4 files at r2.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @jbowens)

[petermattis]

Is that a green checkmark I see?

Almost time. I still need to do another roachtest-on-Pebble run.

[petermattis]

I still need to do another roachtest-on-Pebble run.

256 PASS, 12 FAIL. 1 of the failures is real ("synctest" is now skipped, see #48603). 3 of the failures were due to test infrastructure problems. The remaining 8 failures are flaky tests that periodically fail on RocksDB as well. I looked at all of the failures and nothing Pebble-specific jumped out about them.

I'm going to merge this when CI is green.

[petermattis]

TFTR! 🤞

bors r+

[craig]

Build failed (retrying...)

• GitHub CI (Cockroach)

[craig]

Build succeeded

• GitHub CI (Cockroach)