streamingest: allow replicating system tenants to other tenants

[dt]

Release note: none.
Epic: https://cockroachlabs.atlassian.net/browse/CRDB-33245

[cockroach-teamcity]

This change is 

[dt]

Now with a couple tests and RFAL.

[stevendanna]

LGTM but perhaps we should let @msbutler have a look as well since I already watched this code get written and might be a bit biased.

[msbutler]

Nice! I'm glad all the span config streaming stuff works out of the box! Were you able to verify that the a user connected to the system tenant cannot directly write to the zone config table?

Also, it may be helpful to write out in the commit message some of the subtle ways that streaming from the system tenant is different than streaming from a main tenant (e.g. we'll replicate a zone configs table that will never get used after cutover).

[msbutler]

could you add a docstring which explains why this is necessary?

Also, for a regular main tenant, do they have any non table data in their key space? I.e. if we replicate the key span keys.MakeTenantSpan(3), will that only ever contain table data?

[dt]

or a regular main tenant, do they have any non table data in their key space

No.

replicate the key span keys.MakeTenantSpan(3), will that only ever contain table data

Correct.

[dt]

the subtle ways that streaming from the system tenant is different... we'll replicate a zone configs table that will never get used after cutover

I think we overstated those differences in our chat; e.g. the zones table does get used, that is read by the reconciler and fed to span configs. After #122005 there really aren't many differences. It really is just that some jobs are only supposed to run in the system tenant so their owners may need to be aware of the fact they could end up in a non-system tenant and need to terminate themselves, a la the first commit here where we do that for our pcr job.

[dt]

TFTR!

bors r+

[craig]

Build failed:

• local_roachtest_fips

[dt]

bors r+

[craig]

Build succeeded:

• linux_arm64_build
• linux_amd64_build
• local_roachtest
• acceptance
• macos_amd64_build
• examples_orms
• local_roachtest_fips
• Bazel Essential CI (Cockroach)
• check_generated_code
• macos_arm64_build
• docker_image_amd64
• windows_build
• lint
• linux_amd64_fips_build

[blathers-crl]

Encountered an error creating backports. Some common things that can go wrong:

1. The backport branch might have already existed.
2. There was a merge conflict.
3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.

---

error setting reviewers, but backport branch blathers/backport-release-24.1-122001 is ready: POST https://api.github.com/repos/cockroachdb/cockroach/pulls/122290/requested_reviewers: 422 Reviews may only be requested from collaborators. One or more of the teams you specified is not a collaborator of the cockroachdb/cockroach repository. []

Backport to branch 24.1.x failed. See errors above.

---

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}