release-2.1: changefeedccl: ux fixes and tighter tracked spans #30009
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I thought the SpanResolverIterator was doing span trimming, etc, but it
turns out this lives in level up in `PartitionSpans`. Reuse it.
This was causing performance issues in tests when we grabbed a pre-split
range and didn't trim it, so too much of the keyspace was being watched.
Very occasionally, it would also cause flakes if we ended up watching
part of the timeseries data (which ExportRequest doesn't support).
All `sql` package changes generated mechanically with:
$ gorename -from '"./pkg/sql".DistSQLPlanner.partitionSpans' -to PartitionSpans
$ gorename -from '"./pkg/sql".spanPartition' -to SpanPartition
$ gorename -from '"./pkg/sql".SpanPartition.spans' -to Spans
$ gorename -from '"./pkg/sql".SpanPartition.node' -to Node
Plus a couple updates to the new name in comments, found via git grep.
Release note (bug fix): avoids a very rare premature failure in
CHANGEFEEDs caused by a race condition with range splits
These came from development partner feedback. The following options metioned in the rfc now throw a specific "<option> not yet implemented" error: `format=avro`, `envelope=diff`, `confluent_schema_registry`, `schema_topic`. Unrecognized sink parameters now throw an error instead of being silently ignored. Release note: None
It snuck in. Release note: None
Adding stuff to the buffer can take longer than it did to fetch if the sink is being slow. Definitely worth timing and logging on the nightly roachtests. Release note: None
There's a bit of variation in how long each poll takes (tends to be between 1m and 4m). I'd like to see what happens if we run the test longer. Also sneak in a couple fixes while I'm here. Increase vlog to catch resolved timestamp updates. Make starting kafka more idempotent (for dev iteration). Make kafka installation quieter. Release note: None
|
This change is |
|
lgtm |
|
Thanks for the review! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport 5/5 commits from #29825.
/cc @cockroachdb/release
see commits for details