changefeedccl: ux fixes and tighter tracked spans #29825
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I thought the SpanResolverIterator was doing span trimming, etc, but it
turns out this lives in level up in `PartitionSpans`. Reuse it.
This was causing performance issues in tests when we grabbed a pre-split
range and didn't trim it, so too much of the keyspace was being watched.
Very occasionally, it would also cause flakes if we ended up watching
part of the timeseries data (which ExportRequest doesn't support).
All `sql` package changes generated mechanically with:
$ gorename -from '"./pkg/sql".DistSQLPlanner.partitionSpans' -to PartitionSpans
$ gorename -from '"./pkg/sql".spanPartition' -to SpanPartition
$ gorename -from '"./pkg/sql".SpanPartition.spans' -to Spans
$ gorename -from '"./pkg/sql".SpanPartition.node' -to Node
Plus a couple updates to the new name in comments, found via git grep.
Release note (bug fix): avoids a very rare premature failure in
CHANGEFEEDs caused by a race condition with range splits
These came from development partner feedback. The following options metioned in the rfc now throw a specific "<option> not yet implemented" error: `format=avro`, `envelope=diff`, `confluent_schema_registry`, `schema_topic`. Unrecognized sink parameters now throw an error instead of being silently ignored. Release note: None
It snuck in. Release note: None
|
This change is |
Adding stuff to the buffer can take longer than it did to fetch if the sink is being slow. Definitely worth timing and logging on the nightly roachtests. Release note: None
There's a bit of variation in how long each poll takes (tends to be between 1m and 4m). I'd like to see what happens if we run the test longer. Also sneak in a couple fixes while I'm here. Increase vlog to catch resolved timestamp updates. Make starting kafka more idempotent (for dev iteration). Make kafka installation quieter. Release note: None
mrtracy
approved these changes
Sep 10, 2018
There was a problem hiding this comment.
These are all
Reviewable status:
complete! 0 of 0 LGTMs obtained (and 1 stale)
|
Thanks for the review! bors r=mrtracy |
craig bot
pushed a commit
that referenced
this pull request
Sep 10, 2018
29818: sql: array_to_json builtin r=asubiotto,jordanlewis a=changangela Relates to #20120. 29825: changefeedccl: ux fixes and tighter tracked spans r=mrtracy a=danhhz see commits for details 30004: rangefeed: bump TestProcessorSlowConsumer event timeout r=nvanbenschoten a=nvanbenschoten Fixes #29671. Before this, the test would fail in about 3 minutes of stress. After this change, it didn't fail in over 30 minutes. Will need to be backported. Release note: None Co-authored-by: changangela <angela@cockroachlabs.com> Co-authored-by: Daniel Harrison <daniel.harrison@gmail.com> Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
Build succeeded |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
see commits for details