-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
changefeedccl: ux fixes and tighter tracked spans #29825
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I thought the SpanResolverIterator was doing span trimming, etc, but it turns out this lives in level up in `PartitionSpans`. Reuse it. This was causing performance issues in tests when we grabbed a pre-split range and didn't trim it, so too much of the keyspace was being watched. Very occasionally, it would also cause flakes if we ended up watching part of the timeseries data (which ExportRequest doesn't support). All `sql` package changes generated mechanically with: $ gorename -from '"./pkg/sql".DistSQLPlanner.partitionSpans' -to PartitionSpans $ gorename -from '"./pkg/sql".spanPartition' -to SpanPartition $ gorename -from '"./pkg/sql".SpanPartition.spans' -to Spans $ gorename -from '"./pkg/sql".SpanPartition.node' -to Node Plus a couple updates to the new name in comments, found via git grep. Release note (bug fix): avoids a very rare premature failure in CHANGEFEEDs caused by a race condition with range splits
These came from development partner feedback. The following options metioned in the rfc now throw a specific "<option> not yet implemented" error: `format=avro`, `envelope=diff`, `confluent_schema_registry`, `schema_topic`. Unrecognized sink parameters now throw an error instead of being silently ignored. Release note: None
It snuck in. Release note: None
Adding stuff to the buffer can take longer than it did to fetch if the sink is being slow. Definitely worth timing and logging on the nightly roachtests. Release note: None
There's a bit of variation in how long each poll takes (tends to be between 1m and 4m). I'd like to see what happens if we run the test longer. Also sneak in a couple fixes while I'm here. Increase vlog to catch resolved timestamp updates. Make starting kafka more idempotent (for dev iteration). Make kafka installation quieter. Release note: None
mrtracy
approved these changes
Sep 10, 2018
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status:
complete! 0 of 0 LGTMs obtained (and 1 stale)
Thanks for the review! bors r=mrtracy |
craig bot
pushed a commit
that referenced
this pull request
Sep 10, 2018
29818: sql: array_to_json builtin r=asubiotto,jordanlewis a=changangela Relates to #20120. 29825: changefeedccl: ux fixes and tighter tracked spans r=mrtracy a=danhhz see commits for details 30004: rangefeed: bump TestProcessorSlowConsumer event timeout r=nvanbenschoten a=nvanbenschoten Fixes #29671. Before this, the test would fail in about 3 minutes of stress. After this change, it didn't fail in over 30 minutes. Will need to be backported. Release note: None Co-authored-by: changangela <angela@cockroachlabs.com> Co-authored-by: Daniel Harrison <daniel.harrison@gmail.com> Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
Build succeeded |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
see commits for details