New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

changefeedccl: add metrics for performance debugging #32241

Merged
merged 1 commit into from Nov 28, 2018

Conversation

Projects
None yet
3 participants
@danhhz
Copy link
Contributor

danhhz commented Nov 13, 2018

I've been running a branch with a bunch of extra metrics while debugging
issue #32104. PR'ing the ones that have proven useful in case someone
has performance issues in the wild.

Release note (enterprise change): Added timeseries metrics for debugging
CHANGEFEED performance issues.

@danhhz danhhz requested a review from mrtracy Nov 13, 2018

@danhhz danhhz requested review from cockroachdb/core-prs as code owners Nov 13, 2018

@cockroach-teamcity

This comment has been minimized.

Copy link
Member

cockroach-teamcity commented Nov 13, 2018

This change is Reviewable

@mrtracy
Copy link
Collaborator

mrtracy left a comment

:lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained

@mrtracy
Copy link
Collaborator

mrtracy left a comment

:lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained

@danhhz danhhz force-pushed the danhhz:cdc_metrics branch from 5fffff6 to 0b30c96 Nov 13, 2018

changefeedccl: add metrics for performance debugging
I've been running a branch with a bunch of extra metrics while debugging
issue #32104. PR'ing the ones that have proven useful in case someone
has performance issues in the wild.

Release note (enterprise change): Added timeseries metrics for debugging
CHANGEFEED performance issues.

@danhhz danhhz force-pushed the danhhz:cdc_metrics branch from 0b30c96 to abf2f4d Nov 20, 2018

@danhhz

This comment has been minimized.

Copy link
Contributor

danhhz commented Nov 28, 2018

TFTR!

bors r=mrtracy

craig bot pushed a commit that referenced this pull request Nov 28, 2018

Merge #32241
32241: changefeedccl: add metrics for performance debugging r=mrtracy a=danhhz

I've been running a branch with a bunch of extra metrics while debugging
issue #32104. PR'ing the ones that have proven useful in case someone
has performance issues in the wild.

Release note (enterprise change): Added timeseries metrics for debugging
CHANGEFEED performance issues.

Co-authored-by: Daniel Harrison <daniel.harrison@gmail.com>
@craig

This comment has been minimized.

Copy link

craig bot commented Nov 28, 2018

Build succeeded

@craig craig bot merged commit abf2f4d into cockroachdb:master Nov 28, 2018

3 checks passed

GitHub CI (Cockroach) TeamCity build finished
Details
bors Build succeeded
Details
license/cla Contributor License Agreement is signed.
Details

@danhhz danhhz deleted the danhhz:cdc_metrics branch Nov 28, 2018

danhhz added a commit to danhhz/cockroach that referenced this pull request Nov 29, 2018

changefeedccl: add logs for slow ExportRequests and frontier spans
The metrics added in cockroachdb#32241 are great for monitoring the health of a
changefeed and roughly debugging performance issues. Many of the ones
I've been seeing in testing have been one or many spans being behind, so
add some debug logging with details on the keys and lag, which then
allows for more targeted investigation.

Release note: None

craig bot pushed a commit that referenced this pull request Dec 5, 2018

Merge #32728
32728: changefeedccl: add logs for slow ExportRequests and frontier spans r=mrtracy a=danhhz

The metrics added in #32241 are great for monitoring the health of a
changefeed and roughly debugging performance issues. Many of the ones
I've been seeing in testing have been one or many spans being behind, so
add some debug logging with details on the keys and lag, which then
allows for more targeted investigation.

Release note: None

Co-authored-by: Daniel Harrison <daniel.harrison@gmail.com>

danhhz added a commit to danhhz/cockroach that referenced this pull request Dec 5, 2018

changefeedccl: add logs for slow ExportRequests and frontier spans
The metrics added in cockroachdb#32241 are great for monitoring the health of a
changefeed and roughly debugging performance issues. Many of the ones
I've been seeing in testing have been one or many spans being behind, so
add some debug logging with details on the keys and lag, which then
allows for more targeted investigation.

Release note: None

danhhz added a commit to danhhz/cockroach that referenced this pull request Dec 5, 2018

changefeedccl: add logs for slow ExportRequests and frontier spans
The metrics added in cockroachdb#32241 are great for monitoring the health of a
changefeed and roughly debugging performance issues. Many of the ones
I've been seeing in testing have been one or many spans being behind, so
add some debug logging with details on the keys and lag, which then
allows for more targeted investigation.

Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment