Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

log: generate digest and log it in slow log #9662

Merged
merged 6 commits into from Mar 14, 2019

Conversation

@lysu
Copy link
Member

commented Mar 12, 2019

What problem does this PR solve?

follow up to pingcap/parser#32, add digest to slow log

so we can awk or sql agg sql by digest to analysis query faster(even without pt-digest)

What is changed and how it works?

  • for each query generate digest and cache it in stmtCtx(later sql bind or audit can use it directly)
  • add slow log filed
  • add slow log table field
  • add admin show slow field(maybe we can do more use digest for show slow but maybe in following pr)

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

Code changes

  • Add stmt variable

Side effects

  • Possible performance regression(but lexer is fast)

Related changes

  • Need to cherry-pick to the release 2.1 when master work well

This change is Reviewable

@codecov

This comment has been minimized.

Copy link

commented Mar 12, 2019

Codecov Report

Merging #9662 into master will decrease coverage by 0.0118%.
The diff coverage is 76.4705%.

@@               Coverage Diff                @@
##             master      #9662        +/-   ##
================================================
- Coverage   67.4017%   67.3899%   -0.0119%     
================================================
  Files           377        377                
  Lines         79443      79457        +14     
================================================
  Hits          53546      53546                
- Misses        21117      21128        +11     
- Partials       4780       4783         +3
@jackysp

This comment has been minimized.

Copy link
Member

commented Mar 12, 2019

/run-all-tests

@jackysp
Copy link
Member

left a comment

LGTM

Show resolved Hide resolved session/session.go Outdated

lysu added some commits Mar 14, 2019

@lysu

This comment has been minimized.

Copy link
Member Author

commented Mar 14, 2019

/run-all-tests

@zz-jason
Copy link
Member

left a comment

LGTM

@zz-jason zz-jason added status/LGT2 and removed status/LGT1 labels Mar 14, 2019

@lysu

This comment has been minimized.

Copy link
Member Author

commented Mar 14, 2019

/run-all-tests

@lysu

This comment has been minimized.

Copy link
Member Author

commented Mar 14, 2019

/run-common-test tidb-test=pr/755
/run-integration-common-test tidb-test=pr/755

@lysu

This comment has been minimized.

Copy link
Member Author

commented Mar 14, 2019

/run-common-test tidb-test=pr/755
/run-integration-common-test tidb-test=pr/755

@lysu

This comment has been minimized.

Copy link
Member Author

commented Mar 14, 2019

/run-all-tests tidb-test=pr/755

@lysu lysu merged commit e398322 into pingcap:master Mar 14, 2019

16 checks passed

ci/circleci Your tests passed on CircleCI!
Details
codecov/patch 76.4705% of diff hit (target 0%)
Details
codecov/project Absolute coverage decreased by -0.0118% but relative coverage increased by +9.0688% compared to b76b2da
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
idc-jenkins-ci-tidb/build Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/check_dev Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/code_coverage Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/common-test job succeeded
Details
idc-jenkins-ci-tidb/integration-common-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/integration-compatibility-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/integration-ddl-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/mybatis-test job succeeded
Details
idc-jenkins-ci-tidb/sqllogic-test-1 Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/sqllogic-test-2 Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/unit-test Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details

@lysu lysu deleted the lysu:dev-digest-log branch Mar 14, 2019

lysu added a commit to lysu/tidb that referenced this pull request Apr 9, 2019

lysu added a commit to lysu/tidb that referenced this pull request Apr 11, 2019

ngaut added a commit that referenced this pull request Apr 11, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.