Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: add infoschema client errors #22382

Merged
merged 15 commits into from
Mar 11, 2021
Merged

Conversation

morgo
Copy link
Contributor

@morgo morgo commented Jan 13, 2021

What problem does this PR solve?

Issue Number: close #14433

Revives PR: #20785

Problem Summary:

In the PR #22351 , it is proposed that multiStmt be permitted as a warning, and later changed to a default. This provides an upgrade path for users.

The problem is, errors sent to the client were previously not captured by the server. So it is difficult to tell if a user is depending on the buggy behavior of multiStmt, and if the defaults change will cause them problems.

Thus, the proposal is to cherry-pick to 4.0 and 5.0 to provide a viable way to get past the multiStmt vulnerability.

What is changed and how it works?

What's Changed:

This PR introduces a method to observe errors and warnings sent to clients. For example, using multiStmt as an example:

mysql> select * from CLIENT_ERRORS_SUMMARY_by_user;
+------+--------------+---------------------------------------------------------------------------------------------------------------------------------------+-------------+---------------+---------------------+---------------------+
| USER | ERROR_NUMBER | ERROR_MESSAGE                                                                                                                         | ERROR_COUNT | WARNING_COUNT | FIRST_SEEN          | LAST_SEEN           |
+------+--------------+---------------------------------------------------------------------------------------------------------------------------------------+-------------+---------------+---------------------+---------------------+
| root |         1054 | Unknown column '%-.192s' in '%-.192s'                                                                                                 |           4 |             0 | 2021-01-13 13:14:06 | 2021-01-13 13:14:06 |
| root |         1105 | Unknown error                                                                                                                         |           1 |             0 | 2021-01-13 13:14:25 | 2021-01-13 13:14:25 |
| root |         1146 | Table '%-.192s.%-.192s' doesn't exist                                                                                                 |          12 |             0 | 2021-01-13 13:06:29 | 2021-01-13 13:12:56 |
| root |         8130 | client has multi-statement capability disabled. Run SET GLOBAL tidb_allow_multi_statement='ON' after you understand the security risk |           1 |             1 | 2021-01-13 13:06:53 | 2021-01-13 13:14:25 |
| root |         1365 | Division by 0                                                                                                                         |           0 |             1 | 2021-01-13 13:07:18 | 2021-01-13 13:07:18 |
+------+--------------+---------------------------------------------------------------------------------------------------------------------------------------+-------------+---------------+---------------------+---------------------+
5 rows in set (0.00 sec)

(The warning is the default, when set to OFF, it generated an error).

In total, three new information schema tables have been introduced:

  • client_errors_summary_global
  • client_errors_summary_by_host
  • client_errors_summary_by_user

The design is modeled loosely based on what MySQL 8.0 has in performance_schema. But there are the following differences to be aware of:

  • In MySQL 8.0, reseting the stats is done with a TRUNCATE TABLE command. But since these are in infoschema in TiDB, a command FLUSH CLIENT_ERRORS_SUMMARY is added.
  • In MySQL there is always a row for each error code (regardless if any errors or warnings have been generated). I thought could be misleading, since if it showed all errors in the errno package - some are known not to be generated. Also, it creates a very large table if there are a lot of users or hosts.
  • In TiDB it shows the ERROR_MESSAGE (in sprintf format), not the ERROR_NAME. This is a current limitation based on what is stored in the errno` package. I think it's fine.
  • There is no ERROR_RAISED / ERROR_HANDLED counts, and the columns are just renamed to ERROR_COUNT. TiDB does not have stored procs, and thus no error handling.

How it Works:

The errors and warnings could be generated anywhere in code. I capture them not at generate time, but as they are sent to the client. This means that internal sql execution that triggers warnings etc. is not logged.

There is no persistence of the statistics, and no cluster-wide view.

Related changes

Check List

Tests

  • Unit test

Side effects

  • Should be minimal performance impact since it only needs to acquire the mutex briefly when there is a statement that has caused an error or warning. I checked with tiup tpcc bench to see how many client errors are generated, and it is only 1 during prepare.

Release note

  • A set of client_errors_summary tables has been added to Information Schema. This helps keep track of which errors have been sent to clients.

@morgo morgo requested a review from a team as a code owner January 13, 2021 20:42
@morgo morgo requested review from wshwsh12, djshow832, bb7133 and zz-jason and removed request for a team January 13, 2021 20:42
@github-actions github-actions bot added sig/execution SIG execution sig/sql-infra SIG: SQL Infra labels Jan 13, 2021
errno/infoschema.go Outdated Show resolved Hide resolved
executor/builder.go Outdated Show resolved Hide resolved
errno/infoschema.go Outdated Show resolved Hide resolved
errno/infoschema.go Outdated Show resolved Hide resolved
errno/infoschema.go Outdated Show resolved Hide resolved
morgo and others added 2 commits January 30, 2021 23:26
Co-authored-by: djshow832 <zhangming@pingcap.com>
Co-authored-by: djshow832 <zhangming@pingcap.com>
Copy link
Member

@bb7133 bb7133 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@bb7133
Copy link
Member

bb7133 commented Mar 11, 2021

/lgtm

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • bb7133

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by writing /lgtm in a comment.
Reviewer can cancel approval by writing /lgtm cancel in a comment.

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 11, 2021
@bb7133
Copy link
Member

bb7133 commented Mar 11, 2021

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: acf1f51

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 11, 2021
@ti-chi-bot ti-chi-bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Mar 11, 2021
@ti-chi-bot ti-chi-bot merged commit c4f3989 into pingcap:master Mar 11, 2021
ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Mar 11, 2021
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor

cherry pick to release-4.0 in PR #23267

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Mar 11, 2021
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor

cherry pick to release-5.0-rc in PR #23268

@bb7133
Copy link
Member

bb7133 commented Mar 11, 2021

/run-build-image

@SunRunAway
Copy link
Contributor

Do you have related documentation? @morgo

@morgo
Copy link
Contributor Author

morgo commented Mar 18, 2021

@SunRunAway I do now: pingcap/docs#5062 - can you please help review? :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/execution SIG execution sig/sql-infra SIG: SQL Infra size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Collect and expose client errors/warnings as a table
7 participants