Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

store/helper: fill data in the information.tidb_hot_table for partitioned table #14331

Merged
merged 3 commits into from Jan 10, 2020

Conversation

@tiancaiamao
Copy link
Contributor

tiancaiamao commented Jan 3, 2020

What problem does this PR solve?

INFORMATION.TIDB_HOT_TABLE should take the partitioned table into consideration.

What is changed and how it works?

A partitioned table is not handled properly in tidb_hot_table.
After this change, we can get the correct information:

mysql> select * from tidb_hot_regions;
+----------+----------+---------+------------------+------------+-----------+-------+----------------+--------------+------------+
| TABLE_ID | INDEX_ID | DB_NAME | TABLE_NAME       | INDEX_NAME | REGION_ID | TYPE  | MAX_HOT_DEGREE | REGION_COUNT | FLOW_BYTES |
+----------+----------+---------+------------------+------------+-----------+-------+----------------+--------------+------------+
|       23 |     NULL | mysql   | stats_histograms | NULL       |        48 | read  |              3 |            0 |     211478 |
|       25 |     NULL | mysql   | stats_buckets    | NULL       |        52 | write |              3 |            0 |      93013 |
|      151 |     NULL | test    | sbtest3(p1)      | NULL       |      1156 | write |              3 |            0 |      93013 |
|      150 |     NULL | test    | sbtest3(p0)      | NULL       |      1152 | write |              3 |            0 |      93013 |
+----------+----------+---------+------------------+------------+-----------+-------+----------------+--------------+------------+
4 rows in set (0.08 sec)

Note, 151 and 150 are both belong to sbtest3, and the table name column is displayed as sbtest3(p1) and sbtest3(p0) to indicate they are partitions.

Check List

Tests

  • Manual test (add detailed scripts or steps below)

Mock the data is hard, the old code is not really unit tested.
I use a real tikv cluster and check the result.
B.T.W, a bug is fixed, pd returns something like this:

{
  "as_peer": {
    "1": {
      "total_flow_bytes": 190434.14973961416,
      "regions_count": 3,
      "statistics": [
        {
          "store_id": 1,
          "region_id": 1156,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 50956.69507956832,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:54.147084771+08:00",
          "Version": 127
        },
        {
          "store_id": 1,
          "region_id": 1152,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 93013.72966004584,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:51.146018185+08:00",
          "Version": 126
        },
        {
          "store_id": 1,
          "region_id": 52,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 46463.725,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:48.507062772+08:00",
          "Version": 13
        }
      ]
    },
    "4": {
      "total_flow_bytes": 190434.14973961416,
      "regions_count": 3,
      "statistics": [
        {
          "store_id": 4,
          "region_id": 1156,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 50956.69507956832,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:54.147078479+08:00",
          "Version": 127
        },
        {
          "store_id": 4,
          "region_id": 1152,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 93013.72966004584,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:51.146019774+08:00",
          "Version": 126
        },
        {
          "store_id": 4,
          "region_id": 52,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 46463.725,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:48.507064672+08:00",
          "Version": 13
        }
      ]
    },
    "5": {
      "total_flow_bytes": 190434.14973961416,
      "regions_count": 3,
      "statistics": [
        {
          "store_id": 5,
          "region_id": 1156,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 50956.69507956832,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:54.147083702+08:00",
          "Version": 127
        },
        {
          "store_id": 5,
          "region_id": 1152,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 93013.72966004584,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:51.146020767+08:00",
          "Version": 126
        },
        {
          "store_id": 5,
          "region_id": 52,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 46463.725,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:48.507065709+08:00",
          "Version": 13
        }
      ]
    }
  },
  "as_leader": {
    "1": {
      "total_flow_bytes": 0,
      "regions_count": 0,
      "statistics": []
    },
    "4": {
      "total_flow_bytes": 143970.42473961416,
      "regions_count": 2,
      "statistics": [
        {
          "store_id": 4,
          "region_id": 1156,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 50956.69507956832,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:54.147078479+08:00",
          "Version": 127
        },
        {
          "store_id": 4,
          "region_id": 1152,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 93013.72966004584,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:51.146019774+08:00",
          "Version": 126
        }
      ]
    },
    "5": {
      "total_flow_bytes": 46463.725,
      "regions_count": 1,
      "statistics": [
        {
          "store_id": 5,
          "region_id": 52,
          "hot_degree": 3,
          "AntiCount": 1,
          "kind": 0,
          "flow_bytes": 46463.725,
          "flow_keys": 0,
          "RollingBytesRate": null,
          "last_update_time": "2020-01-03T16:10:48.507065709+08:00",
          "Version": 13
        }
      ]
    }
  }
}

From the PD result we can see flow_bytes is a float, not uint. @disksing @nolouch

"flow_bytes": 50956.69507956832

Related changes

  • Need to cherry-pick to the release branch

Release note

  • Write release note for bug-fix or new feature.
@tiancaiamao

This comment has been minimized.

Copy link
Contributor Author

tiancaiamao commented Jan 3, 2020

@tiancaiamao tiancaiamao requested a review from winoros Jan 6, 2020
@tiancaiamao

This comment has been minimized.

Copy link
Contributor Author

tiancaiamao commented Jan 8, 2020

Copy link
Member

zimulala left a comment

LGTM

@zimulala zimulala added the status/LGT1 label Jan 9, 2020
store/helper/helper.go Outdated Show resolved Hide resolved
@tiancaiamao

This comment has been minimized.

Copy link
Contributor Author

tiancaiamao commented Jan 10, 2020

PTAL @winoros

Copy link
Member

zz-jason left a comment

LGTM

@zz-jason zz-jason added status/LGT2 and removed status/LGT1 labels Jan 10, 2020
@zz-jason

This comment has been minimized.

Copy link
Member

zz-jason commented Jan 10, 2020

could you add some tests for this bugfix?

@tiancaiamao

This comment has been minimized.

Copy link
Contributor Author

tiancaiamao commented Jan 10, 2020

could you add some tests for this bugfix?

Not easy to do it in the unit test. If we mock data, it can't prevent PD from changing the interface in the future.

@tiancaiamao

This comment has been minimized.

Copy link
Contributor Author

tiancaiamao commented Jan 10, 2020

/run-all-tests

@tiancaiamao tiancaiamao merged commit ad71a5a into pingcap:master Jan 10, 2020
15 checks passed
15 checks passed
idc-jenkins-ci-tidb/build Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/build_check_race Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/check_dev Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/check_dev_2 Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/common-test job succeeded
Details
idc-jenkins-ci-tidb/integration-common-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/integration-compatibility-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/integration-copr-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/integration-ddl-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/mybatis-test job succeeded
Details
idc-jenkins-ci-tidb/sqllogic-test-1 Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/sqllogic-test-2 Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/unit-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/wasm-build Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details
@tiancaiamao tiancaiamao deleted the tiancaiamao:hot-table-partition branch Jan 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.