Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

meta data check should ignore AUTO_INCREMENT difference #867

Closed
Lordess opened this issue Nov 29, 2018 · 3 comments
Closed

meta data check should ignore AUTO_INCREMENT difference #867

Lordess opened this issue Nov 29, 2018 · 3 comments
Assignees
Labels
community issue from community resolve problem has been fixed by developer
Milestone

Comments

@Lordess
Copy link
Contributor

Lordess commented Nov 29, 2018

  • dble version:
    2.18.10.1 release

  • preconditions :

  1. Physical tables have auto-increment column:
CREATE TABLE sharding_table (
id BIGINT PRIMARY KEY AUTO_INCREMENT,
clientNum CHAR(20) NOT NULL )
  1. Records are distributed inequally on dataNodes( physical tables ). Thus, AUTO_INCREMENT is different between physical tables in the result of SHOW CREATE TABLE.
-- Execute belows on dble 8066 AFTER dble and dble are ready
INSERT INTO testdb.sharding_table
VALUES
('00000000000000000001'),
('00000000000000000002'),
('00000000000000000003');
  • configs:

schema.xml

<schema name="testdb">
    <table name="sharding_table" rule="mod-by-clientNum" dataNodes="dn1,dn2" />
</schema>

rule.xml

<tableRule name="mod-by-clientNum" >
    <rule>
        <columns>clientNum</columns>
        <algorithms>mod-by-two</algorithm>
    </rule>
</tableRule>

<function name="mod-by-two" class="Hash">
    <property name="partitionCount">2</property>
    <property name="partitionLength">1</property>
</function>

server.xml
ommitted

  • steps:
    step1. Issue a RELOAD command to dble on 9066.
  • expect result:
    1. Sucess.
  • real result:
    1. Got failure. And you will got errors like belows.
yyyy-mm-dd hh:MM:SS.sss  WARN [complexQueryExecutorXXX] (com.actiontech.dble.meta.table.MultiTablesMetaHandler.consistentWarning(MultiTablesMetaHandler.java:192)) - Table[sharding_table] structure are not consistent in different data node!
yyyy-mm-dd hh:MM:SS.sss  INFO [complexQueryExecutorXXX] (com.actiontech.dble.meta.table.MultiTablesMetaHandler.consistentWarning(MultiTablesMetaHandler.java:195)) - Currently detected:
yyyy-mm-dd hh:MM:SS.sss  INFO [complexQueryExecutorXXX] (com.actiontech.dble.meta.table.MultiTablesMetaHandler.consistentWarning(MultiTablesMetaHandler.java:202)) - DataNode:[dn1]:CREATE TABLE `sharding_table` (
  `id` BIGINT PRIMARY KEY AUTO_INCREMENT,
  `clientNum` CHAR(20) NOT NULL )
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin=[dn1]
yyyy-mm-dd hh:MM:SS.sss  INFO [complexQueryExecutorXXX] (com.actiontech.dble.meta.table.MultiTablesMetaHandler.consistentWarning(MultiTablesMetaHandler.java:202)) - DataNode:[dn2]:CREATE TABLE `sharding_table` (
  `id` BIGINT PRIMARY KEY AUTO_INCREMENT,
  `clientNum` CHAR(20) NOT NULL )
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin=[dn2]
  • supplements:
    Why I use auto-increment in a sharding table:
    1. InnoDB has an low level optimization while user asking for inserting records with sorted sequential primary key. I need it.
    2. Values in a business column would change and causing B+ tree rotation and index update. Business column is not ideal primary key.
    3. Business logic has no interest in maintaining such an internal use column. Auto increment is a good idea for self maintaining.
    4. Only do I use "excessive sharding" solution so that I have no need to merge physical tables. Primary key conflict won't happen.

/label ~BUG

@yanhuqing666 yanhuqing666 added this to the 2.18.12.0 milestone Nov 30, 2018
yanhuqing666 added a commit that referenced this issue Dec 5, 2018
#867 meta data check ignore AUTO_INCREMENT difference
@yanhuqing666
Copy link
Member

the result of check @@meatadata may not correct.

@sunsun314 sunsun314 assigned irene-coming and unassigned sunsun314 Dec 5, 2018
@sunsun314 sunsun314 added the resolve problem has been fixed by developer label Dec 5, 2018
@yanhuqing666
Copy link
Member

to @Lordess :
In your case, column id must NOT config as primaryKey in table's attribute of schema.xml, else it will cache a single datanode as routing result

@FlyingMao
Copy link

verified version: 5.6.29-dble-9.9.9.9-53ddf3e-20181214052815

@FlyingMao FlyingMao assigned FlyingMao and unassigned irene-coming Dec 21, 2018
@yanhuqing666 yanhuqing666 added the community issue from community label Jan 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community issue from community resolve problem has been fixed by developer
Projects
None yet
Development

No branches or pull requests

5 participants