Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maxwell shutdown on 'bool' and 'boolean' #67

Closed
akshayi1 opened this issue Jun 18, 2015 · 3 comments
Closed

Maxwell shutdown on 'bool' and 'boolean' #67

akshayi1 opened this issue Jun 18, 2015 · 3 comments

Comments

@akshayi1
Copy link

Same as #64, #65 and #66, Maxwell can't handle 'bool' and (just to be completely sure) 'boolean'. This occurs at the create table stage. To get Maxwell to run again, I had to purge binary logs before now() and delete from maxwell.positions.

BOOL:

bin/maxwell --user='root' --password='root' --host='127.0.0.1' --producer=stdout
Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar 
10:40:30,183 INFO  Maxwell - Maxwell is capturing initial schema
10:40:30,185 DEBUG SchemaStore - Capturing schema
10:40:30,217 DEBUG SchemaStore - Finished capturing schema
10:40:30,294 INFO  TransportImpl - connecting to host: 127.0.0.1, port: 3306
10:40:30,343 INFO  TransportImpl - connected to host: 127.0.0.1, port: 3306, context: AbstractTransport.Context[threadId=31,scramble=}_EvG<6<gt16X;]DyW(],protocolVersion=10,serverHost=127.0.0.1,serverPort=3306,serverStatus=2,serverCollation=8,serverVersion=5.6.24-0ubuntu2-log,serverCapabilities=63487]
10:40:30,344 INFO  AuthenticatorImpl - start to login, user: root, host: 127.0.0.1, port: 3306
10:40:30,348 INFO  AuthenticatorImpl - login successfully, user: root, detail: OKPacket[packetMarker=0,affectedRows=0,insertId=0,serverStatus=2,warningCount=0,message=<null>]
10:40:31,175 DEBUG SchemaPosition - Writing binlog position to maxwell.positions: BinlogPosition[mysql-logs.000003:12471]
10:55:36,558 DEBUG SchemaChange - SQL_PARSE <- "create table mmf_test(col1 smallint, col2 int, col3 bigint,  col4 float, col5 date, col6 datetime, col7 decimal(10,2), col8 varchar(500), col9 text, col10 bool)"
line 1:155 no viable alternative at input 'bool'
10:55:36,590 ERROR MysqlParserListener - (data_type bool)
10:55:36,590 ERROR SchemaChange - Error parsing SQL: 'create table mmf_test(col1 smallint, col2 int, col3 bigint,  col4 float, col5 date, col6 datetime, col7 decimal(10,2), col8 varchar(500), col9 text, col10 bool)'
com.zendesk.maxwell.schema.ddl.MaxwellSQLSyntaxError: bool
    at com.zendesk.maxwell.schema.ddl.MysqlParserListener.visitErrorNode(MysqlParserListener.java:85)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:41)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at com.zendesk.maxwell.schema.ddl.SchemaChange.parse(SchemaChange.java:65)
    at com.zendesk.maxwell.MaxwellParser.processQueryEvent(MaxwellParser.java:206)
    at com.zendesk.maxwell.MaxwellParser.getEvent(MaxwellParser.java:192)
    at com.zendesk.maxwell.MaxwellParser.getEvent(MaxwellParser.java:198)
    at com.zendesk.maxwell.MaxwellParser.run(MaxwellParser.java:98)
    at com.zendesk.maxwell.Maxwell.run(Maxwell.java:69)
    at com.zendesk.maxwell.Maxwell.main(Maxwell.java:75)
2015-06-18 17:55:36,635 FATAL Unable to register shutdown hook because JVM is shutting down.

BOOLEAN:

bin/maxwell --user='root' --password='root' --host='127.0.0.1' --producer=stdout
Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar 
10:58:46,071 INFO  Maxwell - Maxwell is capturing initial schema
10:58:46,073 DEBUG SchemaStore - Capturing schema
10:58:46,111 DEBUG SchemaStore - Finished capturing schema
10:58:47,020 INFO  TransportImpl - connecting to host: 127.0.0.1, port: 3306
10:58:47,044 INFO  TransportImpl - connected to host: 127.0.0.1, port: 3306, context: AbstractTransport.Context[threadId=35,scramble=pWD+Jt,w6D5z*_~!&7#J,protocolVersion=10,serverHost=127.0.0.1,serverPort=3306,serverStatus=2,serverCollation=8,serverVersion=5.6.24-0ubuntu2-log,serverCapabilities=63487]
10:58:47,045 INFO  AuthenticatorImpl - start to login, user: root, host: 127.0.0.1, port: 3306
10:58:47,054 INFO  AuthenticatorImpl - login successfully, user: root, detail: OKPacket[packetMarker=0,affectedRows=0,insertId=0,serverStatus=2,warningCount=0,message=<null>]
10:58:47,063 DEBUG SchemaPosition - Writing binlog position to maxwell.positions: BinlogPosition[mysql-logs.000003:18202]
10:58:55,665 DEBUG SchemaChange - SQL_PARSE <- "create table mmf_test(col1 smallint, col2 int, col3 bigint,  col4 float, col5 date, col6 datetime, col7 decimal(10,2), col8 varchar(500), col9 text, col10 boolean)"
line 1:155 no viable alternative at input 'boolean'
10:58:55,712 ERROR MysqlParserListener - (data_type boolean)
10:58:55,712 ERROR SchemaChange - Error parsing SQL: 'create table mmf_test(col1 smallint, col2 int, col3 bigint,  col4 float, col5 date, col6 datetime, col7 decimal(10,2), col8 varchar(500), col9 text, col10 boolean)'
com.zendesk.maxwell.schema.ddl.MaxwellSQLSyntaxError: boolean
    at com.zendesk.maxwell.schema.ddl.MysqlParserListener.visitErrorNode(MysqlParserListener.java:85)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:41)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:52)
    at com.zendesk.maxwell.schema.ddl.SchemaChange.parse(SchemaChange.java:65)
    at com.zendesk.maxwell.MaxwellParser.processQueryEvent(MaxwellParser.java:206)
    at com.zendesk.maxwell.MaxwellParser.getEvent(MaxwellParser.java:192)
    at com.zendesk.maxwell.MaxwellParser.getEvent(MaxwellParser.java:198)
    at com.zendesk.maxwell.MaxwellParser.run(MaxwellParser.java:98)
    at com.zendesk.maxwell.Maxwell.run(Maxwell.java:69)
    at com.zendesk.maxwell.Maxwell.main(Maxwell.java:75)
@akshayi1
Copy link
Author

For this, I tried to do:

ColumnDef.java

public static ColumnDef build(String tableName, String name, String encoding, String type, int pos, boolean signed, String enumValues[]) {
        type = unalias_type(type);

        switch(type) {

        case "tinyint":
        case "smallint":
        case "mediumint":
        case "bit":
        case "int":
            return new IntColumnDef(tableName, name, type, pos, signed);
....
....
....

static private String unalias_type(String type) {
        switch(type) {
            case "bool":
            case "boolean":
            case "bit":
                return "bit";
            case "int1":
                return "tinyint";
            case "int2":
                return "smallint";
            case "int3":
                return "mediumint";
            case "int4":
            case "integer":
                return "int";
            case "int8":
                return "bigint";
            default:
                return type;
        }
    }

IntColumnDef.java

public boolean matchesMysqlType(int type) {
        switch(this.bits) {
        case 1:
            return type == MySQLConstants.TYPE_BIT;
        case 8:
            return type == MySQLConstants.TYPE_TINY;
        case 16:
            return type == MySQLConstants.TYPE_SHORT;
        case 24:
            return type == MySQLConstants.TYPE_INT24;
        case 32:
            return type == MySQLConstants.TYPE_LONG;
        default:
            return false;
        }
    }

    private final static int bitsFromType(String type) {
        switch(type) {
        case "bit":
            return 1;
        case "tinyint":
            return 8;
        case "smallint":
            return 16;
        case "mediumint":
            return 24;
        case "int":
            return 32;
        default:
            return 0;
        }
    }

I tried a few combinations of the same - without the bit and so on, and nothing seemed to work. It was almost as if my changed code did not even exist! I've kinda given up on getting bool to work.

@osheroff
Copy link
Collaborator

working on this one.

@osheroff osheroff mentioned this issue Jun 18, 2015
@akshayi1
Copy link
Author

#68 fixes this. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants