Handling 'invalid data size' errors

[blalor]

Getting this error, unsure what the event is: invalid data size 57 in event WriteRowsEventV0, less event length 3215523821

If this is a protocol error, what's the best way to capture the data that's causing it to fail? I got there with #12, but that was painful. :-) How do I continue from this or any other error where I'd like to skip to the next event? There appears to be no way to retrieve the current position.

[siddontang]

Hi @blalor

This error seems that the binlog header says the binlog size is 3215523821, but we only receive 57.
Maybe we receive a incorrect binlog, I guess.

Can you use mysqlbinlog to dump your binlog and see what it outputs?

I will create a new branch do following things:

• use crc32 to check the received binlog, now I just skip checking
• skip the error binlog and handle next, but I am not sure that we can handle next correctly :-)

[siddontang]

@blalor you can check #14 and see again

We will skip this binlog and handle next, but I don't know whether we will meet another different errors. :-)
At the same time, we will check binlog crc32 if your mysql opens it.

[blalor]

Ok, I'll take a look at that. I was able to isolate a bad event similar to the one I originally reported here. Here's a test that reproduces the problem, which also happens on #14's branch:

func (t *testSyncerSuite) TestInvalidDataSize(c *C) {
    parser := NewBinlogParser()
    /*
    00000000  8e 55 02 a5 14 80 0a 56  00 00 00 e5 78 a3 0f 08  |.U.....V....x...|
    00000010  00 c1 a2 f9 01 00 00 00  00 0a 00 00 22 00 00 00  |............"...|
    00000020  00 00 00 01 00 00 00 40  00 00 00 00 06 03 73 74  |.......@......st|
    00000030  64 04 21 00 21 00 08 00  05 06 2d 30 34 3a 30 30  |d.!.!.....-04:00|
    00000040  73 65 69 75 6d 61 73 74  65 72 00 42 45 47 49 4e  |••••••••••.BEGIN|
    00000050  12 bb cd bf                                       |....|
    */
    data := []byte{
        0x8e, 0x55, 0x2, 0xa5, 0x14, 0x80, 0xa, 0x56, 0x0, 0x0, 0x0, 0xe5, 0x78, 0xa3, 0xf, 0x8,
        0x0, 0xc1, 0xa2, 0xf9, 0x1, 0x0, 0x0, 0x0, 0x0, 0xa, 0x0, 0x0, 0x22, 0x0, 0x0, 0x0,
        0x0, 0x0, 0x0, 0x1, 0x0, 0x0, 0x0, 0x40, 0x0, 0x0, 0x0, 0x0, 0x6, 0x3, 0x73, 0x74,
        0x64, 0x4, 0x21, 0x0, 0x21, 0x0, 0x8, 0x0, 0x5, 0x6, 0x2d, 0x30, 0x34, 0x3a, 0x30, 0x30,
        0x73, 0x65, 0x69, 0x75, 0x6d, 0x61, 0x73, 0x74, 0x65, 0x72, 0x0, 0x42, 0x45, 0x47, 0x49, 0x4e,
        0x12, 0xbb, 0xcd, 0xbf,
    }

    _, err := parser.parse(data)

    c.Assert(err, IsNil)
}

[siddontang]

Thank you,I will check it tomorrow because I am on a train now. :-)

[blalor]

Ah. Based on mysqlbinlog, two bytes are missing from the beginning of this packet: 0xef, 0x8e. I think the check for semi-sync replication is causing the problem. The doc isn't clear, but I think checking whether an ack needs to be sent for semi-sync replication should only be done if semi-sync is actually enabled.

[siddontang]

Oh, I don't reply ACK for error binlog, I will handle it later. :-)

[gdey]

@siddontang I was just look at the issues. It seems that this issue should be closed. Am I missing something?

[siddontang]

Yes, fixed already. I will close it.