MySqlBulkCopy: MySqlConnector.MySqlException: 'Error during LOAD DATA LOCAL INFILE'

[RobertGlynWilliams]

Hi,

Thank you for this great library! ;)

I periodically get exceptions from the MySqlBulkCopy class.
This happens when the utf8 encoder tries to encode a unicode character into a buffer with a length of 1.

Exception:
The output byte buffer is too small to contain the encoded data, encoding 'Unicode (UTF-8)' fallback 'System.Text.EncoderReplacementFallback'. (Parameter 'bytes')

Example code to reproduce:

static async Task Main(string[] args)
{
    /*
    CREATE TABLE `testtbl` (
	    `a` MEDIUMTEXT NOT NULL,
	    `b` VARCHAR(255) NOT NULL
    )
     */

    var a = new StringBuilder();
    for (int i = 0; i < 1048573; i++)
    {
        a.Append("a");
    }

    var b = "Å string that crashes application";

    var dataTable = new DataTable();
    dataTable.Columns.Add("a");
    dataTable.Columns.Add("b");
    dataTable.Rows.Add(new object[] { a, b });

    using var connection = new MySqlConnection("...;AllowLoadLocalInfile=true");
    await connection.OpenAsync();
    var bulkCopy = new MySqlBulkCopy(connection);
    bulkCopy.DestinationTableName = "testtbl";
    await bulkCopy.WriteToServerAsync(dataTable);
}

[bgrainger]

Thanks for the great repro!

[bgrainger]

I naively assumed that Convert would output the leading bytes and maintain internal state to output the rest of the UTF-8 bytes in the next block. (And apparently I wasn't the only one: npgsql/npgsql#2849.)

The fix will be to call GetByteCount to ensure that at least one char can be converted.

[bgrainger]

Even worse, I already made this mistake but didn't learn from it ☹ #520.

[RobertGlynWilliams]

Thank you for the bugfix 💯 .

[bgrainger]

Fixed in 1.3.5.

[chrisdcmoore]

Hey @bgrainger - I noticed this issue referenced an issue I opened and fixed in Npgsql a while back, and I just wanted to give you a heads up that I've discovered the fix I wrote doesn't appear to be entirely correct - I've opened another issue against Npgsql describing what I think is going on over there, so you may wish to check whether or not it also affects your fix to this issue here!

npgsql/npgsql#3733

Cheers

[bgrainger]

Thanks @chrisdcmoore; it looks like my fix is subject to this same problem. I appreciate your bringing it to my attention.

[bgrainger]

Improved fix in 1.3.9.