Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

utf-8 characters in long string #2952

Closed
monetdb-team opened this issue Nov 30, 2020 · 0 comments
Closed

utf-8 characters in long string #2952

monetdb-team opened this issue Nov 30, 2020 · 0 comments

Comments

@monetdb-team
Copy link

@monetdb-team monetdb-team commented Nov 30, 2020

Date: 2011-12-21 10:59:51 +0100
From: Wouter Alink <>
To: clients devs <>
Version: 11.5.9 (Aug2011-SP3) [obsolete]

Last updated: 2012-01-26 15:31:58 +0100

Comment 16670

Date: 2011-12-21 10:59:51 +0100
From: Wouter Alink <>

User-Agent: Mozilla/5.0 (Windows NT 6.0; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1
Build Identifier:

When inserting a long string via JDBC (using batched prepared statements) containing multi-byte UTF-8 characters, the JDBC-driver raises an exception.

Reproducible: Always

Steps to Reproduce:

import java.nio.charset.Charset;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException;
import java.sql.Statement;

import org.junit.Test;

public class TestChar {

@Test
public void testChars() throws ClassNotFoundException, SQLException {
	
	// 3-byte UTF-8 symbol (a dot, but I think it could be any 3-byte character): 
	byte[] errorBytes = new byte[] { (byte) 0xe2, (byte) 0x80, (byte) 0xa7 };
	String errorStr = new String(errorBytes, Charset.forName("UTF-8"));
	StringBuilder repeatedErrorStr = new StringBuilder();
	for(int i = 0; i < 8170;i++) {
		repeatedErrorStr.append(errorStr);
	}
	
	System.out.println("TestString: " + errorStr);
	
	Class.forName("nl.cwi.monetdb.jdbc.MonetDriver");
	String connectionURL = "jdbc:monetdb://localhost:50000/vv01?language=sql&user=monetdb&password=monetdb";
	Connection conn = DriverManager.getConnection(connectionURL);
	try {
		Statement st = conn.createStatement();
		
		try { st.execute("DROP TABLE x;"); } catch (SQLException e) { }
		st.execute("CREATE TABLE x (c INT, a CLOB, b DOUBLE);");
		
		PreparedStatement ps = conn.prepareStatement("INSERT INTO x VALUES (?,?,?);");
		ps.setLong(1, 1);
		ps.setString(2, repeatedErrorStr.toString());
		ps.setDouble(3, 1.0);
		ps.addBatch();
		ps.executeBatch();

		st.close();
	} catch (SQLException e) {
		e.printStackTrace();
		if (e.getNextException() != null) {
			e.getNextException().printStackTrace();
		}
		throw e;
	} finally {
		conn.close();
	}
}

}

Actual Results:

TestString: ‧
java.sql.BatchUpdateException: Error(s) occurred while executing the batch, see next SQLExceptions for details
at nl.cwi.monetdb.jdbc.MonetStatement.executeBatch(MonetStatement.java:199)
at com.spinque.tools.importStream.io.TestChar.testChars(TestChar.java:44)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.junit.internal.runners.TestMethodRunner.executeMethodBody(TestMethodRunner.java:99)
at org.junit.internal.runners.TestMethodRunner.runUnprotected(TestMethodRunner.java:81)
at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)
at org.junit.internal.runners.TestMethodRunner.runMethod(TestMethodRunner.java:75)
at org.junit.internal.runners.TestMethodRunner.run(TestMethodRunner.java:45)
at org.junit.internal.runners.TestClassMethodsRunner.invokeTestMethod(TestClassMethodsRunner.java:66)
at org.junit.internal.runners.TestClassMethodsRunner.run(TestClassMethodsRunner.java:35)
at org.junit.internal.runners.TestClassRunner$1.runUnprotected(TestClassRunner.java:42)
at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)
at org.junit.internal.runners.TestClassRunner.run(TestClassRunner.java:52)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
java.sql.SQLException: Overflow: don't use multi statements when batching (0)
at nl.cwi.monetdb.jdbc.MonetStatement.internalBatch(MonetStatement.java:250)
at nl.cwi.monetdb.jdbc.MonetStatement.executeBatch(MonetStatement.java:219)
at com.spinque.tools.importStream.io.TestChar.testChars(TestChar.java:44)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.junit.internal.runners.TestMethodRunner.executeMethodBody(TestMethodRunner.java:99)
at org.junit.internal.runners.TestMethodRunner.runUnprotected(TestMethodRunner.java:81)
at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)
at org.junit.internal.runners.TestMethodRunner.runMethod(TestMethodRunner.java:75)
at org.junit.internal.runners.TestMethodRunner.run(TestMethodRunner.java:45)
at org.junit.internal.runners.TestClassMethodsRunner.invokeTestMethod(TestClassMethodsRunner.java:66)
at org.junit.internal.runners.TestClassMethodsRunner.run(TestClassMethodsRunner.java:35)
at org.junit.internal.runners.TestClassRunner$1.runUnprotected(TestClassRunner.java:42)
at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)
at org.junit.internal.runners.TestClassRunner.run(TestClassRunner.java:52)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)

Expected Results:

no exception is expected.

note: with 8170 3-byte characters the exception is raised, with 8169 3-byte characters it is not. Might it perhaps be a buffer issue?

Comment 16671

Date: 2011-12-21 11:17:23 +0100
From: @grobian

you fall outside of the BLOCK size, which apparently makes things go boom with a misleading exception:

java.sql.SQLException: Overflow: don't use multi statements when batching (0)

Comment 16672

Date: 2011-12-21 11:36:30 +0100
From: @grobian

Changeset 2232a2c1f642 made by Fabian Groffen fabian@cwi.nl in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=2232a2c1f642

Changeset description:

JDBC add test for bug #2952

Comment 16673

Date: 2011-12-21 12:18:00 +0100
From: @grobian

Changeset e16c4bb803ce made by Fabian Groffen fabian@cwi.nl in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=e16c4bb803ce

Changeset description:

JDBC: fix "overflow" due to very large statements

Off-by-one error that was never triggered before, because offset and max
were never equal.  However, for bug #2952 the first statement is larger
than what the MAPI block can hold, and hence offset = i = 0.  max is
expected to be a count, though.

Fixes bug #2952

Comment 16674

Date: 2011-12-21 12:21:06 +0100
From: @grobian

Changeset ef01bf784b71 made by Fabian Groffen fabian@cwi.nl in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=ef01bf784b71

Changeset description:

Add test for bug #2952

Comment 16818

Date: 2012-01-26 15:31:58 +0100
From: @sjoerdmullender

The Dec2011 version has been release, so declaring this bug as FIXED.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant