Skip to content

Conversation

@staticlibs
Copy link
Collaborator

Previously, when auto-commit if off, the implicit transaction was started automatically when PreparedStatement#execute(void) is called. This was found to cause the problems with
Statement#executeQuery(String), that calls prepare(String) and execute() internally. As transaction was starting only after prepare call - the prepare itself was running with effective auto-commit mode (in a separate implicit transaction). It is harmless in most cases, but was found to break transaction-wide caching in duckdb-iceberg.

The problem was only happening when the execute(String) is running outside of active transaction, for example as a first call in freshly opened connection. Still this was causing non-negligible overhead.

This change makes the implicit transaction to be started before prepare call, so the same transaction is active during execute call.

There are no changes in logic when auto-commit is on (default per JDBC spec), so the Iceberg caching is still broken in this mode. This is intended to be fixed in future, for now to have effective caching with duckdb-iceberg it is necessary to disable auto-commit, it can be done by one of the following methods:

In connection config:

Properties config = new Properties();
config.put(DuckDBDriver.JDBC_AUTO_COMMIT, false);
Connection conn = DriverManager.getConnection(url, config);

In connection URL:

Connection conn = DriverManager.getConnection("jdbc:duckdb:path/to/my.db;jdbc_auto_commit=false;");

On a connection manually:

conn.setAutoCommit(false);

Testing: there are no easily observable effects from this change, the transactional logic is covered by existing tests, Iceberg caching is tested manually.

Previously, when auto-commit if off, the implicit transaction was
started automatically when `PreparedStatement#execute(void)` is called.
This was found to cause the problems with
`Statement#executeQuery(String)`, that calls `prepare(String)` and
`execute()` internally. As transaction was starting only after `prepare`
call - the `prepare` itself was running with effective auto-commit mode
(in a separate implicit transaction). It is harmless in most cases, but
was found to break transaction-wide caching in `duckdb-iceberg`.

The problem was only happening when the `execute(String)` is running
outside of active transaction, for example as a first call in freshly
opened connection. Still this was causing non-negligible overhead.

This change makes the implicit transaction to be started before
`prepare` call, so the same transaction is active during `execute`
call.

There are no changes in logic when auto-commit is on (default per JDBC
spec), so the Iceberg caching is still broken in this mode. This is
intended to be fixed in future, for now to have effective caching with
`duckdb-iceberg` it is necessary to disable auto-commit, it can be done
by one of the following methods:

In connection config:

```java
Properties config = new Properties();
config.put(DuckDBDriver.JDBC_AUTO_COMMIT, false);
Connection conn = DriverManager.getConnection(url, config);
```

In connection URL:

```java
Connection conn = DriverManager.getConnection("jdbc:duckdb:path/to/my.db;jdbc_auto_commit=false;");
```

On a connection manually:

```java
conn.setAutoCommit(false);
```

Testing: there are no easily observable effects from this change, the
transactional logic is covered by existing tests, Iceberg caching is
tested manually.
@staticlibs staticlibs merged commit ec50d5b into duckdb:main Nov 28, 2025
10 checks passed
@staticlibs staticlibs deleted the prepare_exec_tran branch November 28, 2025 08:15
staticlibs added a commit to staticlibs/duckdb-java that referenced this pull request Nov 28, 2025
This is a backport of the PR duckdb#467 to `v1.4-andium` stable branch.

Previously, when auto-commit if off, the implicit transaction was
started automatically when `PreparedStatement#execute(void)` is called.
This was found to cause the problems with
`Statement#executeQuery(String)`, that calls `prepare(String)` and
`execute()` internally. As transaction was starting only after `prepare`
call - the `prepare` itself was running with effective auto-commit mode
(in a separate implicit transaction). It is harmless in most cases, but
was found to break transaction-wide caching in `duckdb-iceberg`.

The problem was only happening when the `execute(String)` is running
outside of active transaction, for example as a first call in freshly
opened connection. Still this was causing non-negligible overhead.

This change makes the implicit transaction to be started before
`prepare` call, so the same transaction is active during `execute`
call.

There are no changes in logic when auto-commit is on (default per JDBC
spec), so the Iceberg caching is still broken in this mode. This is
intended to be fixed in future, for now to have effective caching with
`duckdb-iceberg` it is necessary to disable auto-commit, it can be done
by one of the following methods:

In connection config:

```java
Properties config = new Properties();
config.put(DuckDBDriver.JDBC_AUTO_COMMIT, false);
Connection conn = DriverManager.getConnection(url, config);
```

In connection URL:

```java
Connection conn = DriverManager.getConnection("jdbc:duckdb:path/to/my.db;jdbc_auto_commit=false;");
```

On a connection manually:

```java
conn.setAutoCommit(false);
```

Testing: there are no easily observable effects from this change, the
transactional logic is covered by existing tests, Iceberg caching is
tested manually.
staticlibs added a commit that referenced this pull request Nov 28, 2025
This is a backport of the PR #467 to `v1.4-andium` stable branch.

Previously, when auto-commit if off, the implicit transaction was
started automatically when `PreparedStatement#execute(void)` is called.
This was found to cause the problems with
`Statement#executeQuery(String)`, that calls `prepare(String)` and
`execute()` internally. As transaction was starting only after `prepare`
call - the `prepare` itself was running with effective auto-commit mode
(in a separate implicit transaction). It is harmless in most cases, but
was found to break transaction-wide caching in `duckdb-iceberg`.

The problem was only happening when the `execute(String)` is running
outside of active transaction, for example as a first call in freshly
opened connection. Still this was causing non-negligible overhead.

This change makes the implicit transaction to be started before
`prepare` call, so the same transaction is active during `execute`
call.

There are no changes in logic when auto-commit is on (default per JDBC
spec), so the Iceberg caching is still broken in this mode. This is
intended to be fixed in future, for now to have effective caching with
`duckdb-iceberg` it is necessary to disable auto-commit, it can be done
by one of the following methods:

In connection config:

```java
Properties config = new Properties();
config.put(DuckDBDriver.JDBC_AUTO_COMMIT, false);
Connection conn = DriverManager.getConnection(url, config);
```

In connection URL:

```java
Connection conn = DriverManager.getConnection("jdbc:duckdb:path/to/my.db;jdbc_auto_commit=false;");
```

On a connection manually:

```java
conn.setAutoCommit(false);
```

Testing: there are no easily observable effects from this change, the
transactional logic is covered by existing tests, Iceberg caching is
tested manually.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant