Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apply Settings.batchSize also to other batch API #14840

Closed
8 tasks done
lukaseder opened this issue Mar 23, 2023 · 6 comments
Closed
8 tasks done

Apply Settings.batchSize also to other batch API #14840

lukaseder opened this issue Mar 23, 2023 · 6 comments

Comments

@lukaseder
Copy link
Member

lukaseder commented Mar 23, 2023

jOOQ 3.14 introduced the Settings.batchSize (#10693), to govern the size of a batch collected in a BatchedConnection. It's not unlikely users would expect this Settings also to apply to any other Batch API usage, given the generic name of the Settings. For example:
https://stackoverflow.com/q/75820566/521799

Changing this can be done in 2 ways:

  • By adding another flag that allows for opting in or out of this additional behaviour
  • By changing the existing behaviour, assuming that most users keep the default of Integer.MAX_VALUE and set the batchSize only explicitly on a BatchedConnection, when needed. This is the one.

This also affects:

  • The Loader implementation, as it uses the Batch API behind the scenes.
  • BatchSingle
  • BatchMultiple
  • BatchCRUD is using BatchSingle or BatchMultiple behind the scenes, and as such, receives the same chunked behaviour.

This does not affect

  • The R2DBC batch implementation (yet). I might implement this on demand.
  • The ExecuteListener lifecycle (we're not reusing the PreparedStatement, calling clearBatch())
@lukaseder
Copy link
Member Author

Looking into this now. The implementation will specifically exclude the R2DBC implementation for now.

@lukaseder
Copy link
Member Author

There are 2 ways to implement this:

  1. Create chunks of queries (BatchMultiple) / bind variable sets (BatchSingle) first, then run multiple executions
  2. Reuse ExecuteContext and PreparedStatement (if applicable), and execute the same statement multiple times, using clearBatch() in between.

The first approach is easier to implement, but I think the second one is preferrable, especially in the BatchSingle case, where this approach allows for reusing the PreparedStatement, avoiding multiple renderings as well.

@lukaseder
Copy link
Member Author

The BatchSingle implementation has been updated.

@lukaseder
Copy link
Member Author

lukaseder commented May 3, 2023

Hmm, regrettably, the ExecuteContext is initialised with a fixed idea of how many batch queries are in the batch, so I'm afraid we won't be able to re-use the PreparedStatement without significant changes to the ExecuteListener lifecycle in the case of BatchMultiple

@lukaseder
Copy link
Member Author

I'll investigate this a bit more. Not too happy with how BatchSingle and BatchMultiple differ. It seems that the ExecuteContext::batchRows method doesn't return the right update count anyway, in the BatchSingle case.

@lukaseder
Copy link
Member Author

I'll investigate ExecuteListener lifecycle events separately: #15038

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Development

No branches or pull requests

1 participant