Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clients cannot specify batch size for inserts #12

Open
osawyerr opened this issue Dec 11, 2020 · 1 comment
Open

Clients cannot specify batch size for inserts #12

osawyerr opened this issue Dec 11, 2020 · 1 comment

Comments

@osawyerr
Copy link

osawyerr commented Dec 11, 2020

In the SODA driver the batch size used in the PreparedStatement for persisting a collection of documents is hardcoded to 100. It would be a useful addition to enable the caller to specify the batch size themselves to override the default and optimise inserts for their workloads. For example, in our tests for our workload setting the batch size to 10,000 when using raw JDBC to persist 100,000 JSON (BLOB) documents results in a massive performance increase over using 100.

Perhaps it would be useful to let the caller specify what batch size to use to optimise their inserts. Something along the lines of:

public List<OracleDocument> insertAndGet(Iterator<OracleDocument> documents, int batchSize)
    throws OracleBatchException;

alternatively allowing callers to specify the batch size to use in the "hints" options.

public List<OracleDocument> insertAndGet(Iterator<OracleDocument> documents, Map<String, ?> options)
    throws OracleBatchException;

Thoughts?

@osawyerr osawyerr changed the title SODA client's cannot specify batch size Clients cannot specify batch size for inserts Dec 11, 2020
@morgiyan
Copy link
Member

morgiyan commented Dec 11, 2020

Thank you for the suggestion, makes sense. Will look into adding this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants