Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: make prepared insert batches use bulk inserts #99319

Open
Tracked by #91831
cucaroach opened this issue Mar 22, 2023 · 0 comments
Open
Tracked by #91831

sql: make prepared insert batches use bulk inserts #99319

cucaroach opened this issue Mar 22, 2023 · 0 comments
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-sql-queries SQL Queries Team

Comments

@cucaroach
Copy link
Contributor

cucaroach commented Mar 22, 2023

Make prepared batches process inserts in batches instead of row by row (ie when we see prepare/bind/exec on pgwire, treat “exec” to mean “batch” and only exec when batch gets big enough or we see a SYNC). I think this will work. Requires maintaining fail or success on a per row basis, JDBC prepared batch API supports this, its all run as one transaction but some rows can succeed and some can fail. In theory this should be the fastest insert path, no CSV parsing like in COPY and no ParseAndRequire overhead since values are strongly typed on the wire.

The idea is to follow the vectorized COPY lead and build up a coldata.Batch of each row and then throw those at the insert distsql processor.

Jira issue: CRDB-25814

@cucaroach cucaroach added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Mar 22, 2023
@cucaroach cucaroach self-assigned this Mar 22, 2023
@cucaroach cucaroach added this to Triage in SQL Queries via automation Mar 22, 2023
@cucaroach cucaroach added this to To do in Migrations via automation Mar 22, 2023
@blathers-crl blathers-crl bot added the T-sql-queries SQL Queries Team label Mar 22, 2023
@cucaroach cucaroach moved this from Triage to 23.2 Release in SQL Queries Mar 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-sql-queries SQL Queries Team
Projects
Status: Backlog
SQL Queries
23.2 Release
Development

No branches or pull requests

1 participant