Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch writing to postgresql (timescale) with pgx isn't really a batch write to the db #17

Closed
micsjo opened this issue Jan 11, 2023 · 2 comments

Comments

@micsjo
Copy link
Contributor

micsjo commented Jan 11, 2023

Using batch from pgx doesn't write the data to the db in a batch write.

In effect it just piles all the inserts in a a big package (the "batch") and sends the big package in one go.

The db then commences to do all the inserts in one transaction but in sequence. If the sequence of inserts takes too much time, the connection will be severed and the fail ignored by pgx.

Batches are very limited in capacity for high volume use cases and need a better implementation.

@micsjo
Copy link
Contributor Author

micsjo commented Jan 11, 2023

#18 fixes the issue

@javaducky
Copy link
Contributor

Fixed with #18.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants