Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add WriteBatch to WriteAPIBlocking #295

Closed
pabigot opened this issue Jan 1, 2022 · 1 comment
Closed

Add WriteBatch to WriteAPIBlocking #295

pabigot opened this issue Jan 1, 2022 · 1 comment

Comments

@pabigot
Copy link
Contributor

pabigot commented Jan 1, 2022

Proposal:
Extend the blocking API to allow writing a batch of already-aggregated line protocol records.

Current behavior:
The blocking API only accepts individual points, or points already converted to records. The implemented functions then go on to do the conversion and aggregation to create a batch.

Desired behavior:
To write a batch that I've created myself.

Use case:
I'm continuing my attempt to use this package in a context where I can't tolerate lost data due to the server being inaccessible.

The non-blocking API has a variety of problems raised in other issues, but the showstopper is that it will drop data without allowing the application to recover the data; e.g. in this path but also in a shutdown situation.

The documentation of WriteAPIBlocking notes that it should be used when:

there is a need to have explicit control of failed batches.

The problem in my situation is that WriteAPIBlocking accepts only on individual points or line-protocol records (possibly in a slice, but still separated). In my application, if I need to persist unwritten points the natural encoding is newline-separated (or terminated) line protocol records in a byte sequence. However, when I read that back off disk and prepare to resend it the current API requires me to split it into individual records at the newline, so that WriteRecord can then add the newlines back and aggregate it again.

This is cumbersome, but trivially avoided by extending the API to allow writing a batch.

@vlastahajek
Copy link
Contributor

The already existing WriteRecord function can also take just a single string. And such string can be a batch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants