Flush - is there a way to check for buffer length or is it really required to manual flush?

[razuro]

I'm currently using HTTP (not TCP), and I wanted to optimize the buffer capacity. So instead of manually flushing after each entry;

• is there a way to check if the buffer can fit more data, then auto flush and auto close if the next message exceed the capacity?
• I saw autoFlush() and DEFAULT_HTTP_AUTO_FLUSH_ROWS so it seems manual flush is actually optional?

[glasstiger]

"I wanted to optimize the buffer capacity" -> Can I ask what you mean exactly by this?

By default auto flush is enabled.
The buffer's content is sent to the server every second or after every 75000 rows, whichever comes first.
There are config options to modify these defaults: auto_flush_rows and auto_flush_interval.
If the buffer fills up before its content is sent to the server, it is automatically extended.
The buffer can expand until it reaches the max buffer size, which is set to 100MB by default.
The max buffer size can be modified via the max_buf_size option.

We tried to make it convenient for you to use the library, and tried to avoid forcing you to check the buffer capacity on every write.
The idea is that for most use cases auto flushing should work, you might need to tune those parameters mentioned above.
If auto flushing is not an option for some reason, it is possible to disable it.

[razuro]

Thanks @glasstiger, that answered everything :)

Just for context, I was on version 2.1.0 previously, was using TCP, and I'm not sure whether that auto flush was an option before. I need to count rows of data, and if it exceed certain rows, I'll be manually flushing, example:

if (counter % rowCount === 0) {
        await sender.flush()
      }

With the new version and ability to use HTTP, I don't think I need to worry about buffer size or the possible case of exceeding it anymore 👍

Thanks again!

PS: Unlike the old version QuestDB docs, the new one doesn't show nodejs-questdb-client docs

[glasstiger]

Thanks @glasstiger, that answered everything :)

Just for context, I was on version 2.1.0 previously, was using TCP, and I'm not sure whether that auto flush was an option before. I need to count rows of data, and if it exceed certain rows, I'll be manually flushing, example:

if (counter % rowCount === 0) {
        await sender.flush()
      }

With the new version and ability to use HTTP, I don't think I need to worry about buffer size or the possible case of exceeding it anymore 👍

Thanks again!

PS: Unlike the old version QuestDB docs, the new one doesn't show nodejs-questdb-client docs

You are correct that auto flushing was introduced recently in v3, and also that QuestDB docs have been restructured, and now this is the page which references the node.js client:
https://questdb.io/docs/clients/ingest-node/
It takes you to the proper node.js client doc where you can see the examples.