Slow writes into influx

[Korkd]

I've been trying to troubleshoot an issue where writes into influx seem to come in very slowly over time. My setup is as follows:

influxdb 0.8.8
fluent-plugin-influxdb (0.1.4)
fluentd (0.12.8)

fluentd config:
<match fuse.**>
type influxdb
host ***
port 8086
dbname ***
user ***
password ***
time_precision ms
buffer_type memory
buffer_chunk_limit 524288 # 512 * 1024
buffer_queue_limit 1024
flush_interval 30
retry_limit 17
retry_wait 1.0

I've played around and tried a number of different combinations of the config.

I have 2 fluentd instances dumping into a single influxdb instance. All machines are barely registering any CPU usage when messages are being sent, but it takes 20-30 minutes for the buffers to clear out into influx after all the data has been written to fluentd. This is around 72000 messages total (each message is ~100 bytes) being sent.

Turning debug logging on in influx I see a ton of spam about authenticating the user specified in the fluentd config, 10-20 a second. Any thoughts on what the issue might be?

[repeatedly]

Hmm...

fluent-plugin-influxdb/lib/fluent/plugin/out_influxdb.rb

Line 49 in 92eb4ac

@influxdb.write_point(tag, record)

Currently, we use write_ponit for each record.
It may causes slowly insertion. I will check the performance with array version.

[repeatedly]

Sorry for the delay.
Could you try this PR code? > #12

[repeatedly]

@Korkd ping? If you okay, I will merge PR and release new version.

[Korkd]

Sorry to take so long to get back to you, haven't had a chance to look into this again until today. Good news though, it appears to have solved the problem. I'm no longer getting long delays between results!

[repeatedly]

Good :)