It sends events into Elasticsearch. It uses _bulk API to send events in batches.
If a network error occurs, the batch will infinitely try to be delivered to the random endpoint.
endpoints []string required
The list of elasticsearch endpoints in the following format: SCHEMA://HOST:PORT
username string
Username for HTTP Basic Authentication.
password string
Password for HTTP Basic Authentication.
api_key string
Base64-encoded token for authorization; if set, overrides username/password.
ca_cert string
Path or content of a PEM-encoded CA file.
index_format string default=file-d-%
It defines the pattern of elasticsearch index name. Use % character as a placeholder. Use index_values to define values for the replacement.
E.g. if index_format="my-index-%-%" and index_values="service,time" and event is {"service"="my-service"}
then index for that event will be my-index-my-service-2020-01-05. First % replaced with service field of the event and the second
replaced with current time(see time_format option)
index_values []string default=[@time]
A comma-separated list of event fields which will be used for replacement index_format.
There is a special field time which equals the current time. Use the time_format to define a time format.
E.g. [service, time]
time_format string default=2006-01-02
The time format pattern to use as value for the time placeholder.
Check out func Parse doc for details.
connection_timeout cfg.Duration default=5s
It defines how much time to wait for the connection.
workers_count cfg.Expression default=gomaxprocs*4
It defines how many workers will be instantiated to send batches.
batch_size cfg.Expression default=capacity/4
A maximum quantity of events to pack into one batch.
batch_size_bytes cfg.Expression default=0
A minimum size of events in a batch to send. If both batch_size and batch_size_bytes are set, they will work together.
batch_flush_timeout cfg.Duration default=200ms
After this timeout batch will be sent even if batch isn't full.
batch_op_type string default=index options=index|create
Operation type to be used in batch requests. It can be index or create. Default is index.
Check out _bulk API doc for details.
retry int default=10
Retries of insertion. If File.d cannot insert for this number of attempts, File.d will fall with non-zero exit code or skip message (see fatal_on_failed_insert).
fatal_on_failed_insert bool default=false
After an insert error, fall with a non-zero exit code or not Experimental feature
retention cfg.Duration default=1s
Retention milliseconds for retry to DB.
retention_exponentially_multiplier int default=2
Multiplier for exponential increase of retention between retries
Generated using insane-doc