(v1.3.11) (v1.6.9) [types removal] Specifying types in bulk requests is deprecated. (with Elasticsearch 7.10.0)

[Theoooooo]

Bug Report

Describe the bug
I've used Fluentbit for a couple of months now and Elasticsearch keep changing there methods to insert data inside the service.
In future updates, the "Type" value inside the sended JSON will be removed but there is no option to remove this field with the default Elasticsearch output plugin.

To Reproduce

• Install the latest version for the OS CentOS through the official repo available at the official documentation (v1.3.11)
• Configure a output toward the service Elasticsearch (v7.10.0 in my case)
• Check inside the log file "depreciation.log" tons of warning message " [types removal] Specifying types in bulk requests is deprecated."

Expected behavior
I've wanted to have an option to remove the type field inside the plugin with an option like "use_type False". Overwise i need to configure a "MODIFY" block to remove this field for every messages that goes through the service to Elasticsearch.

Screenshots

Your Environment
CentOS7 1804
Fluentbit v1.3.11
Elasticsearch 7.10.0

Additional context
Exemple output configuration :

[OUTPUT]
    Name es
    Match tronce_syslog_*
    Host ES_HOST
    Port 9200
    HTTP_User USERNAME
    HTTP_Passwd PASSWORD
    Logstash_Format True
    Logstash_Prefix tronce_syslog
    Logstash_DateFormat %Y-%m-%d

Warning logs :

[2020-12-16T17:14:57,034][DEPRECATION][o.e.d.a.b.BulkRequestParser] [HOSTNAME] [types removal] Specifying types in bulk requests is deprecated.
[2020-12-16T17:14:57,035][DEPRECATION][o.e.d.a.b.BulkRequestParser] [HOSTNAME] [types removal] Specifying types in bulk requests is deprecated.
[2020-12-16T17:14:57,039][DEPRECATION][o.e.d.a.b.BulkRequestParser] [HOSTNAME] [types removal] Specifying types in bulk requests is deprecated.
[2020-12-16T17:14:57,040][DEPRECATION][o.e.d.a.b.BulkRequestParser] [HOSTNAME] [types removal] Specifying types in bulk requests is deprecated.
[2020-12-16T17:14:57,042][DEPRECATION][o.e.d.a.b.BulkRequestParser] [HOSTNAME] [types removal] Specifying types in bulk requests is deprecated.

And this is just a small exemple. I've got this error for every request Fluentbit does to Elasticsearch. As you guess, the log file grow up in size pretty quickly and i haven't find any option inside the official documentation about this issue

I saw this issue was fixed to fluend but not for fluentbit :/

[Platou]

Any updates on this? @Theoooooo @edsiper

[Theoooooo]

Welp, i've been struggling to test if this was fixed in the latest version because of the issue i've posted a few weeks ago #2886 (sorry this issue is quiet lengthy)
From this point, this issue is preventing me to update to the lastest because this routine wildcard issue is sending data at every output i've configured.

I didn't have the time to properly do testing procedure to see what's going on regarding this wildcard routine issue.

But if someone can walk into this issue and check if my statements on this issue are correct, that would be awesome. I can give you more precisions about this issue if it's not clear enough for you or someone else.

[nokute78]

Note:

The elasticsearch documentation
https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html

Same Issue of fluentd
uken/fluent-plugin-elasticsearch#785
uken/fluent-plugin-elasticsearch#766

The patch of fluentd
uken/fluent-plugin-elasticsearch#774

[nokute78]

Thank you for reporting issue.
I sent a patch #3140. Could you check it ?

[Theoooooo]

Thank you for reporting issue.
I sent a patch #3140. Could you check it ?

Thanks for the update on this issue,

As i said above in my last comment, i can't upgrade because of the issue #2886. Without this getting fixed, i can't test any changes made.
(This issue is quiet lengthy, if you need informations about it, i'm free to gave you what you need)