Error ElasticSearch Bulk item failed index result created type error

[Valent1ne93]

When switching from ELK 7.12.1 to ELK 8.5.3, there is an error when sending logs with NLog 5 (NLog Version 5.1.0, NLog.Targets.ElasticSearch Version 7.7.0)

Error ElasticSearch: Bulk item failed: index:dev-000005 result:created type: error:

Can it be related to the fact that ElasticSearch 8 does not have such a concept as _type which is expected to see in the NLog response?

The ElasticSearch messages were written correctly and without errors
After analyzing the network traffic of the application we saw that like in the case of ELK 7 and ELK 8 the response code is 200 and the only difference is the absence of field _type in the response from ELK 8

If it is so, what can we do to fix it?

[snakefoot]

From the Wiki:

When connecting to Elastic-Server 8.0 make sure to use documentType="" to avoid unknown parameter error.

[Valent1ne93]

We have it in the nlog configuration. Without this parameter the message would not be written to the ELK and the following error would occur

# OriginalException: Elasticsearch.Net.ElasticsearchClientException: Request failed to execute. Call: Status code 400 from: POST /_bulk. ServerError: Type: illegal_argument_exception Reason: "Action/metadata line [1] contains an unknown parameter [_type]"
# Request:
<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
# Response:
<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
 Exception: Elasticsearch.Net.ElasticsearchClientException: Request failed to execute. Call: Status code 400 from: POST /_bulk. ServerError: Type: illegal_argument_exception Reason: "Action/metadata line [1] contains an unknown parameter [_type]"

[Valent1ne93]

Here is an example of our configuration

<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      autoReload="true"
      internalLogLevel="Info"
      internalLogFile="nlog.log">

  <extensions>
    <add assembly="NLog.Web.AspNetCore"/>
    <add assembly="NLog.Targets.ElasticSearch"/>
  </extensions>

  <!-- the targets to write to -->
  <targets async="true">
    <target name="elastic" xsi:type="BufferingWrapper" flushTimeout="5000">       
      <target xsi:type="ElasticSearch"
         includeAllProperties="true"
         uri="http://elasticsearch:9200"
         requireAuth="true"
         index="dev"
         documentType=""
         username="username"
         password="password">
        <field name="AppName" layout="App-dev"/>
        <field name="Logger" layout="${logger}"/>
        <field name="CorrelationId" layout="${aspnet-request-headers:HeaderNames=X-Correlation-Id}"/>
        <field name="RequestUrl" layout="${aspnet-request-url}"/>
        <field name="MvcAction" layout="${aspnet-mvc-action}"/>
        <field name="MvcController" layout="${aspnet-mvc-controller}"/>
        <field name="mdlc" layoutType="System.Object">
          <layout xsi:type="JsonLayout" includeMdlc="true" />
        </field>
      </target>
    </target>

    <target name="file" xsi:type="File" fileName="${basedir}/logs/log-${shortdate}.log"
               layout="${longdate}|${logger}|${uppercase:${level}}|${message}|${exception:format=tostring}" />
  </targets>
  <!-- rules to map from logger name to target -->
  <rules>
    <logger name="Microsoft.*" maxlevel="Debug" final="true" /> <!-- BlackHole without writeTo -->
    <logger name="*" minlevel="Debug" writeTo="elastic, file" />
  </rules>
</nlog>

[hellraiser-rnd]

Link below is a code section that generates current error

NLog.Targets.ElasticSearch/src/NLog.Targets.ElasticSearch/ElasticSearchTarget.cs

Line 386 in 2fb9a27

foreach (var itemWithError in result.ItemsWithErrors)

It looks like wrong interpretation of CREATED status as error

[snakefoot]

Yes it would be nice to reverse the attempt to parse the publish reply, so it was more fire and forget. Consider to revert: #149

[snakefoot]

Could you try reverting to the previous release NLog.Targets.ElasticSearch 7.6.0 and report if it fixes the error ?

[mergandevinsander]

This issue related to #156

[smriti2901]

I have the same issue where I have NLog.Targets.ElasticSearch version 7.7.0. My Nlog configuration does have documentType = "" set. I upgraded my NEST library to version 7.17.5 and added ELASTIC_CLIENT_APIVERSIONING=true so I am no longer getting the below error -

2023-09-05 13:33:16.4864 Error ElasticSearch: Server error: ServerError: 400Type: illegal_argument_exception Reason: "Action/metadata line [1] contains an unknown parameter [_type]

But I get this error now -

2023-09-05 13:43:15.4303 Error ElasticSearch: Bulk item failed: index:.ds-logs-myindex-2023.09.03-000143 result:created type: error:

[snakefoot]

ElasticSearch: Server error: ServerError: 400Type: illegal_argument_exception Reason: "Action/metadata line [1] contains an unknown parameter [_type]

When connecting to Elastic-Server 8.0 make sure to use documentType="" to avoid unknown parameter error. See also: https://github.com/markmcdowell/NLog.Targets.ElasticSearch/wiki

[smriti2901]

@snakefoot Do you know how to resolve this error?

Error ElasticSearch: Bulk item failed: index:.ds-logs-myindex-2023.09.03-000143 result:created type: error:

[snakefoot]

No idea why it "fails" with status result: created.

Maybe relevant: #164 (comment) about setting environment-variable ELASTIC_CLIENT_APIVERSIONING to true for Elastic-server. See also: https://www.elastic.co/guide/en/elasticsearch/reference/current/rest-api-compatibility.html

[matthias-jauernig]

So... what's the status here? Half year later we still see those errors with NLog 5.2.4 and NLog.Targets.ElasticSearch 7.7.0 in the internal NLog log file.
Error ElasticSearch: Bulk item failed: index:my_index result:created type: error:

[Mehdi-MAMOU]

Hi,

I have the same error with : ELK 8.7.1 and NLOG Elastic search 7.7.0
Here is my configuration :

<extensions>
	<add assembly="NLog.Targets.ElasticSearch"/>
</extensions>

<targets async="true">
	<target xsi:type="ElasticSearch" name="elastic"
			index="adb2ccustomerlogs"
			opCodeCreate="true"
			cloudId="fc-elastic-XXXXXXXX"
			includeAllProperties="true"
			requireAuth="true"
			apiKey="XXXXX"
			apiKeyId="XXXXX-"
			documentType=""
			layout="${longdate}|${uppercase:${level}}|${message} ${exception:format=tostring}"
			pipeline="adb2ccustomer">
		<field name="fields.app" layoutType="System.String" layout="adb2ccustomer"/>
		<field name="fields.env" layoutType="System.String" layout="prod"/>
	</target>
</targets>

<rules>
	<logger name="*" minlevel="Info" writeTo="elastic"/>
</rules>

When i pass the internal nlog level to "Trace", i have more details on the error :
2023-10-27 14:15:57.5022 Error ElasticSearch: Bulk item failed: index:.ds-XXXXXXXXXX result:created type: error:
2023-10-27 14:15:57.5178 Trace ElasticSearch: Send Log DebugInfo=Invalid NEST response built from a successful (200) low level call on POST: /_bulk
Invalid Bulk items:
operation[0]: create returned 201 _index: .ds-XXXXXXXXXX: _id: 2h99cYsB0HQc2aDd38EX _version: 1 error:
Audit trail of this API call:

• [1] HealthyResponse: Node: https://XXXXXXXXXX.XXXXXXXX.azure.elastic-cloud.com/ Took: 00:00:01.0673700
  Request:
  <Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
  Response:
  <Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>

Can someone help me ?

[ridvandev]

Could you try reverting to the previous release NLog.Targets.ElasticSearch 7.6.0 and report if it fixes the error ?

I can verify downgrading to this version fixes this error.

[snakefoot]

@ridvandev I can verify downgrading to this version fixes this error.

Thank you for the confirmation.

@markmcdowell Would you be interested in a pull-request that reverts #149 and then release a new version 7.8.0 ?

Maybe followed by updating to latest ElasticSearch and change default-parameters to support ElasticSearch v8 by default and release version 8.0 ?

[markmcdowell]

Sounds like a good idea