Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7.17 filebeat elasticsearch.slowlog ingested log entry has full json log as message #39935

Open
gigerdo opened this issue Jun 18, 2024 · 0 comments
Labels
Team:Monitoring Stack Monitoring team

Comments

@gigerdo
Copy link
Member

gigerdo commented Jun 18, 2024

When using filebeat to ingest ES slowlogs, the resulting document does not have the correct message. Instead the message contains the full JSON log entry:

image

Reproduction:

  • Version 7.17.21 for Elasticsearch and Filebeat
  • Setup filebeat to ingest slowlogs using the elasticsearch.slowlog module (For example by using the log+metrics feature in elastic cloud)

Here an example of a log entry as it appears in the ES log file:

{"type": "index_search_slowlog", "timestamp": "2024-06-13T11:54:45,125Z", "level": "WARN", "component": "i.s.s.query", "cluster.name": "13fd6fa94ab840c088e86a8cd8faa3b8", "node.name": "instance-0000000000", "message": "[kibana_sample_data_ecommerce][0]", "took": "5.5ms", "took_millis": "5", "total_hits": "0+ hits", "types": "[]", "stats": "[]", "search_type": "QUERY_THEN_FETCH", "total_shards": "1", "source": "{\"size\":500,\"query\":{\"bool\":{\"filter\":[{\"range\":{\"order_date\":{\"from\":\"2024-06-13T11:39:45.023Z\",\"to\":\"2024-06-13T11:54:45.023Z\",\"include_lower\":true,\"include_upper\":true,\"format\":\"strict_date_optional_time\",\"boost\":1.0}}}],\"adjust_pure_negative\":true,\"boost\":1.0}},\"version\":true,\"_source\":false,\"stored_fields\":\"*\",\"fields\":[{\"field\":\"*\",\"include_unmapped\":true},{\"field\":\"customer_birth_date\",\"format\":\"strict_date_optional_time\"},{\"field\":\"order_date\",\"format\":\"strict_date_optional_time\"},{\"field\":\"products.created_on\",\"format\":\"strict_date_optional_time\"}],\"script_fields\":{},\"sort\":[{\"order_date\":{\"order\":\"desc\",\"unmapped_type\":\"boolean\"}}],\"track_total_hits\":-1,\"highlight\":{\"pre_tags\":[\"@kibana-highlighted-field@\"],\"post_tags\":[\"@/kibana-highlighted-field@\"],\"fragment_size\":2147483647,\"fields\":{\"*\":{}}}}", "id": "b060ba95-ddea-4aa2-8a36-82a7a36b1564", "cluster.uuid": "UgI0rr5lSqeQtW98mENVxQ", "node.id": "BbJ5ZoU7QE2Q-U6fMAXgng" , "trace.id": "e738203f84d7190e649598ee3a54152d"  }
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 18, 2024
@jennypavlova jennypavlova added the Team:Monitoring Stack Monitoring team label Jun 18, 2024
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Monitoring Stack Monitoring team
Projects
None yet
Development

No branches or pull requests

2 participants