[network_traffic] Fix to index events with memcached stats responses

[chrisberkhout]

What does this PR do?

Fixes the Network Packet Capture integration to not drop events with memcached STATS response data, by using a flattened field to hold the stats data in a single object.

Details

The Network Packet Capture integration currently fails to index events with memcached STATS response data,
as described in issue #6230.

The stats data in the event is an array of objects with name and value keys:

{
  "memcache": {
    "response": {
      "stats": [
        {
          "name": "pid",
          "value": "1"
        },
        {
          "name": "uptime",
          "value": "183329"
        },
        ...
      ], ...
    }, ...
  }, ...
}

Many stat names are listed in the memcached documentation (93 general-purpose statistics, and over 100 others). However, actual responses may include fewer, or additional (undocumented) stats.

Since the purpose of the integration is to sniff network packets and dissect known protocols, rather than to support monitoring and operation of memcached clusters, I chose to index the stats data as a single flattened object, which makes the data readable although not easily queryable:

{
  "memcache": {
    "response": {
      "stats": {
        "pid": "1",
        "uptime": "183329",
        ...
      }, ...
    }, ...
  }, ...
}

Alternative representations could be more queryable, but at the cost of indexing many additional documents (nested), or mapping a large and changeable number of fields.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Related issues

• Closes [network_traffic] Event indexing failure in system test #6230

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-06-22T11:42:19.718+0000

• Duration: 65 min 42 sec

Test stats 🧪

Test	Results
Failed	0
Passed	154
Skipped	0
Total	154

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (1/1)	💚
Files	100.0% (2/2)	💚
Classes	100.0% (2/2)	💚
Methods	98.276% (57/58)	👎 -1.724
Lines	88.356% (129/146)	👎 -5.235
Conditionals	100.0% (0/0)	💚

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 5a550885796081ba1f37f713d569e00e04826ab4

History

• 💔 Build #30 failed 5a51ed8d9a276872c8bfe95df67176c57011fa94

[taylor-swanson]

LGTM

[elasticmachine]

Package network_traffic - 1.19.1 containing this change is available at https://epr.elastic.co/search?package=network_traffic