Adding persistence event

[ricardoungureanu]

Change Summary

Adding Persistence events mappings:

• custom_schemas/custom_persistence.yml
• custom_subsets/elastic_endpoint/persistence/persistence.yaml

Sample values

Sample document:

{
    "@timestamp": "2023-01-27T11:10:34.467591Z",
    "Persistence": {
        "args": [
            "/usr/local/share/asdkajdkajsdlkjasldkjqe/update.exe",
            "-c",
            "0xdeadbeef"
        ],
        "keepalive": true,
        "name": "OSX_EVIL_DROPPER_2",
        "path": "/Users/ricardo/Library/LaunchAgents/co.eaf.test.plist",
        "runatload": true
    },
    "agent": {
        "id": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
        "type": "endpoint",
        "version": "8.7.0-SNAPSHOT"
    },
    "data_stream": {
        "dataset": "endpoint.events.file",
        "namespace": "default",
        "type": "logs"
    },
    "ecs": {
        "version": "1.11.0"
    },
    "elastic": {
        "agent": {
            "id": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
        }
    },
    "event": {
        "action": "launch_daemon",
        "category": [
            "file"
        ],
        "created": "2023-01-27T11:10:34.467591Z",
        "dataset": "endpoint.events.file",
        "id": "MxCkm/WHfzXze820+++++/vL",
        "kind": "event",
        "module": "endpoint",
        "sequence": 6681,
        "type": [
            "info"
        ]
    },
    "group": {
        "Ext": {
            "real": {
                "id": 20,
                "name": "staff"
            }
        },
        "id": 20,
        "name": "staff"
    },
    "host": {
        "architecture": "x86_64",
        "hostname": "dev.local",
        "id": "dabadaba-0000-0000-0000-000000000000",
        "ip": [
            "127.0.0.1",
            "::1",
            "fe80::1",
            "172.16.167.128",
            "fe80::1ca1:f634:4dcc:fa52",
            "fe80::98d8:7768:5ab6:9ed0",
            "fe80::cb7d:558b:936d:b09e",
            "fe80::ce81:b1c:bd2c:69e"
        ],
        "mac": [
            "00:0c:29:9c:f9:70"
        ],
        "name": "dev.local",
        "os": {
            "Ext": {
                "variant": "macOS"
            },
            "family": "macos",
            "full": "macOS 12.6",
            "kernel": "Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64",
            "name": "macOS",
            "platform": "macos",
            "type": "macos",
            "version": "12.6"
        }
    },
    "message": "Endpoint file event",
    "process": {
        "Ext": {
            "ancestry": [
                "YWFhYWFhYWEtYWFhYS1hYWFhLWFhYWEtYWFhYWFhYWFhYWFhLTU2ODkzLTE2NzQ3NDcyODQ=",
                "YWFhYWFhYWEtYWFhYS1hYWFhLWFhYWEtYWFhYWFhYWFhYWFhLTU2ODkyLTE2NzQ3NDcyODQ=",
                "YWFhYWFhYWEtYWFhYS1hYWFhLWFhYWEtYWFhYWFhYWFhYWFhLTg1OS0xNjc0NDkxNDM1",
                "YWFhYWFhYWEtYWFhYS1hYWFhLWFhYWEtYWFhYWFhYWFhYWFhLTEtMTY3NDQ5MDMzNg==",
                "YWFhYWFhYWEtYWFhYS1hYWFhLWFhYWEtYWFhYWFhYWFhYWFhLTAtMTY3NDQ5MDMzNg=="
            ]
        },
        "code_signature": {
            "exists": true,
            "signing_id": "org.python.python",
            "status": "No error.",
            "subject_name": "Developer ID Application: Ned Deily (DJ3H93M7VJ)",
            "team_id": "DJ3H93M7VJ",
            "trusted": true
        },
        "entity_id": "YWFhYWFhYWEtYWFhYS1hYWFhLWFhYWEtYWFhYWFhYWFhYWFhLTc5OTcxLTE2NzQ4MTc3MTA=",
        "executable": "/Library/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python",
        "name": "Python",
        "pid": 79971
    },
    "user": {
        "Ext": {
            "real": {
                "id": 501,
                "name": "ricardo"
            }
        },
        "id": 501,
        "name": "ricardo"
    }
}

Release Target

Q/A

For mapping changes:

• I ran make after making the schema changes, and committed all changes
• If these field(s) are "exception"-able, I made a companion PR to Kibana adding it (see Readme)
• If this is a metadata change, I also updated both transform destination schemas to match

For Transform changes:

• The new transform successfully starts in Kibana
• The corresponding transform destination schema was updated if necessary

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-01-30T17:14:45.661+0000

• Duration: 8 min 46 sec

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[kevinlog]

if this is just a new subset of fields in file events and not a new data_stream, we don't need a new top level custom_subset for persistence.

We should probably just have another section for Persistence under the existing file.yml file. Similar to this: https://github.com/elastic/endpoint-package/blob/main/custom_subsets/elastic_endpoint/file/file.yaml#L152

[kevinlog]

if we're not introducing a new data_stream, this shouldn't get created by make. It's being made here because there is a new top level custom_subset for persistence.

By addressing this comment: https://github.com/elastic/endpoint-package/pull/336/files#r1090819658 - it should clean this up with another make clean and make all

[kevinlog]

Checked it out and tried, LGTM.

I'm able to install the package and see the new mappings:

I can see the sample event streams in successfully and I can see the data:

[elasticmachine]

Package endpoint - 8.7.0 containing this change is available at https://epr.elastic.co/search?package=endpoint