GCP Audit Logs Integration v2.14.0 - Ingest Pipeline fails

[ar3diu]

Due to an error in a "Convert" processor, the ingest pipeline fails to properly parse and map the log fields:

For example, json.protoPayload.methodName should have been mapped to event.action but didn't happen when the following field was present:
error.message: 'private' is not an IP string literal.

This can be fixed by changing an existing Convert Processor from logs-gcp.audit-2.14.0 managed ingest pipeline to the following:

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[efd6]

@rediasec Can you provide a sanitised event.original from a failing event so that this can be investigated.

[ar3diu]

{"insertId":"7dc9850e-a52e-4ef2-930b-c5db435cdcb0","labels":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"system:redacted\" of ClusterRole \"system:redacted\" to User \"system:redacted\""},"logName":"projects/redacted/logs/cloudaudit.googleapis.com%2Factivity","operation":{"first":true,"id":"7dc9850e-a52e-4ef2-930b-c5db435cdcb0","last":true,"producer":"k8s.io"},"protoPayload":{"@type":"type.googleapis.com/google.cloud.audit.AuditLog","authenticationInfo":{"principalEmail":"system:redacted"},"authorizationInfo":[{"granted":true,"permission":"io.k8s.coordination.v1.leases.update","resource":"coordination.k8s.io/v1/namespaces/redacted/leases/redacted","resourceAttributes":{}}],"methodName":"io.k8s.coordination.v1.leases.update","requestMetadata":{"callerIp":"private","callerSuppliedUserAgent":"redacted/v0.0.0 (linux/amd64) kubernetes/$Format","destinationAttributes":{},"requestAttributes":{}},"resourceName":"coordination.k8s.io/v1/namespaces/redacted/leases/redacted","serviceName":"k8s.io","status":{}},"receiveTimestamp":"2022-11-08T05:59:58.298462161Z","resource":{"labels":{"cluster_name":"redacted","location":"redacted","project_id":"redacted"},"type":"k8s_cluster"},"timestamp":"2022-11-08T05:59:57.114227Z"}

[oliver-creed]

Any news on this one i see the same thing?

[ar3diu]

I confirm that I have the same issue with the latest version of the integration: 2.16.0

[leandrojmp]

Just got the same issue and following @anrsec suggestion I edited the convert processor.

The only difference is that I'm using this conditional:

ctx.json?.protoPayload?.requestMetadata?.callerIp != null && !['gce-internal-ip', 'private'].contains(ctx.json?.protoPayload?.requestMetadata?.callerIp)

From google documentation, it seems that the callerIp field can have, besides an IP address, the values gce-internal-ip or private.

The IP address of the caller. For a caller from the internet, this will be the public IPv4 or IPv6 address. For calls made from inside Google's internal production network from one GCP service to another, callerIp will be redacted to "private". For a caller from a Compute Engine VM with a external IP address, callerIp will be the VM's external IP address. For a caller from a Compute Engine VM without a external IP address, if the VM is in the same organization (or project) as the accessed resource, callerIp will be the VM's internal IPv4 address, otherwise callerIp will be redacted to "gce-internal-ip"

I can make a PR with this fix.

[leandrojmp]

Added the PR with the suggested change by @anrsec

ctx.json?.protoPayload?.requestMetadata?.callerIp != null && ctx.json?.protoPayload?.requestMetadata?.callerIp != "gce-internal-ip"  && ctx.json?.protoPayload?.requestMetadata?.callerIp != "private"