Audit logs for AutoPutMapping don't include index name

[tvernum]

I haven't tracked this down, but I noticed it in 7.11.2

I issued a bulk request that triggered auto index creation, and the indices:admin/mapping/auto_put audit entries don't include the index name.

{
  "type": "audit",
  "timestamp": "2021-06-07T16:03:26,680+1000",
  "node.id": "i1xZMYXlQgymz97Hgd8Omw",
  "event.type": "transport",
  "event.action": "access_granted",
  "authentication.type": "REALM",
  "user.name": "test",
  "user.realm": "default_native",
  "user.roles": [
    "test"
  ],
  "origin.type": "rest",
  "origin.address": "[::1]:52142",
  "request.id": "UIUzvuJ0Rs2xgDLAz3pMmg",
  "action": "indices:admin/mapping/auto_put",
  "request.name": "PutMappingRequest"
}
{
  "type": "audit",
  "timestamp": "2021-06-07T16:03:26,681+1000",
  "node.id": "i1xZMYXlQgymz97Hgd8Omw",
  "event.type": "transport",
  "event.action": "access_granted",
  "authentication.type": "REALM",
  "user.name": "test",
  "user.realm": "default_native",
  "user.roles": [
    "test"
  ],
  "origin.type": "rest",
  "origin.address": "[::1]:52142",
  "request.id": "UIUzvuJ0Rs2xgDLAz3pMmg",
  "action": "indices:admin/mapping/auto_put",
  "request.name": "PutMappingRequest"
}

[elasticmachine]

Pinging @elastic/es-security (Team:Security)

[tvernum]

For context,

An AutoPutMapping action is triggered when an ingest to an index would cause the index's mapping to change.

This happens out of the box with Elasticsearch because we ship with dynamic mapping enabled.

That means you can install a brand new ES node with a config like this:

node.name: node01

xpack.license.self_generated.type: trial

xpack.security.enabled: true
xpack.security.audit.enabled: true

And then write to a new index like this:

PUT /index-1/_doc/1
{
  "field": "value"
}

and the index will be automatically created, including a default mapping.

That will generate an audit log roughly like this:

• {"type":"audit", "timestamp":"2021-11-01T14:09:27,753+1100", "node.id":"KSFmGXsLTuyOkypUndKw6A", "event.type":"transport", "event.action":"access_granted", "authentication.type":"REALM", "user.name":"elastic", "user.realm":"reserved", "user.roles":["superuser"], "origin.type":"rest", "origin.address":"[::1]:64276", "request.id":"19hZ78lERL6rKcyyVt7-HQ", "action":"indices:data/write/index", "request.name":"IndexRequest", "indices":["index-1"]}
• {"type":"audit", "timestamp":"2021-11-01T14:09:27,754+1100", "node.id":"KSFmGXsLTuyOkypUndKw6A", "event.type":"transport", "event.action":"access_granted", "authentication.type":"REALM", "user.name":"elastic", "user.realm":"reserved", "user.roles":["superuser"], "origin.type":"rest", "origin.address":"[::1]:64276", "request.id":"19hZ78lERL6rKcyyVt7-HQ", "action":"indices:data/write/bulk", "request.name":"BulkRequest"}
• {"type":"audit", "timestamp":"2021-11-01T14:09:27,757+1100", "node.id":"KSFmGXsLTuyOkypUndKw6A", "event.type":"transport", "event.action":"access_granted", "authentication.type":"REALM", "user.name":"elastic", "user.realm":"reserved", "user.roles":["superuser"], "origin.type":"rest", "origin.address":"[::1]:64276", "request.id":"19hZ78lERL6rKcyyVt7-HQ", "action":"indices:admin/auto_create", "request.name":"CreateIndexRequest", "indices":["index-1"]}
• {"type":"audit", "timestamp":"2021-11-01T14:09:28,417+1100", "node.id":"KSFmGXsLTuyOkypUndKw6A", "event.type":"transport", "event.action":"access_granted", "authentication.type":"REALM", "user.name":"elastic", "user.realm":"reserved", "user.roles":["superuser"], "origin.type":"rest", "origin.address":"[::1]:64276", "request.id":"19hZ78lERL6rKcyyVt7-HQ", "action":"indices:data/write/bulk[s]", "request.name":"BulkShardRequest", "indices":["index-1"]}
• {"type":"audit", "timestamp":"2021-11-01T14:09:28,421+1100", "node.id":"KSFmGXsLTuyOkypUndKw6A", "event.type":"transport", "event.action":"access_granted", "authentication.type":"REALM", "user.name":"elastic", "user.realm":"reserved", "user.roles":["superuser"], "origin.type":"rest", "origin.address":"[::1]:64276", "request.id":"19hZ78lERL6rKcyyVt7-HQ", "action":"indices:data/write/index:op_type/index", "request.name":"BulkItemRequest", "indices":["index-1"]}
• {"type":"audit", "timestamp":"2021-11-01T14:09:28,423+1100", "node.id":"KSFmGXsLTuyOkypUndKw6A", "event.type":"transport", "event.action":"access_granted", "authentication.type":"REALM", "user.name":"elastic", "user.realm":"reserved", "user.roles":["superuser"], "origin.type":"rest", "origin.address":"[::1]:64276", "request.id":"19hZ78lERL6rKcyyVt7-HQ", "action":"indices:data/write/bulk[s][p]", "request.name":"BulkShardRequest", "indices":["index-1"]}
• {"type":"audit", "timestamp":"2021-11-01T14:09:28,447+1100", "node.id":"KSFmGXsLTuyOkypUndKw6A", "event.type":"transport", "event.action":"access_granted", "authentication.type":"REALM", "user.name":"elastic", "user.realm":"reserved", "user.roles":["superuser"], "origin.type":"rest", "origin.address":"[::1]:64276", "request.id":"19hZ78lERL6rKcyyVt7-HQ", "action":"indices:admin/mapping/auto_put", "request.name":"PutMappingRequest"}

Walking through those entries we get these actions:

1. indices:data/write/index - The incoming request "index a document"
2. indices:data/write/bulk - Internally to ES, we route all documents writes through the bulk action so that there is a single path for all types of ingestion
3. indices:admin/auto_create - Since the index doesn't exist we automatically create the index
4. indices:data/write/bulk[s] - Then we write a bulk entries (although in this case there is only 1) to the relevant shard
5. indices:data/write/index:op_type/index - Within the shard action we check each item to ensure the user is allowed to perform that action on the shard. This is an index action, with an operation type of index (as opposed to create or update)
6. indices:data/write/bulk[s][p] - Perform the bulk action on the primary shard
7. indices:admin/mapping/auto_put - Because thre is a new field in the document that doesn't exist in the mapping, and dynamic mapping is enabled for this index (and all others), we automatically put a new mapping.

And while the auto_create index audit log from step 3 has "indices":["index-1"] , the auto_put mapping audit log (step 7) does not. But it should because it is an action on a single index.

[justincr-elastic]

Thank you. Your detailed explanation helped, especially the examples.

I have one quick follow up question. Should step 2 indices:data/write/bulk have "indices":["index-1"] too? It seems like all of the examples have "indices":["index-1"] except 2 and 7.

[tvernum]

It would probably be helpful if BulkRequest contained indices. It's less of an issue because if actually does anything it will do so by triggering a BulkShardRequest which contains the index name.
Depending on how we tackle the issue, I think it would be reasonable to add to the audit log, but if there are particular obstacles then we could go without it.