EntityAnalytics AD: add support for ECS entity fields by chemamartinez · Pull Request #18337 · elastic/integrations

chemamartinez · 2026-04-10T14:49:52Z

Proposed commit message

Maps lastLogonTimestamp, UAC flags, and DN-valued LDAP attributes
(directReports, managedObjects) to user.entity.* and host.entity.* fields:

- user.entity.lifecycle.last_activity from lastLogonTimestamp.
- user.entity.attributes.mfa_enabled set to true when the SMARTCARD_REQUIRED
  flag is present in userAccountControl. No additional permissions or
  configuration required, as userAccountControl is always fetched.
- user.entity.relationships.supervises ({id, name, domain}) from the
  directReports attribute. Requires adding directReports to the User Attributes
  setting. Note: specifying any value in User Attributes switches the LDAP
  request from "all attributes" to an explicit list, so all previously implicit
  attributes must be added manually.
- user.entity.relationships.administers ({host.id, host.name, user.domain})
  from the managedObjects attribute. Requires adding managedObjects to the User
  Attributes setting, with the same caveat as above.
- host.entity.lifecycle.last_activity from lastLogonTimestamp.
- host.entity.attributes.managed set to true when WORKSTATION_TRUST_ACCOUNT
  or SERVER_TRUST_ACCOUNT is present in userAccountControl.
- host.entity.relationships.supervises ({id, name, domain}) from the
  directReports attribute.
- host.entity.relationships.administers ({host.id, host.name, user.domain})
  from the managedObjects attribute.

DN values in directReports and managedObjects are parsed into structured
objects extracting the CN component as name, the full DN as id, and the
DC components joined as a dotted domain string.

Test samples are AI-generated logs with random data.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

Related issues

Closes Entity Analytics Active Directory: add support for entity fields #18288

github-actions · 2026-04-10T14:52:06Z

Vale Linting Results

Summary: 2 warnings found

⚠️ Warnings (2)

File	Line	Rule	Message
packages/entityanalytics_ad/docs/README.md	350	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/entityanalytics_ad/docs/README.md	351	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.

The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.

elastic-vault-github-plugin-prod · 2026-04-10T15:13:21Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

macroscopeapp · 2026-04-17T13:45:54Z

+          }
+          return result;
+        }
+        def buildRel(def dns) {


🟢 Low ingest_pipeline/device.yml:286

In the buildRel function, when a DN lacks a CN= or DC= component, the corresponding name or domain entry is skipped entirely while id is still added. This produces arrays of mismatched lengths — for example, 3 IDs but only 2 names — making it impossible for consumers to correlate which name belongs to which ID by index. Consider restructuring to emit parallel arrays where each position corresponds across fields, such as an array of objects where each object contains the matched id, name, and domain for a single DN.

🤖 Copy this AI Prompt to have your agent fix this:

In file packages/entityanalytics_ad/data_stream/entity/elasticsearch/ingest_pipeline/device.yml around line 286: In the `buildRel` function, when a DN lacks a `CN=` or `DC=` component, the corresponding `name` or `domain` entry is skipped entirely while `id` is still added. This produces arrays of mismatched lengths — for example, 3 IDs but only 2 names — making it impossible for consumers to correlate which name belongs to which ID by index. Consider restructuring to emit parallel arrays where each position corresponds across fields, such as an array of objects where each object contains the matched `id`, `name`, and `domain` for a single DN.

We're aware of this limitation, but there are a few reasons why we've opted to keep flat parallel arrays rather than restructuring to nested objects:

Nested mappings are not an option as ES|QL doesn't support nested field types (see Elasticsearch docs).

Index correlation is already broken at the storage layer — since the sub-fields (id, name, domain) are mapped as flat keyword arrays under a group, Elasticsearch provides no positional guarantee between them regardless of how the script emits the data.

Consumers of these fields only need the full set of values for each attribute independently (e.g. all related IDs, all related names). Correlating a specific name to a specific ID within the same relationship group is not a requirement we need to support right now.

Given these constraints, emitting a flat list of each value without cross-field correlation is the right trade-off for us.

elasticmachine · 2026-04-23T16:03:30Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

chemamartinez · 2026-04-23T16:29:22Z

Moved to draft again because of an ongoing discussion about subfields in relationships fields.

uri-weisman · 2026-04-29T07:42:43Z

+                  type: keyword
+                - name: domain
+                  type: keyword
+            - name: administered_by


Entity store already uses administers and not administered_by, so we'll stick with that (that was one of Andrew's suggestions).

elasticmachine · 2026-04-30T16:03:24Z

💚 Build Succeeded

Buildkite Build
Commit: 47775a9

History

💚 Build #42134 succeeded e5c20e3
💚 Build #41657 succeeded e828235
💚 Build #41230 succeeded 8c016cd

cc @chemamartinez

efd6

Can you update the proposed commit message to more accurately reflect the change?

efd6 · 2026-05-04T05:31:46Z

+          fields:
+            - name: last_activity
+              type: date
+        - name: relationships


This is not yet defined in ECS. Is there an issue or PR for it?

There is an RFC at the moment: elastic/ecs#2598.

Once the RFC is merged, they are going to add to ECS the missing fields.

efd6 · 2026-05-04T05:33:34Z

Suggest bumping the ECS version to v9.3.0, then we have entity.attributes and entity.lifecycle defined.

Even though we bump ECS version, entity.attributes and entity.lifecycle are defined right now as an object without any subfield. This is not the desired final form of these fields, where fields inside should be defined as well.

This in conjunction with #18337 (comment) makes me reluctant to approve yet.

I understand and agree it is not the ideal scenario. Same happens with #18525 and #18621.

The RFC has been merged already and the next step is to push the decided entity schema into ECS.

chemamartinez added 6 commits April 10, 2026 10:56

Add mfa_enabled field support

a270b08

Add managed field support

6f52662

Add last_activity field support

1cf05a6

Add relationships fields support

9b1f8bf

Add test cases for new entity fields

5ec6d7d

Update changelog and bump version

d6f2435

chemamartinez self-assigned this Apr 10, 2026

chemamartinez added enhancement New feature or request Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] Integration:entityanalytics_ad Active Directory Entity Analytics labels Apr 10, 2026

Update changelog link

8c016cd

macroscopeapp Bot reviewed Apr 10, 2026

View reviewed changes

Comment thread packages/entityanalytics_ad/data_stream/entity/elasticsearch/ingest_pipeline/device.yml

andrewkroh added the documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. label Apr 10, 2026

Remove last_logon_timestamp if date processor fails

e2608f0

macroscopeapp Bot reviewed Apr 17, 2026

View reviewed changes

Avoid DN to be split by escaped commas

e828235

chemamartinez marked this pull request as ready for review April 23, 2026 16:03

chemamartinez requested a review from a team as a code owner April 23, 2026 16:03

chemamartinez marked this pull request as draft April 23, 2026 16:28

uri-weisman reviewed Apr 29, 2026

View reviewed changes

Update relationships fields according to new specs

e5c20e3

macroscopeapp Bot reviewed Apr 30, 2026

View reviewed changes

Comment thread packages/entityanalytics_ad/data_stream/entity/elasticsearch/ingest_pipeline/user.yml

Comment thread packages/entityanalytics_ad/data_stream/entity/elasticsearch/ingest_pipeline/device.yml

Add review suggestion about removing last_logon_timestamp on failure

6cf4deb

chemamartinez marked this pull request as ready for review April 30, 2026 15:34

macroscopeapp Bot reviewed Apr 30, 2026

View reviewed changes

Comment thread packages/entityanalytics_ad/data_stream/entity/elasticsearch/ingest_pipeline/user.yml

Avoid DN to be split by escaped commas in user pipeline

47775a9

efd6 reviewed May 4, 2026

View reviewed changes

chemamartinez requested a review from efd6 May 5, 2026 08:01

Conversation

chemamartinez commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

Related issues

Uh oh!

github-actions Bot commented Apr 10, 2026

Vale Linting Results

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod Bot commented Apr 10, 2026

🚀 Benchmarks report

Uh oh!

Uh oh!

macroscopeapp Bot Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Apr 23, 2026

Uh oh!

chemamartinez commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Apr 30, 2026

💚 Build Succeeded

History

Uh oh!

efd6 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

chemamartinez commented Apr 10, 2026 •

edited

Loading

chemamartinez commented Apr 23, 2026 •

edited

Loading