Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple nested fields are missing from generated yml #2235

Open
P1llus opened this issue Jul 17, 2023 · 2 comments
Open

Multiple nested fields are missing from generated yml #2235

P1llus opened this issue Jul 17, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@P1llus
Copy link
Member

P1llus commented Jul 17, 2023

Dynamic ECS template tests and even manual ECS mappings for Elastic integrations are failing due to muliple fields missing from nested variations in the generated yml files (both nested and flat).

For example process.user.name is there, but not process.user.full_name, there seems to be many different variations are missing which is most likely pointing towards a bigger issue somewhere?

A few fields as an example that are missing:

image

@P1llus P1llus added the bug Something isn't working label Jul 17, 2023
@ebeahan
Copy link
Member

ebeahan commented Jul 17, 2023

TLDR: This behavior is intended, but the ECS docs need corrected for process.user.* and process.group.* to properly detail which nested fields are expected.

The Linux Event Model RFC as originally proposed would have introduced an excessive (at least in the 100s) amount of unused nested fields under process.* and process.parent.*. To avoid having to reinvent how the ECS generator handles field reuses, a global subset filter (implemented in #1847) limits field reuses to the ones explicitly used in the Linux Event Model/Session Viewer.

A schema attribute was added (short_override) for certain field reuses to customize their description and list which nested fields are explicitly in the description, like here: https://github.com/elastic/ecs/blob/main/schemas/process.yml#L58. The reuse descriptions for process.user.*, process.group.*, process.parent.user.*, etc. need updated to note which fields are reused.

Dynamic ECS template tests and even manual ECS mappings for Elastic integrations are failing due to muliple fields missing from nested variations in the generated yml files (both nested and flat).

If there are integrations that do need a field, like process.user.full_name, these fields can be added to the config and will appear in the schema: https://github.com/elastic/ecs/blob/main/schemas/subsets/main.yml#L179.

@mjwolf
Copy link
Contributor

mjwolf commented Apr 29, 2024

I've created this PR to add process.group, which is related to this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants