Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Synthetic source support for objects with nested unmapped fields #106825

Open
salvatore-campagna opened this issue Mar 27, 2024 · 1 comment
Open

Synthetic source support for objects with nested unmapped fields #106825

salvatore-campagna opened this issue Mar 27, 2024 · 1 comment
Assignees
@kkrik-es
Labels
>enhancement :StorageEngine/Logs You know, for Logs :StorageEngine/Mapping The storage related side of mappings Team:StorageEngine

Comments

@salvatore-campagna
Copy link
Contributor

salvatore-campagna commented Mar 27, 2024 •
edited

Description

Synthetic source is still missing support for the Object field type. Supporting it when enabled: false is already addresses by #106460 by having a stored field storing the whole json blob as is. Anyway, we need to support synthetic source reconstruction of the field also when enabled: true.

When that is the case it means we can have mapped or unmapped sub-fields which might be of any type and we are missing support for object fields including unmapped fields. We need to delegate to the corresponding nested mapper created by the dynamic mapping logic if the field was dynamically mapped or by storing the json value "as-is" if the mapping didn't happen because the number of dynamically mapped fields was already above the limit.
@salvatore-campagna salvatore-campagna self-assigned this Mar 27, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@salvatore-campagna salvatore-campagna changed the title Synthetic source support for objects when enabled: true Synthetic source support for objects with nested unmapped fields Mar 28, 2024
@kkrik-es kkrik-es added the :StorageEngine/Mapping The storage related side of mappings label Apr 10, 2024
@kkrik-es kkrik-es mentioned this issue Apr 17, 2024
Merged
This was referenced Apr 30, 2024
Merged
Merged
elasticsearchmachine pushed a commit that referenced this issue May 2, 2024
@kkrik-es
Skip setIgnoredValues when there are no ignored values (#108179)
2c2a85b 
The current logic iterates over all fields in every doc to check for
ignored fields, even when no fields were retrieved from _ignored_source.
The fix should address the performance regression we noticed, where
`setIgnoredValues` dominates cpu time:

<img width="1417" alt="Screenshot 2024-05-02 at 12 11 01"
src="https://github.com/elastic/elasticsearch/assets/131142368/66c5f944-21d4-41b1-955f-9ca434151cbd">

Related to #106825
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kkrik-es kkrik-es

Labels
>enhancement :StorageEngine/Logs You know, for Logs :StorageEngine/Mapping The storage related side of mappings Team:StorageEngine
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@elasticsearchmachine @salvatore-campagna @kkrik-es