-
Notifications
You must be signed in to change notification settings - Fork 73
Description
Acknowledgements
- I have searched (https://github.com/utmstack/UTMStack/issues) for past instances of this issue
- I have verified that my UTMStack version is up-to-date
Describe the bug
LIST_CHART visualizations currently fetch full documents from OpenSearch, including extremely large fields such as event.original, winlog.event_data.*, and message.
This results in response payloads exceeding 100MB, causing the OpenSearch Java client (HttpClient5) to fail.
LIST_CHART is a hybrid visualization:
- It uses aggregations to build the hierarchical structure.
- It uses documents to populate the table.
Because documents are required, size > 0 must remain.
However, the _source must be restricted to avoid returning massive payloads.
Regression Issue
- Select this option if this issue appears to be a regression.
Expected Behavior
LIST_CHART visualizations should return lightweight documents containing only the fields required by the table, while still using aggregations to build the hierarchical structure.
The OpenSearch response should remain well below 100MB, allowing the Java client (HttpClient5) to process it without errors.
Current Behavior
LIST_CHART currently fetches full documents, including extremely large fields such as:
- event.original
- winlog.event_data.*
- message
- stacktrace
This results in OpenSearch responses exceeding 100–150MB, causing the client to fail with:
UtmVisualizationResource.run: ElasticsearchService.search: OpenSearch.search: entity content is too long [143527767] for the configured buffer limit [104857600]
Reproduction Steps
- Create or open a LIST_CHART visualization with deep aggregations
(e.g., name → severity → status → category → dataSource → timestamp). - Apply filters that match Windows Event Log data.
- Ensure the visualization requests documents (size > 0).
- Execute the request through the backend (not directly in OpenSearch Dashboards).
- Observe the backend error due to oversized _source payload.
Possible Solution
Implement _source filtering only for LIST_CHART, excluding all fields and re‑including only the fields required by the visualization.
Additional Information/Context
No response
UTMStack Version
11.2.3
Operating System and version
Ubuntu
Hypervisor and Version | Server Vendor and Model
Browser and version
Metadata
Metadata
Assignees
Labels
Type
Projects
Status