Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Logs Explorer] Field statistics don't return any data #178970

Closed
ablnk opened this issue Mar 19, 2024 · 7 comments
Closed

[Logs Explorer] Field statistics don't return any data #178970

ablnk opened this issue Mar 19, 2024 · 7 comments
Assignees
Labels
bug Fixes for quality problems that affect the customer experience Feature:Discover Discover Application Feature:LogsExplorer Logs Explorer feature Project:OneDiscover Enrich Discover with contextual awareness / Merge with Logs Explorer Team:DataDiscovery Discover App Team (Document Explorer, Saved Search, Surrounding documents, Graph) Team:obs-ux-logs Observability Logs User Experience Team
Projects

Comments

@ablnk
Copy link

ablnk commented Mar 19, 2024

Version:
Serverless project v 8.14.0

Description:
Field statistics in Logs Explorer don't return any data regardless of the set time interval (e.g. for 1 minute or for 7 days of log data).

Steps to reproduce:

  1. Go to Discover - Logs Explorer.
  2. Click on Field statistics tab and wait for data to load.

Expected behavior:
Field statistics must be returned.

Screenshots:
fieldstats

@ablnk ablnk added bug Fixes for quality problems that affect the customer experience Team:obs-ux-logs Observability Logs User Experience Team labels Mar 19, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

@tonyghiani tonyghiani self-assigned this Mar 20, 2024
@tonyghiani
Copy link
Contributor

I'm taking a look at this issue, as this behaviour seems to occur since the fields added to the grid are Smart Fields (a combination of many fields), which are so not mapped directly to any field and listed in the Field statistics.

@qn895 qn895 self-assigned this Mar 21, 2024
@paulb-elastic
Copy link
Contributor

@tonyghiani do you know if this only happen on serverless, or is it a general issue?

@tonyghiani
Copy link
Contributor

do you know if this only happen on serverless, or is it a general issue?

Hi @paulb-elastic, we know the cause of this issue, which came as a consequence of adding support for the smart fields columns in Logs Explorer.

The issue only affects Logs Explorer in both stateful/serverless deployment and only when using the default smart fields.
As we'll very soon work on pushing down the support for smart fields into the Discover codebase, the plan is for smart fields to be a first-class citizen in Discover, including the related support for field statistics which should fix this issue.

@paulb-elastic
Copy link
Contributor

Many thanks @tonyghiani for the detailed analysis

@davismcphee davismcphee added Project:OneDiscover Enrich Discover with contextual awareness / Merge with Logs Explorer Feature:Discover Discover Application Team:DataDiscovery Discover App Team (Document Explorer, Saved Search, Surrounding documents, Graph) Feature:LogsExplorer Logs Explorer feature labels Apr 24, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@kibanamachine kibanamachine added this to Inbox in Discover Apr 24, 2024
@Kerry350 Kerry350 self-assigned this May 21, 2024
Kerry350 added a commit that referenced this issue Jun 3, 2024
…tab when using Smart Fields (#184172)

## Summary

This addresses the bugs listed in
#181674
(#181621 and
#178970).

Smart Fields are compound fields, as such there is no "real" backing
field for them. In these scenarios where we can't use them directly we
fall back to the `fallbackFields` configured for them.

[@amyjtechwriter please could you check the
copy](#181621 (comment)).
I think `derived from...` would have sounded better, but it felt at odds
with the scenario where you might specifically pick a field that is also
a fallback field for a smart field.

## Reviewer notes

- There were quite a lot of cross cutting concerns here between the
`unified-field-list`, `unified-data-table`, core Discover code,
`discover-utils`, and the `data_vizualizer` plugin for the field
statistics table. Where I'd have liked to potentially put more things in
the discover-utils it wasn't possible due to circular project
references.

- I've not added any functional tests for now as we'd be adding them to
the Observability Logs Explorer which is going to be deprecated in due
course. This ties in with part three of
#181674.

- A `renderFieldName` prop has been added to the field statistics table,
this seemed to be the least intrusive way forward rather than a new /
comprehensive cell rendering change (similar to the external cell
renderers).

- I've used the original
[`fallbackFields`](https://github.com/elastic/kibana/blob/main/src/plugins/discover/common/data_types/logs/constants.ts#L35)
configurations but the [virtual column components actually render a bit
extra](https://github.com/elastic/kibana/blob/main/packages/kbn-discover-utils/src/utils/get_message_field_with_fallbacks.ts#L13).
Am I missing anything with regards to potentially extending the
`fallbackFields` definitions?

## Screenshots

<img width="1253" alt="Screenshot 2024-05-23 at 20 25 57"
src="https://github.com/elastic/kibana/assets/471693/c2982f45-c8fa-4caa-935d-7007745ae2ce">

<img width="1477" alt="Screenshot 2024-05-23 at 20 26 21"
src="https://github.com/elastic/kibana/assets/471693/3145fbd1-071e-42fd-a94d-beb15c29c739">

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
@gbamparop
Copy link
Contributor

Fixed by #184172

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Feature:Discover Discover Application Feature:LogsExplorer Logs Explorer feature Project:OneDiscover Enrich Discover with contextual awareness / Merge with Logs Explorer Team:DataDiscovery Discover App Team (Document Explorer, Saved Search, Surrounding documents, Graph) Team:obs-ux-logs Observability Logs User Experience Team
Projects
No open projects
Development

No branches or pull requests

8 participants