Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DataView] Improve fields performance #167595

Closed
12 tasks done
kertal opened this issue Sep 29, 2023 · 2 comments
Closed
12 tasks done

[DataView] Improve fields performance #167595

kertal opened this issue Sep 29, 2023 · 2 comments
Assignees
@kertal
Labels
epic Feature:Data Views Data Views code and UI - index patterns before 8.0 impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. loe:x-large Extra Large Level of Effort Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.

Comments

@kertal
Copy link
Member

kertal commented Sep 29, 2023 •
edited by ninoslavmiskovic
Loading

DataViews are used all around Kibana, this issue aims to improve the performance by
a) Remove redundant calls to the fields caps API (currently this is done every time a DataView is being loaded)
b) Remove redundant fetching of DataViews (e.g. in Discover when switching data views)
c) Optimizing DataView field_caps requests to return data more quickly (by e.g. improved caching, excluding defined tiers)

Tasks
Beta Give feedback
  1. [DataViews] Add caching for the request of fields #169622
    Feature:Data Views Feature:ES|QL Team:DataDiscovery impact:high loe:large performance
    mattkime
  2. [Dashboard] Field-caps requests are made for unused default dataviews #162403
    :DataDiscovery/fix-it-week Team:DataDiscovery Team:Presentation bug impact:low triage_needed
    mattkime
  3. [Discover] Redundant data view fields request when switching data views #167221
    :DataDiscovery/fix-it-week Feature:Discover Team:DataDiscovery bug impact:high loe:small
    mattkime
  4. [DataViews] Provide functionality to exclude configured data tiers in Advanced Settings #167741
    Feature:Data Views Team:DataDiscovery enhancement impact:medium loe:needs-research performance
    mattkime
  5. [Discover][ES|QL] Improve changing of index pattern performance #169835
    Feature:Data Views Feature:Discover Feature:ES|QL Team:DataDiscovery impact:medium loe:small performance
    mattkime
  6. [DataView] Improve performance of field capabilities utils #172275
    Feature:Data Views Team:DataDiscovery impact:medium loe:small performance
    mattkime

DataViewLazy only loads fields as requested instead of the full field list up front.

DataViewLazy implementation
Beta Give feedback
  1. [DataViews] Create AbstractDataView class #167736
    Feature:Data Views Team:DataDiscovery impact:high loe:medium refactoring
    mattkime
  2. [DataViews] Implement DataViewLazy class based on AbstractDataView #167750
    Feature:Data Views Team:DataDiscovery impact:high loe:large
    mattkime
  3. [DataViews] Make use of DataViewLazy in DataViews API #176426
    Feature:Data Views Team:DataDiscovery impact:high loe:small
    mattkime
  4. [DataViews] Make use of DataView Lazy in SearchSource #175081
    Feature:Data Views Team:DataDiscovery impact:high loe:needs-research
    mattkime

Consumers of DataViewLazy
Beta Give feedback
  1. [DataViews] Evaluate switching ML alert rules to make use of DataViewLazy #173996
    :ml Feature:Data Views Team:DataDiscovery impact:medium loe:needs-research technical debt v8.16.0
    darnautov
  2. [Security Solution][Detection Engine] Detection rules should use dataViewLazy to reduce field caps data transfer #189113
    Feature:Detection Rules Team:Detection Engine performance
    vitaliidm
@botelastic botelastic bot added the needs-team Issues missing a team label label Sep 29, 2023
@kertal kertal added the Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. label Sep 29, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Sep 29, 2023
@kertal kertal added Feature:Data Views Data Views code and UI - index patterns before 8.0 needs-team Issues missing a team label labels Sep 29, 2023
@botelastic botelastic bot removed the needs-team Issues missing a team label label Sep 29, 2023
@kertal kertal changed the title [DataView] Reduce redundant field requests for better performance [DataView] Optimize field requests for better performance Oct 2, 2023
@kertal kertal changed the title [DataView] Optimize field requests for better performance [DataView] Improve fields performance Oct 2, 2023
@kertal kertal added impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. epic and removed Meta labels Nov 28, 2023
@mattkime mattkime mentioned this issue Mar 19, 2024
Closed
@kertal kertal added the loe:x-large Extra Large Level of Effort label Mar 21, 2024
@kertal kertal assigned kertal and unassigned mattkime Sep 25, 2024
@ninoslavmiskovic
Copy link
Contributor

Closing as completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kertal kertal

Labels
epic Feature:Data Views Data Views code and UI - index patterns before 8.0 impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. loe:x-large Extra Large Level of Effort Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@mattkime @kertal @elasticmachine @davismcphee @ninoslavmiskovic