Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Data] Add updated essql expression function #132332

Merged
merged 11 commits into from
May 26, 2022
Merged

Conversation

dokmic
Copy link
Contributor

@dokmic dokmic commented May 17, 2022

Summary

Resolves #129478.

Checklist

For maintainers

@dokmic dokmic added release_note:enhancement review Feature:Search Querying infrastructure in Kibana Feature:ExpressionLanguage Interpreter expression language (aka canvas pipeline) Team:AppServicesSv v8.3.0 labels May 17, 2022
@dokmic dokmic marked this pull request as ready for review May 17, 2022 16:11
@dokmic dokmic requested review from a team as code owners May 17, 2022 16:11
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app-services (Team:AppServicesSv)

@dokmic dokmic requested a review from ppisljar May 17, 2022 16:11

return lastValueFrom(
search.search<EssqlSearchStrategyRequest, EssqlSearchStrategyResponse>(req, {
strategy: ESSQL_SEARCH_STRATEGY,
Copy link
Contributor

@Dosant Dosant May 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So canvas now uses a new essql function that uses a new generic strategy from data plugin?
Compared to the new generic strategy, canvas ESSQL_SEARCH_STRATEGY did internal pagination through scroll till the end server-side.
If I understand correctly, then canvas now lost it now, correct?

If yes, then we need to add it back somewhere.

Options I was thinking of:

  1. Add it as an option to search strategy and do it internally server-side as it was before (bad because doesn't fit into async search and search polling idea until we support client-side search strategies)
  2. Handle it on the expression function level, maybe fit it into the partial results framework

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what did the scroll achieve that we don't with the new strategy ? canvas on the client received the list of all results. It will get the same now ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

canvas has the same behavior, seems they intended to support larger requests at some point but feature newer worked.

we will switch to the new strategy and we can think about supporting larger requests in the future.

Copy link
Member

@ppisljar ppisljar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

canvas essql search strategy can be deleted as well

@dokmic
Copy link
Contributor Author

dokmic commented May 19, 2022

canvas essql search strategy can be deleted as well

@ppisljar we cannot delete that yet, it is used in the canvas's esdocs function.

@flash1293
Copy link
Contributor

This seems to have the same issue as the canvas essql function - it doesn't respect the count if it's larger than 100. It works fine in dev tools:

GET _sql
{
  "query": """
  SELECT bytes FROM kibana_sample_data_logs
  """,
  "fetch_size": 1000
}

This gives me 1000 rows.

But

essql query="SELECT bytes FROM kibana_sample_data_logs" count=10000 
| render as="debug" 

will give me only 100 rows in Canvas

@dokmic dokmic force-pushed the feature/129478 branch 3 times, most recently from bd2bda1 to c863d0b Compare May 26, 2022 10:35
Copy link
Member

@ppisljar ppisljar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@crob611
Copy link
Contributor

crob611 commented May 26, 2022

@flash1293 that's a strange bug with count. It looks like it only happens when you are doing the render as debug though. If you just do straight render, you'll get a table paginated through the expected number of results.

Copy link
Contributor

@crob611 crob611 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Presentation changes look good to me. New function works as expected in Canvas.

@dokmic
Copy link
Contributor Author

dokmic commented May 26, 2022

@elasticmachine merge upstream

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
canvas 1128 1126 -2
data 511 513 +2
total -0

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
data 2874 2875 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
canvas 1.0MB 1.0MB -1.9KB
controls 426.4KB 426.4KB +15.0B
dashboard 425.4KB 425.4KB +15.0B
discover 514.8KB 514.8KB +15.0B
inputControlVis 77.6KB 77.6KB +15.0B
unifiedSearch 176.7KB 176.7KB +15.0B
total -1.8KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
data 415.9KB 420.7KB +4.8KB
securitySolution 249.4KB 249.4KB +15.0B
total +4.8KB
Unknown metric groups

API count

id before after diff
data 3489 3491 +2

ESLint disabled line counts

id before after diff
data 58 60 +2

References to deprecated APIs

id before after diff
canvas 67 64 -3
data 384 385 +1
total -2

Total ESLint disabled count

id before after diff
data 62 64 +2

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting Feature:ExpressionLanguage Interpreter expression language (aka canvas pipeline) Feature:Search Querying infrastructure in Kibana release_note:enhancement review v8.4.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[expressions] new sql expression function
7 participants