Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fleet] Add ?full option to get package info endpoint to return all package fields #144343

Merged
merged 13 commits into from Nov 8, 2022
Merged

[Fleet] Add ?full option to get package info endpoint to return all package fields #144343

merged 13 commits into from Nov 8, 2022

Conversation

hop-dev
Copy link
Contributor

@hop-dev hop-dev commented Nov 1, 2022 •
edited

Summary

Closes #143198.

Add ?full=true parameter to the get packager info endpoint. This param returns the package info from the parsed archive, meaning paramaters such as data_streams.x.elasticsearch.source_mode (which is not supported by the EPR API) will now be returned from the kibana API.

This new param is only used by the create/edit package policy pages as they need the detailed data stream information.

I have made it so that we only ever cache the "full" package info, not the info from the package registry. In the future I want to make it clearer which package info is full and which is not #144640.

Checklist

Delete any items that are not applicable to this PR.

@hop-dev hop-dev force-pushed the 143198-get-full-package-api branch 4 times, most recently from 943801f to 327f13b Compare November 4, 2022 17:09
@hop-dev hop-dev changed the title 143198 get full package api [Fleet] Add ?full option to get package info endpoint to return all package fields Nov 4, 2022
@hop-dev hop-dev marked this pull request as ready for review November 4, 2022 19:07
@hop-dev hop-dev requested a review from a team as a code owner November 4, 2022 19:07
@hop-dev hop-dev added release_note:enhancement Team:Fleet Team label for Observability Data Collection Fleet team v8.6.0 labels Nov 4, 2022
@hop-dev hop-dev self-assigned this Nov 4, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@juliaElastic
Copy link
Contributor

juliaElastic commented Nov 7, 2022 •
edited

Do we have any concerns on the performance of getting full package info and remove caching of package registry info?
EDIT: I saw you discuss that in the follow-up issue.

@juliaElastic
Copy link
Contributor

@elasticmachine merge upstream

@hop-dev
Copy link
Contributor Author

hop-dev commented Nov 7, 2022 •
edited

@juliaElastic I've been doing some performance testing today and added a testing script. For getting full package info vs getting from the package registry here is the comparison over the 131 non-experimental packages. I think the results are acceptable considering after the first call the results are cached. I am looking to see if there are any quick wins to be had:

Average EPM time (GET /pkg/version: 0.37458015267175565s
Average Archive time (/pkg/version?full): 1.003114503816794s

Top 3 worst response times for the ?full endpoint are:

  • ti_util 2.629s
  • AWS 2.425s
  • Oracle 1.983s

(Note: also looking into why the cached times are so variable, I was expecting them to be consistently quick)

Screenshot 2022-11-07 at 15 49 32

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
fleet 861.3KB 861.4KB +24.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
fleet 115.5KB 115.5KB -2.0B
Unknown metric groups

ESLint disabled in files

id before after diff
osquery 1 2 +1

ESLint disabled line counts

id before after diff
enterpriseSearch 19 21 +2
fleet 58 65 +7
osquery 108 113 +5
securitySolution 440 446 +6
total +20

Total ESLint disabled count

id before after diff
enterpriseSearch 20 22 +2
fleet 66 73 +7
osquery 109 115 +6
securitySolution 517 523 +6
total +21

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @hop-dev

juliaElastic
juliaElastic approved these changes Nov 8, 2022
View reviewed changes
Copy link
Contributor

@juliaElastic juliaElastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, haven't tested locally.

@hop-dev hop-dev merged commit 99f1d07 into elastic:main Nov 8, 2022
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Nov 8, 2022
@hop-dev hop-dev deleted the 143198-get-full-package-api branch November 8, 2022 09:24
@hop-dev hop-dev mentioned this pull request Nov 8, 2022
Merged
@MiriamAparicio MiriamAparicio mentioned this pull request Nov 9, 2022
Merged
MiriamAparicio added a commit that referenced this pull request Nov 9, 2022
@MiriamAparicio
[APM] Skip test after change in fleet url params (#144905)
df43ad4 
Skipped failing test after change in fleep url params
#144343

Test will be fixed during test plan
#144907
@MiriamAparicio
Copy link
Contributor

Hi @hop-dev, I think this change broke the add APM integration page in fleet.

Screen.Recording.2022-11-10.at.14.31.43.mov

@MiriamAparicio MiriamAparicio mentioned this pull request Nov 10, 2022
Closed
@juliaElastic juliaElastic mentioned this pull request Dec 8, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@juliaElastic juliaElastic juliaElastic approved these changes

Assignees

@hop-dev hop-dev

Labels
backport:skip This commit does not require backporting release_note:enhancement Team:Fleet Team label for Observability Data Collection Fleet team v8.6.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Fleet] Return manifest info from get package info endpoints
6 participants
@hop-dev @elasticmachine @juliaElastic @kibana-ci @MiriamAparicio @kibanamachine