Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[packages] migrate all plugins to packages #148130

Merged
merged 13 commits into from
Feb 9, 2023

Conversation

spalger
Copy link
Contributor

@spalger spalger commented Dec 27, 2022

Fixes #149344

This PR migrates all plugins to packages automatically. It does this using node scripts/lint_packages to automatically migrate kibana.json files to kibana.jsonc files. By doing this automatically we can simplify many build and testing procedures to only support packages, and not both "packages" and "synthetic packages" (basically pointers to plugins).

The majority of changes are in operations related code, so we'll be having operations review this before marking it ready for review. The vast majority of the code owners are simply pinged because we deleted all kibana.json files and replaced them with kibana.jsonc files, so we plan on leaving the PR ready-for-review for about 24 hours before merging (after feature freeze), assuming we don't have any blockers (especially from @elastic/kibana-core since there are a few core specific changes, though the majority were handled in #149370).

@spalger spalger force-pushed the implement/plugins-are-packages branch from 05b9748 to 6bb0b83 Compare December 28, 2022 00:01
@spalger spalger added the skip-ci label Jan 3, 2023
@spalger spalger force-pushed the implement/plugins-are-packages branch from d10d1c4 to edb4a7e Compare January 4, 2023 15:56
@spalger spalger added skip-ci and removed skip-ci labels Jan 4, 2023
@spalger spalger force-pushed the implement/plugins-are-packages branch from ed2e449 to 17555ec Compare January 5, 2023 06:17
@spalger spalger removed the skip-ci label Jan 5, 2023
@spalger spalger force-pushed the implement/plugins-are-packages branch 20 times, most recently from 2bcf1b3 to 687cefa Compare January 13, 2023 00:57
@spalger spalger force-pushed the implement/plugins-are-packages branch 3 times, most recently from 8b6215c to c3758da Compare January 13, 2023 23:00
@spalger spalger force-pushed the implement/plugins-are-packages branch from eadda8e to 436df62 Compare February 8, 2023 23:01
@spalger spalger enabled auto-merge (squash) February 8, 2023 23:01
@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] Threat Intelligence Tests #4 / Indicators Indicators timeline interactions should add entry in timeline when clicking in an indicator flyout overview block

Metrics [docs]

Adoption-tracked APIs

APIs whose adoption is being tracked (they set the tag @track-adoption in their JSDocs)

id before after diff
core 4 - -4

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/chart-expressions-common - 4 +4
@kbn/import-resolver 39 40 +1
@kbn/json-ast 28 35 +7
@kbn/test 225 227 +2
@kbn/ts-projects 26 25 -1
core 1030 - -1030
total -1017

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id before after diff
core 17 - -17

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
aiops 745.5KB 745.5KB -5.0B
apm 3.3MB 3.3MB -26.0B
canvas 1024.0KB 1024.0KB -18.0B
cases 394.1KB 394.1KB -2.0B
cloudChat 2.8KB 2.8KB -1.0B
cloudDataMigration 3.6KB 3.6KB -1.0B
cloudDefend 223.8KB 223.8KB -3.0B
cloudExperiments 40.5KB 40.5KB -1.0B
cloudFullStory 4.3KB 4.3KB -1.0B
cloudGainsight 2.1KB 2.1KB -1.0B
cloudSecurityPosture 142.5KB 142.5KB -3.0B
crossClusterReplication 166.5KB 166.5KB -1.0B
dataVisualizer 379.1KB 379.1KB -5.0B
enterpriseSearch 2.1MB 2.1MB -15.0B
fileUpload 852.2KB 852.2KB -1.0B
fleet 926.8KB 926.8KB -6.0B
graph 450.8KB 450.8KB -3.0B
grokdebugger 59.1KB 59.1KB -1.0B
indexLifecycleManagement 156.2KB 156.2KB -1.0B
indexManagement 522.3KB 522.3KB -1.0B
infra 1.3MB 1.3MB -19.0B
ingestPipelines 431.2KB 431.2KB -1.0B
kubernetesSecurity 39.6KB 39.6KB -1.0B
lens 1.3MB 1.3MB -15.0B
licenseManagement 62.9KB 62.9KB -1.0B
lists 152.4KB 152.4KB -1.0B
logstash 30.6KB 30.6KB -1.0B
maps 2.7MB 2.7MB -15.0B
ml 3.5MB 3.5MB -32.0B
monitoring 455.4KB 455.4KB -5.0B
observability 584.1KB 584.1KB -19.0B
osquery 1.1MB 1.1MB -10.0B
painlessLab 16.1KB 16.1KB -1.0B
profiling 256.7KB 256.7KB -1.0B
remoteClusters 84.9KB 84.9KB -1.0B
reporting 58.8KB 58.8KB -4.0B
rollup 131.2KB 131.2KB -1.0B
savedObjectsTagging 40.8KB 40.8KB -4.0B
searchprofiler 159.0KB 159.0KB -1.0B
security 557.3KB 557.3KB -21.0B
securitySolution 13.8MB 13.8MB +134.0B
sessionView 382.9KB 382.9KB -1.0B
snapshotRestore 267.4KB 267.4KB -1.0B
spaces 174.0KB 174.0KB -11.0B
stackAlerts 80.3KB 80.3KB -3.0B
stackConnectors 419.3KB 419.2KB -62.0B
synthetics 1.4MB 1.4MB -23.0B
threatIntelligence 52.0KB 52.0KB -3.0B
timelines 30.0KB 30.0KB -8.0B
transform 366.4KB 366.4KB -3.0B
triggersActionsUi 763.9KB 763.8KB -58.0B
upgradeAssistant 149.9KB 149.9KB -1.0B
ux 160.4KB 160.4KB -3.0B
watcher 271.6KB 271.6KB -2.0B
total -295.0B

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
@kbn/core - 1 +1
@kbn/repo-packages 3 2 -1
total -0

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
actions 15.6KB 15.6KB -1.0B
aiops 5.3KB 5.3KB -1.0B
alerting 41.0KB 41.0KB -1.0B
apm 31.9KB 31.9KB -1.0B
banners 6.5KB 6.5KB -1.0B
canvas 13.3KB 13.3KB -1.0B
cases 130.0KB 130.0KB -1.0B
cloud 3.4KB 3.4KB -1.0B
cloudChat 4.8KB 4.8KB -1.0B
cloudDataMigration 4.4KB 4.4KB -1.0B
cloudDefend 3.6KB 3.6KB -1.0B
cloudExperiments 10.0KB 10.0KB -1.0B
cloudFullStory 3.4KB 3.4KB -1.0B
cloudGainsight 3.5KB 3.5KB -1.0B
cloudLinks 3.0KB 3.0KB -1.0B
cloudSecurityPosture 10.7KB 10.7KB -1.0B
crossClusterReplication 11.9KB 11.9KB -1.0B
customBranding 1.5KB 1.5KB -1.0B
dashboardEnhanced 15.3KB 15.3KB -1.0B
dataVisualizer 25.5KB 25.5KB -1.0B
discoverEnhanced 5.9KB 5.9KB -1.0B
embeddableEnhanced 6.7KB 6.7KB -1.0B
enterpriseSearch 29.6KB 29.6KB -1.0B
features 3.6KB 3.6KB -1.0B
fileUpload 7.2KB 7.2KB -1.0B
fleet 123.7KB 123.7KB -1.0B
globalSearch 6.3KB 6.3KB -1.0B
globalSearchBar 20.5KB 20.5KB -1.0B
globalSearchProviders 4.6KB 4.6KB -1.0B
graph 7.4KB 7.4KB -1.0B
grokdebugger 4.5KB 4.5KB -1.0B
indexLifecycleManagement 26.9KB 26.9KB -1.0B
indexManagement 27.1KB 27.1KB -1.0B
infra 87.1KB 87.1KB -1.0B
ingestPipelines 13.5KB 13.5KB -1.0B
kubernetesSecurity 3.7KB 3.7KB -1.0B
lens 34.0KB 34.0KB -1.0B
licenseManagement 10.2KB 10.2KB -1.0B
licensing 9.1KB 9.1KB -1.0B
lists 3.7KB 3.7KB -1.0B
logstash 14.0KB 14.0KB -1.0B
maps 53.1KB 53.1KB -1.0B
ml 42.5KB 42.5KB -1.0B
monitoring 24.1KB 24.1KB -1.0B
observability 83.3KB 83.3KB -1.0B
osquery 50.3KB 50.3KB -1.0B
painlessLab 10.3KB 10.3KB -1.0B
profiling 13.7KB 13.7KB -1.0B
remoteClusters 7.8KB 7.8KB -1.0B
reporting 42.5KB 42.5KB -1.0B
rollup 11.6KB 11.6KB -1.0B
runtimeFields 11.0KB 11.0KB -1.0B
savedObjectsTagging 19.0KB 19.0KB -1.0B
screenshotting 7.9KB 7.9KB -1.0B
searchprofiler 19.8KB 19.8KB -1.0B
security 57.9KB 57.9KB -1.0B
securitySolution 52.9KB 52.9KB -1.0B
sessionView 5.7KB 5.7KB -1.0B
snapshotRestore 27.5KB 27.5KB -1.0B
spaces 20.4KB 20.4KB -1.0B
stackAlerts 15.6KB 15.6KB -1.0B
stackConnectors 28.0KB 28.0KB -1.0B
synthetics 26.5KB 26.5KB -1.0B
threatIntelligence 31.3KB 31.3KB -1.0B
timelines 87.1KB 87.1KB -1.0B
transform 16.8KB 16.8KB -1.0B
triggersActionsUi 117.8KB 117.8KB -1.0B
upgradeAssistant 19.4KB 19.4KB -1.0B
urlDrilldown 15.4KB 15.4KB -1.0B
ux 5.9KB 5.9KB -1.0B
visTypeGauge 12.1KB 12.1KB -6.0B
visTypeHeatmap 12.6KB 12.6KB -6.0B
visTypePie 10.3KB 10.3KB -6.0B
visTypeVislib 12.8KB 12.8KB -12.0B
visTypeXy 31.1KB 31.1KB -6.0B
watcher 13.4KB 13.4KB -1.0B
total -107.0B
Unknown metric groups

API count

id before after diff
@kbn/chart-expressions-common - 4 +4
@kbn/get-repo-files 6 0 -6
@kbn/json-ast 32 41 +9
@kbn/test 269 271 +2
core 2848 - -2848
total -2839

ESLint disabled in files

id before after diff
dataViews 2 1 -1
observability 7 6 -1
ux 3 2 -1
total -3

ESLint disabled line counts

id before after diff
@kbn/core - 6 +6
@kbn/optimizer 3 1 -2
core 7 - -7
esUiShared 12 11 -1
total -4

References to deprecated APIs

id before after diff
@kbn/core - 37 +37
@kbn/core-application-browser-internal 2 1 -1
@kbn/core-application-browser-mocks 2 1 -1
@kbn/core-elasticsearch-server-internal 4 2 -2
@kbn/core-lifecycle-browser 6 2 -4
@kbn/core-plugins-browser-internal 8 5 -3
@kbn/core-plugins-server 3 0 -3
@kbn/core-plugins-server-internal 20 10 -10
@kbn/core-root-browser-internal 5 4 -1
@kbn/core-saved-objects-api-browser 137 54 -83
@kbn/core-saved-objects-api-server 89 67 -22
@kbn/core-saved-objects-browser 6 2 -4
@kbn/core-saved-objects-browser-internal 393 238 -155
@kbn/core-saved-objects-browser-mocks 55 32 -23
@kbn/core-saved-objects-common 14 6 -8
@kbn/core-saved-objects-import-export-server-internal 70 56 -14
@kbn/core-saved-objects-migration-server-internal 44 22 -22
@kbn/core-saved-objects-server 16 12 -4
@kbn/core-saved-objects-server-internal 4 2 -2
@kbn/core-test-helpers-so-type-serializer 8 4 -4
@kbn/core-ui-settings-server-internal 15 12 -3
@kbn/eslint-plugin-imports 1 0 -1
@kbn/find-used-node-modules 3 0 -3
@kbn/repo-source-classifier 1 0 -1
actions 37 22 -15
advancedSettings 7 3 -4
aiops 9 7 -2
alerting 71 46 -25
apm 39 38 -1
canvas 183 147 -36
cases 154 104 -50
cloudSecurityPosture 17 9 -8
console 2 1 -1
controls 39 27 -12
core 37 - -37
dashboard 141 74 -67
dashboardEnhanced 18 9 -9
data 151 114 -37
dataViews 206 150 -56
dataVisualizer 21 18 -3
discover 57 43 -14
embeddable 39 23 -16
encryptedSavedObjects 6 4 -2
enterpriseSearch 13 10 -3
expressions 26 13 -13
fileUpload 2 1 -1
fleet 79 61 -18
globalSearchProviders 4 2 -2
graph 128 86 -42
home 41 34 -7
infra 174 110 -64
kibanaOverview 8 7 -1
kibanaUtils 10 5 -5
lens 210 110 -100
lists 99 93 -6
management 2 1 -1
maps 67 41 -26
ml 191 164 -27
monitoring 6 4 -2
observability 70 66 -4
osquery 53 41 -12
presentationUtil 17 12 -5
savedObjects 234 121 -113
savedObjectsFinder 41 24 -17
savedObjectsManagement 43 35 -8
savedObjectsTagging 110 75 -35
savedObjectsTaggingOss 44 29 -15
savedSearch 74 36 -38
security 65 64 -1
securitySolution 389 357 -32
spaces 17 13 -4
synthetics 164 115 -49
taskManager 4 2 -2
transform 34 29 -5
triggersActionsUi 11 8 -3
uiActionsEnhanced 20 10 -10
unifiedSearch 27 25 -2
upgradeAssistant 22 20 -2
visTypeTimeseries 74 72 -2
visualizations 276 136 -140
watcher 14 8 -6
total -1555

Total ESLint disabled count

id before after diff
@kbn/core - 6 +6
@kbn/optimizer 5 3 -2
core 7 - -7
dataViews 14 13 -1
esUiShared 12 11 -1
observability 60 59 -1
ux 12 11 -1
total -7

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@spalger spalger disabled auto-merge February 9, 2023 03:06
@spalger spalger merged commit 1b85815 into elastic:main Feb 9, 2023
@spalger spalger deleted the implement/plugins-are-packages branch February 9, 2023 03:06
@kibanamachine kibanamachine added v8.8.0 backport:skip This commit does not require backporting labels Feb 9, 2023
@przemyslawciurzynski
Copy link

@spalger I'm using the latest version of kibana because of the added metrics features. I wanted to create a new visualization type for a very edge case. I tried to do it using the plugin generator but than realized that the file extension should be corrected. Unfortunately, after that the application stops working. I started a thread on it here
I would appreciate your help on how to move forward

mistic pushed a commit to mistic/kibana that referenced this pull request May 10, 2023
Fixes elastic#149344

This PR migrates all plugins to packages automatically. It does this
using `node scripts/lint_packages` to automatically migrate
`kibana.json` files to `kibana.jsonc` files. By doing this automatically
we can simplify many build and testing procedures to only support
packages, and not both "packages" and "synthetic packages" (basically
pointers to plugins).

The majority of changes are in operations related code, so we'll be
having operations review this before marking it ready for review. The
vast majority of the code owners are simply pinged because we deleted
all `kibana.json` files and replaced them with `kibana.jsonc` files, so
we plan on leaving the PR ready-for-review for about 24 hours before
merging (after feature freeze), assuming we don't have any blockers
(especially from @elastic/kibana-core since there are a few core
specific changes, though the majority were handled in elastic#149370).

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit 1b85815)

# Conflicts:
#	.github/CODEOWNERS
#	src/dev/build/tasks/bundle_fleet_packages.ts
#	src/dev/tsconfig.json
#	yarn.lock
@mistic
Copy link
Member

mistic commented May 10, 2023

💚 All backports created successfully

Status Branch Result
8.7

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

mistic added a commit that referenced this pull request May 11, 2023
# Backport

This will backport the following commits from `main` to `8.7`:
- [[packages] migrate all plugins to packages
(#148130)](#148130)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT
[{"author":{"name":"Spencer","email":"spencer@elastic.co"},"sourceCommit":{"committedDate":"2023-02-09T03:06:50Z","message":"[packages]
migrate all plugins to packages (#148130)\n\nFixes
https://github.com/elastic/kibana/issues/149344\r\n\r\nThis PR migrates
all plugins to packages automatically. It does this\r\nusing `node
scripts/lint_packages` to automatically migrate\r\n`kibana.json` files
to `kibana.jsonc` files. By doing this automatically\r\nwe can simplify
many build and testing procedures to only support\r\npackages, and not
both \"packages\" and \"synthetic packages\" (basically\r\npointers to
plugins).\r\n\r\nThe majority of changes are in operations related code,
so we'll be\r\nhaving operations review this before marking it ready for
review. The\r\nvast majority of the code owners are simply pinged
because we deleted\r\nall `kibana.json` files and replaced them with
`kibana.jsonc` files, so\r\nwe plan on leaving the PR ready-for-review
for about 24 hours before\r\nmerging (after feature freeze), assuming we
don't have any blockers\r\n(especially from @elastic/kibana-core since
there are a few core\r\nspecific changes, though the majority were
handled in #149370).\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"1b8581540295fde746dae6b4a09d74fb5821bfef","branchLabelMapping":{"^v8.8.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","backport:skip","v8.8.0"],"number":148130,"url":"https://github.com/elastic/kibana/pull/148130","mergeCommit":{"message":"[packages]
migrate all plugins to packages (#148130)\n\nFixes
https://github.com/elastic/kibana/issues/149344\r\n\r\nThis PR migrates
all plugins to packages automatically. It does this\r\nusing `node
scripts/lint_packages` to automatically migrate\r\n`kibana.json` files
to `kibana.jsonc` files. By doing this automatically\r\nwe can simplify
many build and testing procedures to only support\r\npackages, and not
both \"packages\" and \"synthetic packages\" (basically\r\npointers to
plugins).\r\n\r\nThe majority of changes are in operations related code,
so we'll be\r\nhaving operations review this before marking it ready for
review. The\r\nvast majority of the code owners are simply pinged
because we deleted\r\nall `kibana.json` files and replaced them with
`kibana.jsonc` files, so\r\nwe plan on leaving the PR ready-for-review
for about 24 hours before\r\nmerging (after feature freeze), assuming we
don't have any blockers\r\n(especially from @elastic/kibana-core since
there are a few core\r\nspecific changes, though the majority were
handled in #149370).\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"1b8581540295fde746dae6b4a09d74fb5821bfef"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.8.0","labelRegex":"^v8.8.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/148130","number":148130,"mergeCommit":{"message":"[packages]
migrate all plugins to packages (#148130)\n\nFixes
https://github.com/elastic/kibana/issues/149344\r\n\r\nThis PR migrates
all plugins to packages automatically. It does this\r\nusing `node
scripts/lint_packages` to automatically migrate\r\n`kibana.json` files
to `kibana.jsonc` files. By doing this automatically\r\nwe can simplify
many build and testing procedures to only support\r\npackages, and not
both \"packages\" and \"synthetic packages\" (basically\r\npointers to
plugins).\r\n\r\nThe majority of changes are in operations related code,
so we'll be\r\nhaving operations review this before marking it ready for
review. The\r\nvast majority of the code owners are simply pinged
because we deleted\r\nall `kibana.json` files and replaced them with
`kibana.jsonc` files, so\r\nwe plan on leaving the PR ready-for-review
for about 24 hours before\r\nmerging (after feature freeze), assuming we
don't have any blockers\r\n(especially from @elastic/kibana-core since
there are a few core\r\nspecific changes, though the majority were
handled in #149370).\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"1b8581540295fde746dae6b4a09d74fb5821bfef"}}]}]
BACKPORT-->

---------

Co-authored-by: Spencer <spencer@elastic.co>
jgowdyelastic added a commit that referenced this pull request Jul 31, 2024
The ML team should be a code owner of the file upload plugin.
This [PR](#148130) removed them by
accident while removing duplicate entries.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:Operations Team label for Operations Team v8.7.2 v8.8.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

migrate all plugins to packages