[plugin-helpers] improve 3rd party KP plugin support #75019

spalger · 2020-08-14T02:55:55Z

Closes #63069
Closes #70412
Closes #61731

This PR updates the plugin-helpers to be compatible with KP plugins instead of legacy plugins. It drastically refactors the code to both modernize it, improve it's type safety, and remove legacy cruft.

The dev and test:server tasks were removed, along with the old commander based interface. There is now a single command available from the plugin-helpers, build.

The old js API was also removed in favor of existing APIs from packages like @kbn/dev-utils.

The build command creates distributable versions of plugins in a very similar way as it used to, except that those plugins now include a kibana.json file and have browser bundles which are created by the @kbn/optimizer. Additionally, server code is pre-processed by babel allowing us to remove babel support from production completely.

A docs PR will follow with a generation-to-distribution example of developing a 3rd party plugin.

…gin-helpers-kp-support

tylersmalley

LGTM!

…g on depth, avoid one level of yarn...

…nt the output if it fails

…gin-helpers-kp-support

kibanamachine · 2020-08-27T21:58:21Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 08006af

Build metrics

distributable file count

id	value	diff	baseline
total	44361	-8765	53126

History

💚 Build #71029 succeeded 8e37822
💔 Build #71021 failed 8f8bae2
💔 Build #70998 failed c8f56f3
💔 Build #70988 failed db1f6b2
💔 Build #70975 failed 7143db3

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Co-authored-by: Tyler Smalley <tylersmalley@me.com> Co-authored-by: spalger <spalger@users.noreply.github.com> # Conflicts: # packages/kbn-plugin-helpers/package.json # x-pack/scripts/functional_tests.js # yarn.lock

…76209) Co-authored-by: Tyler Smalley <tylersmalley@me.com> Co-authored-by: spalger <spalger@users.noreply.github.com>

…s-for-710 * 'master' of github.com:elastic/kibana: (43 commits) [APM] Chart units don't update when toggling the chart legends (elastic#74931) [ILM] Add support for frozen phase in UI (elastic#75968) Hides advanced json for count metric (elastic#74636) add client-side feature usage API (elastic#75486) [Maps] add drilldown support map embeddable (elastic#75598) [Enterprise Search] Request handler refactors/enhancements + update existing routes to use shared handler (elastic#76106) [Resolver] model `location.search` in redux (elastic#76140) [APM] Prevent imports of public in server code (elastic#75979) fix eslint issue skip flaky suite (elastic#76223) [APM] Transaction duration anomaly alerting integration (elastic#75719) [Transforms] Avoid using "Are you sure" (elastic#75932) [Security Solution][Exceptions] - Fix bug of alerts not updating after closure from exceptions modal (elastic#76145) [plugin-helpers] improve 3rd party KP plugin support (elastic#75019) [docs/getting-started] link to yarn v1 specifically (elastic#76169) [Security_Solution][Resolver] Resolver loading and error state (elastic#75600) Fixes App Search documentation links (elastic#76133) Fix alerts unable to create / update when the name has trailing whitepace(s) (elastic#76079) [Resolver] Fix useSelector usage (elastic#76129) [Enterprise Search] Migrate util and components from ent-search (elastic#76051) ... # Conflicts: # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/data_tier_allocation/node_allocation.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/edit_policy.tsx # x-pack/plugins/index_lifecycle_management/public/application/services/policies/types.ts # x-pack/plugins/index_lifecycle_management/public/application/services/policies/warm_phase.ts

fbaligand · 2020-09-03T09:12:46Z

Hi @spalger,

In this new implementation, is the plugin-helper able to build both old and new platform plugins?

spalger · 2020-09-03T15:24:03Z

Nope, it's only able to build KP plugins because legacy plugin support is being removed for 7.10 so there's no point in overcomplicating the tool so that it understands both. Just trying to move forward and better support the tooling with KP support.

fbaligand · 2020-09-03T16:18:40Z

OK, thanks for the answer.
Given this article, I believed that legacy API support will be removed in kibana 7.11:
https://www.elastic.co/fr/blog/introducing-a-new-architecture-for-kibana

joshdover · 2020-09-03T16:33:34Z

OK, thanks for the answer.
Given this article, I believed that legacy API support will be removed in kibana 7.11:
elastic.co/fr/blog/introducing-a-new-architecture-for-kibana

That's a mistake on my part. I'll have that post updated today. We originally estimated that this work would take longer and be a lower priority, hence the original 7.11 estimate. We decided to bump the priority of removing the legacy system for performance reasons and are now on track to remove legacy support 100% in 7.10.

fbaligand · 2020-09-03T17:23:41Z

Nice to know ;)
That’s cool if it significantly impacts kibana performance!

fastlorenzo · 2020-11-12T10:11:00Z

OK, thanks for the answer.
Given this article, I believed that legacy API support will be removed in kibana 7.11:
elastic.co/fr/blog/introducing-a-new-architecture-for-kibana

That's a mistake on my part. I'll have that post updated today. We originally estimated that this work would take longer and be a lower priority, hence the original 7.11 estimate. We decided to bump the priority of removing the legacy system for performance reasons and are now on track to remove legacy support 100% in 7.10.

Just to confirm, is this now part of 7.10 release?

mshustov · 2020-11-12T13:39:01Z

@fastlorenzo yes. The legacy plugin system has been removed in 7.10 The docs about plugin migration to Kibana platform are going be published soon #82600

fastlorenzo · 2020-11-15T00:44:20Z

Not sure if a bug and if related to this change, but I can no longer access the public assets from the browser since migrating to 7.10.0 (cf. #83387 )

fastlorenzo · 2020-11-15T01:20:07Z

packages/kbn-plugin-helpers/src/tasks/write_server_files.ts

+        buffer: true,
+        ignore: [
+          '**/*.d.ts',
+          '**/public/**',


Might be causing the public/assets not being copied to the build folder (#83387 )

Hmm, that makes sense. Will get a PR up for 7.10.1 Monday.

You can probably either add the public/assets directory to the serverSourcePatterns setting.

If you're only dealing with a couple files you could also just import the file using the file-loader, ie

kibana/x-pack/plugins/maps/public/connected_components/mb_map/mb_map.js

Line 19 in b8e2e85

import mbWorkerUrl from '!!file-loader!mapbox-gl/dist/mapbox-gl-csp-worker';

Nice find, sorry about that.

Glad to hear you know how to fix it 👍

I've tried adding public/**/*, public/assets/**/*, etc in serverSourcePatterns, but nothing worked. My guess is that it's linked to the same issue.

I have a workaround for now: build the plugin with --skip-archive, manually add the content of public/assets to the build folder, and manually package the zip file.

Example with plugin named redelk:

yarn build --skip-archive mkdir build/kibana/redelk/public cp -R public/assets/ build/kibana/redelk/public cd build zip redelk-7.10.0.zip kibana -r

Thanks to share your workaround!

spalger added Team:Operations Team label for Operations Team v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.10.0 labels Aug 14, 2020

spalger mentioned this pull request Aug 14, 2020

[plugin-helpers] improve 3rd party KP plugin support #66687

Closed

spalger force-pushed the implement/plugin-helpers-kp-support branch 22 times, most recently from 37b6ea0 to b8e932d Compare August 24, 2020 19:08

[plugin-helpers] improve 3rd party KP plugin support

f0c0768

spalger force-pushed the implement/plugin-helpers-kp-support branch from b8e932d to f0c0768 Compare August 24, 2020 19:37

spalger marked this pull request as ready for review August 24, 2020 20:14

Merge branch 'master' of github.com:elastic/kibana into implement/plu…

eefa5b1

…gin-helpers-kp-support

tylersmalley approved these changes Aug 27, 2020

View reviewed changes

spalger added 6 commits August 27, 2020 10:42

avoid using .all stdout collection, it seems unreliable

7143db3

running yarn inside yarn produces different levels of output dependin…

db1f6b2

…g on depth, avoid one level of yarn...

try more approaches to getting yarn command output on CI

c8f56f3

stop writing yarn output to the terminal, just run it quietly and pri…

8f8bae2

…nt the output if it fails

😡

08006af

Merge branch 'master' of github.com:elastic/kibana into implement/plu…

8e37822

…gin-helpers-kp-support

spalger merged commit 64311d3 into elastic:master Aug 27, 2020

spalger deleted the implement/plugin-helpers-kp-support branch August 27, 2020 21:56

spalger mentioned this pull request Aug 27, 2020

[7.x] [plugin-helpers] improve 3rd party KP plugin support (#75019) #76209

Merged

spalger added a commit that referenced this pull request Aug 27, 2020

[7.x] [plugin-helpers] improve 3rd party KP plugin support (#75019) (#…

c8a93b6

…76209) Co-authored-by: Tyler Smalley <tylersmalley@me.com> Co-authored-by: spalger <spalger@users.noreply.github.com>

zhongnansu mentioned this pull request Sep 22, 2020

plugin fails to run after installed to kibana opendistro-for-elasticsearch/kibana-reports#40

Closed

s-newman mentioned this pull request Oct 5, 2020

Switch to improved plugin-helpers build process scorestack/scorestack#247

Open

fastlorenzo reviewed Nov 15, 2020

View reviewed changes

fastlorenzo mentioned this pull request Nov 15, 2020

[plugin] Public assets no longer exposed #83387

Closed

spalger mentioned this pull request Nov 16, 2020

[kbn/plugin-helpers/build] copy the public assets of a plugin #83458

Merged

frankeros mentioned this pull request Jan 18, 2021

Bugfixing for 7.10.0 wazuh/wazuh-dashboard-plugins#2727

Closed

19 tasks

tylersmalley mentioned this pull request Mar 22, 2022

Plugin bundle is not created for external new platform plugins resulting in 404 #70426

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[plugin-helpers] improve 3rd party KP plugin support #75019

[plugin-helpers] improve 3rd party KP plugin support #75019

spalger commented Aug 14, 2020 •

edited

tylersmalley left a comment

kibanamachine commented Aug 27, 2020

fbaligand commented Sep 3, 2020

spalger commented Sep 3, 2020

fbaligand commented Sep 3, 2020

joshdover commented Sep 3, 2020

fbaligand commented Sep 3, 2020

fastlorenzo commented Nov 12, 2020

mshustov commented Nov 12, 2020

fastlorenzo commented Nov 15, 2020

fastlorenzo Nov 15, 2020

spalger Nov 15, 2020

fastlorenzo Nov 15, 2020

fbaligand Nov 15, 2020

[plugin-helpers] improve 3rd party KP plugin support #75019

[plugin-helpers] improve 3rd party KP plugin support #75019

Conversation

spalger commented Aug 14, 2020 • edited

tylersmalley left a comment

Choose a reason for hiding this comment

kibanamachine commented Aug 27, 2020

💚 Build Succeeded

Build metrics

distributable file count

History

fbaligand commented Sep 3, 2020

spalger commented Sep 3, 2020

fbaligand commented Sep 3, 2020

joshdover commented Sep 3, 2020

fbaligand commented Sep 3, 2020

fastlorenzo commented Nov 12, 2020

mshustov commented Nov 12, 2020

fastlorenzo commented Nov 15, 2020

fastlorenzo Nov 15, 2020

Choose a reason for hiding this comment

spalger Nov 15, 2020

Choose a reason for hiding this comment

fastlorenzo Nov 15, 2020

Choose a reason for hiding this comment

fbaligand Nov 15, 2020

Choose a reason for hiding this comment

spalger commented Aug 14, 2020 •

edited