migrate TaskManager Plugin to the Kibana Platform #53869

gmmorris · 2019-12-31T16:10:14Z

Summary

This PR migrated the existing TaskManager plugin from Legacy to Kibana Platform.
We retain the Legacy API to prevent a breaking change, but under the hood, the legacy plugin is now using the Kibana Platform plugin.

Another reason we retain the Legacy plugin to support several features that the Platform team has yet to migrate to Kibana Platform (mapping, SO schema and migrations).

closes #49629

Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

~~This was checked for cross-browser compatibility, including a check against IE11~~
~~Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support~~
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
~~This was checked for keyboard-only and screenreader accessibility~~

For maintainers

This was checked for breaking API changes and was labeled appropriately
This includes a feature addition or change that requires a release note and was labeled appropriately

elasticmachine · 2019-12-31T16:12:10Z

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

kibanamachine · 2019-12-31T17:23:02Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: f7c61c7

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* master: Fix suggested value for time_zone in range query (elastic#53841) Clean up generic hooks, use react-use instead (elastic#53822) Bump year in NOTICE.txt Add kibanamachine support to Github PR comments (elastic#53852) Add tests to ensure AAD isn't broken after performing a change on an alert / action (elastic#53333) Skip failing test suite [Vega] Sample [Flights] Airport Connections (Hover Over Airport) visualization not working (elastic#53799) Do not remount applications between page navigations (elastic#53851) [Canvas] Refactor Canvas to no longer use componentWillReceiveProps (elastic#52129) [Canvas] Migrate usage collector to NP plugin (elastic#53303)

gmmorris · 2020-01-02T15:48:54Z

@elasticmachine merge upstream

…na into task-manager/schema-config * 'task-manager/schema-config' of github.com:gmmorris/kibana: [Maps] push mapbox value extraction from VectorStyle and into DynamicStyleProperty (elastic#53806)

* master: Rename `/api/security/oidc` to `/api/security/oidc/callback`. (elastic#53886) Updating transitive dependencies to use handlebars@4.5.3 (elastic#53899) [Reporting/Tests] consolidate functional test configs (elastic#52671) [Reporting] Correct the docvalue_fields params in the search query Download CSV from Dashboard Panel (elastic#52833) [Test/Newsfeed] Re-enable test and add news item to be filtered (elastic#53905) cleanup server-log action (elastic#53326) [Uptime] Delete uptime eslint rule skip (elastic#50912) [skip-ci] Expression Lifecycle Docs (elastic#51494) [Endpoint] add react router to endpoint app (elastic#53808) [SIEM][Detection Engine] Silence 409 errors on signal creation (elastic#53859) [Maps] get max_result_window and max_inner_result_window from index settings (elastic#53500) [ML] New Platform server shim: update analytics routes to use new platform router (elastic#53521) fixes typo on engine detection page (elastic#53877)

* master: Move index patterns: src/legacy/core_plugins/data 👉 src/plugins/data (elastic#53794) moved Task Manager server code under "server" directory (elastic#53777)

* master: increase delay to make sure license refetched (elastic#53882) Allow custom NP plugin paths in production (elastic#53562) [Maps] show custom color ramps in legend (elastic#53780) [Lens] Expression type on document can be null (elastic#53883) [SIEM] [Detection engine] Add user permission to detection engine (elastic#53778) Update dependency @elastic/charts to v16.0.2 (elastic#52619) Set consistent EOL symbol in core API docs (elastic#53815) [Logs UI] Refactor query bar state to hooks (elastic#52656) [Maps] pass getFieldFormatter to DynamicTextProperty (elastic#53937) Invalidate alert API Key when generating a new one (elastic#53732) [Logs UI] HTTP API for log entries (elastic#53798) [kbn/pm] add caching to bootstrap (elastic#53622) adds createdAt and updatedAt fields to alerting (elastic#53793) [SR] Enable component integration tests (elastic#53893)

* master: allows Alerts to recover gracefully from Executor errors (elastic#53688) [Console] Fix OSS build (elastic#53885) migrate xsrf / version-check / custom-headers handlers to NP (elastic#53684) use NP deprecations in uiSettings (elastic#53755) adds strict types to Alerting Client (elastic#53821) [Dashboard] Empty screen redesign (elastic#53681) Migrate config deprecations and `ShieldUser` functionality to the New Platform (elastic#53768)

gmmorris · 2020-01-08T15:48:00Z

@elastic/kibana-platform @elastic/kibana-gis @elastic/kibana-app

Hi y'all,
Migrating Task Manager to the Kibana Platform required making changes across anywhere using TM plus exposing some types in places (as TM is kind of a special case in how it uses SavedObjects) which is why y'all have been pinged.

Please let me know if you have any concerns or questions. :)

gmmorris · 2020-01-08T15:49:02Z

src/core/server/mocks.ts

@@ -37,6 +37,7 @@ export { elasticsearchServiceMock } from './elasticsearch/elasticsearch_service.
 export { httpServiceMock } from './http/http_service.mock';
 export { loggingServiceMock } from './logging/logging_service.mock';
 export { savedObjectsClientMock } from './saved_objects/service/saved_objects_client.mock';
+export { savedObjectsRepositoryMock } from './saved_objects/service/lib/repository.mock';


Task Manager interacts with the repository directly, so accessing this mock was needed for testing purposes.

it's okay. The SO repository is a part of public contract

kibana/src/core/server/saved_objects/saved_objects_service.ts

Line 260 in 7ffe385

createInternalRepository: (extraTypes?: string[]) =>

x-pack/legacy/plugins/lens/index.ts

kindsun · 2020-01-08T18:20:16Z

x-pack/legacy/plugins/maps/server/test_utils/index.js

-    task_manager: {
-      registerTaskDefinitions: () => undefined,
-      schedule: () => Promise.resolve(),
-      fetch: mockTaskFetch,


The best thing to do here would be to just remove this portion as you have, and not replace it with anything. We aren't using task manager as of the recent merge of #52834. Really this whole function getMockKbnServer could stand to go, but for the purposes of this PR, maybe just remove task_manager and its value.

Cool, removed :)

mshustov

platform changes 👍

mshustov · 2020-01-09T07:20:42Z

src/core/server/mocks.ts

@@ -37,6 +37,7 @@ export { elasticsearchServiceMock } from './elasticsearch/elasticsearch_service.
 export { httpServiceMock } from './http/http_service.mock';
 export { loggingServiceMock } from './logging/logging_service.mock';
 export { savedObjectsClientMock } from './saved_objects/service/saved_objects_client.mock';
+export { savedObjectsRepositoryMock } from './saved_objects/service/lib/repository.mock';


it's okay. The SO repository is a part of public contract

kibana/src/core/server/saved_objects/saved_objects_service.ts

Line 260 in 7ffe385

createInternalRepository: (extraTypes?: string[]) =>

* master: (23 commits) [Vis: Default editor] Reactify the timelion editor (elastic#52990) [Discover] fix histogram min interval (elastic#53979) [Telemetry] [Monitoring] Only retry fetching usage once monito… (elastic#54309) [docs][APM] Add runtime index config documentation (elastic#53907) [SIEM] Detection engine timeline (elastic#53783) Filter scripted fields preview field list to source fields (elastic#53826) Management - New platform api (elastic#52579) Reset region and Account when switching inventory (elastic#54287) [SIEM] [Case] Case workflow api schema (elastic#51535) Code coverage setup on CI (elastic#49003) [ML] DF Analytics Results: adds link to docs (elastic#54189) Update schemas boolean, byteSize, and duration to coerce strings (elastic#54177) [Metrics UI] Pass relevant shouldAllowEdit capabilities into SettingsPage (elastic#49781) [Canvas] Fixes bugs with autoplay and refresh (elastic#53149) [ML] DF Analytics Classification: ensure confusion matrix can be fetched (elastic#53629) Fix Vega react eslint errors (elastic#54259) Remove non existing codeowners (elastic#54274) use correct type (elastic#54244) [Dashboard] Removing 100% as dshDashboardViewport height (elastic#54263) add `examples/` to no-restricted-path config (elastic#54252) ...

gmmorris · 2020-01-08T16:25:36Z

x-pack/legacy/plugins/task_manager/server/index.ts

        index: Joi.string()
          .description('The name of the index used to store task information.')
          .default('.kibana_task_manager')
          .invalid(['.tasks']),


Schema still uses index in the legacy plugin, so we'll keep that config here until we no longer need it in the legacy plugin

x-pack/legacy/plugins/lens/index.ts

gmmorris · 2020-01-09T08:47:11Z

x-pack/legacy/plugins/maps/server/test_utils/index.js

-    task_manager: {
-      registerTaskDefinitions: () => undefined,
-      schedule: () => Promise.resolve(),
-      fetch: mockTaskFetch,


Cool, removed :)

* master: Remove eslint overwrite for src/legacy/core_plugins/kibana (elastic#54222)

kindsun

Thanks for cleaning things up a bit for us! Maps lgtm!

pmuellr

LGTM

* master: (69 commits) [Graph] Fix various a11y issues (elastic#54097) Add ApplicationService app status management (elastic#50223) logs in one time (elastic#54447) Deprecate using `elasticsearch.ssl.certificate` without `elasticsearch.ssl.key` and vice versa (elastic#54392) [Optimizer] Fix a stack overflow with watch_cache when it attempts to delete very large folders. (elastic#54457) Security - Role Mappings UI (elastic#53620) [SIEM] [Detection engine] Permission II (elastic#54292) Allow User to Cleanup Repository from UI (elastic#53047) [Detection engine] Some UX for rule creation (elastic#54471) share specific instances of some ui packages (elastic#54079) [ML] APM modules configs for RUM Javascript and NodeJS (elastic#53792) [APM] Delay rendering invalid license notification (elastic#53924) [Graph] Improve error message on graph requests (elastic#54230) [ILM] Kibana should allow a min_age setting of 0ms in ILM policy phases (elastic#53719) Unit Tests for common/lib (elastic#53736) [Graph] Only show explorable fields (elastic#54101) remove linting rule exception for markdown (elastic#54232) [Monitoring] Fetch shard data more efficiently (elastic#54028) [Maps] Add hiddenLayers option to embeddable map input (elastic#54355) Pass termOrder and hasTermsAgg properties to serializeThresholdWatch function (elastic#54391) ...

wylieconlon

I don't have any major concerns about this, but I did leave some comments. Did not do a thorough review of the changes other than to Lens

x-pack/legacy/plugins/lens/index.ts

wylieconlon · 2020-01-13T16:12:32Z

x-pack/legacy/plugins/lens/server/plugin.tsx

+  taskManager: TaskManagerStartContract;
+}
+
+const taskManagerStartContract$ = new Subject<TaskManagerStartContract>();


I'm surprised that Lens is responsible here for getting an instance of the task manager. Why isn't this instantiated somewhere else?

This is just because Lens is still a Legacy plugin.
Once Lens is migrated to NP, this should happen automatically.

Looking at it again I realise there's another issue in Lens specifically - during the Setup stage It's trying to use a TaskManager API that's only at Start stage, so we need to wrap that in some async method of using it - that's what the Subject is doing.

kibanamachine · 2020-01-13T18:04:28Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 5ad09d2

History

💚 Build #19671 succeeded 4173614
💚 Build #18947 succeeded 9e63e9b
💔 Build #18914 failed 35d55e7
💚 Build #18768 succeeded 39b6857
💚 Build #18744 succeeded f5ec90c

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Migrates the existing TaskManager plugin from Legacy to Kibana Platform. We retain the Legacy API to prevent a breaking change, but under the hood, the legacy plugin is now using the Kibana Platform plugin. Another reason we retain the Legacy plugin to support several features that the Platform team has yet to migrate to Kibana Platform (mapping, SO schema and migrations).

Migrates the existing TaskManager plugin from Legacy to Kibana Platform. We retain the Legacy API to prevent a breaking change, but under the hood, the legacy plugin is now using the Kibana Platform plugin. Another reason we retain the Legacy plugin to support several features that the Platform team has yet to migrate to Kibana Platform (mapping, SO schema and migrations). Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Migrates the existing TaskManager plugin from Legacy to Kibana Platform. We retain the Legacy API to prevent a breaking change, but under the hood, the legacy plugin is now using the Kibana Platform plugin. Another reason we retain the Legacy plugin to support several features that the Platform team has yet to migrate to Kibana Platform (mapping, SO schema and migrations).

gmmorris and others added 3 commits December 23, 2019 19:06

moved Task Manager server code under "server" directory

3155252

Merge branch 'master' into task-manager/server-dir

2cbc508

expose Task Manager via a Kibana Platform plugin

f7c61c7

gmmorris added Feature:Alerting Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v7.6.0 v8.0.0 labels Dec 31, 2019

gmmorris added 4 commits January 2, 2020 10:07

fixed TM types

00efbf0

fixed TM usage in maps

b0c47b6

fixed test fixture in maps

1108dfb

elasticmachine and others added 16 commits January 2, 2020 07:49

Merge branch 'master' into task-manager/schema-config

f317390

fixed empty TaskManager issue in oss_telemetry

5f8e2e5

Merge branch 'task-manager/schema-config' of github.com:gmmorris/kiba…

ce915d8

…na into task-manager/schema-config * 'task-manager/schema-config' of github.com:gmmorris/kibana: [Maps] push mapbox value extraction from VectorStyle and into DynamicStyleProperty (elastic#53806)

Merge branch 'master' into task-manager/schema-config

85be0b8

* master: Move index patterns: src/legacy/core_plugins/data 👉 src/plugins/data (elastic#53794) moved Task Manager server code under "server" directory (elastic#53777)

removed optional chaining

1e582b8

fixed tm reference in collectors

e6862da

fixed tm immport in alerting FT

2047538

cleaned up udage of new TM in lens

731c476

use NP elasticsearch client in legacy plugin

5c93173

removed unneeded import

37fa4b1

pass TM directly to NP plugin

d797059

moved to using the NP SavedObjectsRepository

9327532

moved Legacy Task Manager implementation into the New Platform Plugin

e4263e5

gmmorris marked this pull request as ready for review January 8, 2020 15:43

gmmorris requested a review from a team January 8, 2020 15:43

gmmorris requested review from a team as code owners January 8, 2020 15:43

gmmorris added the Feature:Task Manager label Jan 8, 2020

gmmorris commented Jan 8, 2020

View reviewed changes

gmmorris added 2 commits January 8, 2020 16:23

clean up usage of NP TM in legacy plugins

a2cce60

use legacy helper in our own legacy TM obviously

39b6857

pmuellr reviewed Jan 8, 2020

View reviewed changes

x-pack/legacy/plugins/lens/index.ts Show resolved Hide resolved

kindsun reviewed Jan 8, 2020

View reviewed changes

mshustov approved these changes Jan 9, 2020

View reviewed changes

gmmorris added 2 commits January 9, 2020 08:33

cleaned up dependency on TM in other plugins

35d55e7

gmmorris commented Jan 9, 2020

View reviewed changes

kertal requested review from wylieconlon and chrisdavies January 9, 2020 11:12

Merge branch 'master' into task-manager/schema-config

9e63e9b

* master: Remove eslint overwrite for src/legacy/core_plugins/kibana (elastic#54222)

kindsun approved these changes Jan 9, 2020

View reviewed changes

pmuellr approved these changes Jan 9, 2020

View reviewed changes

wylieconlon approved these changes Jan 13, 2020

View reviewed changes

allow lens t ostart even if TM isnt available

5ad09d2

gmmorris merged commit ea9a7b8 into elastic:master Jan 13, 2020

gmmorris mentioned this pull request Jan 13, 2020

[7.x] migrate TaskManager Plugin to the Kibana Platform (#53869) #54630

Merged

gmmorris added the backported label Jan 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

migrate TaskManager Plugin to the Kibana Platform #53869

migrate TaskManager Plugin to the Kibana Platform #53869

gmmorris commented Dec 31, 2019 •

edited

Loading

elasticmachine commented Dec 31, 2019

kibanamachine commented Dec 31, 2019

gmmorris commented Jan 2, 2020

gmmorris commented Jan 8, 2020

gmmorris Jan 8, 2020

mshustov Jan 9, 2020

kindsun Jan 8, 2020

gmmorris Jan 9, 2020

mshustov left a comment

mshustov Jan 9, 2020

gmmorris Jan 8, 2020

gmmorris Jan 9, 2020

kindsun left a comment

pmuellr left a comment

wylieconlon left a comment

wylieconlon Jan 13, 2020

gmmorris Jan 13, 2020

gmmorris Jan 13, 2020

kibanamachine commented Jan 13, 2020

migrate TaskManager Plugin to the Kibana Platform #53869

migrate TaskManager Plugin to the Kibana Platform #53869

Conversation

gmmorris commented Dec 31, 2019 • edited Loading

Summary

Checklist

For maintainers

elasticmachine commented Dec 31, 2019

kibanamachine commented Dec 31, 2019

💔 Build Failed

gmmorris commented Jan 2, 2020

gmmorris commented Jan 8, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mshustov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kindsun left a comment

Choose a reason for hiding this comment

pmuellr left a comment

Choose a reason for hiding this comment

wylieconlon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibanamachine commented Jan 13, 2020

💚 Build Succeeded

History

gmmorris commented Dec 31, 2019 •

edited

Loading