Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[App Arch] [Cleanup] Saved object migrations from kibana plugin to new platform #64289

Closed
wants to merge 4 commits into from
Closed

[App Arch] [Cleanup] Saved object migrations from kibana plugin to new platform #64289

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ed3a959
[App Arch] [Cleanup] Saved object migrations from kibana plugin to ne…
alexwizp Apr 23, 2020
bef7546
Merge remote-tracking branch 'upstream/master'
alexwizp Apr 23, 2020
43244f3
add two generics TInputDoc and TOutputDoc
alexwizp Apr 23, 2020
d547ab1
fix check_published_api_changes.js
alexwizp Apr 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/development/core/server/kibana-plugin-core-server.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,7 @@ The plugin integrates with the core system via lifecycle events: `setup`<!-- -->
| [SavedObjectsClientWrapperFactory](./kibana-plugin-core-server.savedobjectsclientwrapperfactory.md) | Describes the factory used to create instances of Saved Objects Client Wrappers. |
| [SavedObjectsFieldMapping](./kibana-plugin-core-server.savedobjectsfieldmapping.md) | Describe a [saved object type mapping](./kibana-plugin-core-server.savedobjectstypemappingdefinition.md) field.<!-- -->Please refer to [elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html) For the mapping documentation |
| [SavedObjectsNamespaceType](./kibana-plugin-core-server.savedobjectsnamespacetype.md) | The namespace type dictates how a saved object can be interacted in relation to namespaces. Each type is mutually exclusive: \* single (default): this type of saved object is namespace-isolated, e.g., it exists in only one namespace. \* multiple: this type of saved object is shareable, e.g., it can exist in one or more namespaces. \* agnostic: this type of saved object is global.<!-- -->Note: do not write logic that uses this value directly; instead, use the appropriate accessors in the [type registry](./kibana-plugin-core-server.savedobjecttyperegistry.md)<!-- -->. |
| [SavedObjectUnsanitizedDoc](./kibana-plugin-core-server.savedobjectunsanitizeddoc.md) | We want to have two types, one that guarantees a "references" attribute will exist and one that allows it to be null. Since we're not migrating all the saved objects to have a "references" array, we need to support the scenarios where it may be missing (ex migrations). |
| [ScopeableRequest](./kibana-plugin-core-server.scopeablerequest.md) | A user credentials container. It accommodates the necessary auth credentials to impersonate the current user.<!-- -->See [KibanaRequest](./kibana-plugin-core-server.kibanarequest.md)<!-- -->. |
| [ServiceStatusLevel](./kibana-plugin-core-server.servicestatuslevel.md) | A convenience type that represents the union of each value in [ServiceStatusLevels](./kibana-plugin-core-server.servicestatuslevels.md)<!-- -->. |
| [SharedGlobalConfig](./kibana-plugin-core-server.sharedglobalconfig.md) | |
Expand Down
2 changes: 1 addition & 1 deletion docs/development/core/server/kibana-plugin-core-server.savedobjectmigrationfn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ A migration function for a [saved object type](./kibana-plugin-core-server.saved
<b>Signature:</b>

```typescript
export declare type SavedObjectMigrationFn = (doc: SavedObjectUnsanitizedDoc, context: SavedObjectMigrationContext) => SavedObjectUnsanitizedDoc;
export declare type SavedObjectMigrationFn<TInputDoc extends SavedObjectUnsanitizedDoc | SavedObjectSanitizedDoc = SavedObjectUnsanitizedDoc, TOutputDoc extends SavedObjectUnsanitizedDoc | SavedObjectSanitizedDoc = SavedObjectUnsanitizedDoc> = (doc: TInputDoc, context: SavedObjectMigrationContext) => TOutputDoc;
```

## Example
Expand Down
13 changes: 13 additions & 0 deletions .../development/core/server/kibana-plugin-core-server.savedobjectunsanitizeddoc.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectUnsanitizedDoc](./kibana-plugin-core-server.savedobjectunsanitizeddoc.md)

## SavedObjectUnsanitizedDoc type

We want to have two types, one that guarantees a "references" attribute will exist and one that allows it to be null. Since we're not migrating all the saved objects to have a "references" array, we need to support the scenarios where it may be missing (ex migrations).

<b>Signature:</b>

```typescript
export declare type SavedObjectUnsanitizedDoc = SavedObjectDoc & Partial<Referencable>;
```
10 changes: 5 additions & 5 deletions src/core/server/saved_objects/migrations/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
* under the License.
*/

import { SavedObjectUnsanitizedDoc } from '../serialization';
import { SavedObjectUnsanitizedDoc, SavedObjectSanitizedDoc } from '../serialization';
import { SavedObjectsMigrationLogger } from './core/migration_logger';

/**
Expand All @@ -39,10 +39,10 @@ import { SavedObjectsMigrationLogger } from './core/migration_logger';
*
* @public
*/
export type SavedObjectMigrationFn = (
doc: SavedObjectUnsanitizedDoc,
context: SavedObjectMigrationContext
) => SavedObjectUnsanitizedDoc;
export type SavedObjectMigrationFn<
TInputDoc extends SavedObjectUnsanitizedDoc | SavedObjectSanitizedDoc = SavedObjectUnsanitizedDoc,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to add two generics cause it allow us to avoid a lot of casting in future.

Like e.g. here: src/plugins/dashboard/server/saved_objects/dashboard_migrations.ts
Also using this approach we can guarantee that all Doc's extended from SavedObjectUnsanitizedDoc or SavedObjectSanitizedDoc

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To follow our slack discussion:

This is an alternative of #63943

However on #63943 we choose to only generify the attributes property of SavedObjectUnsanitizedDoc, as we didn't see any (valid) reasons plugin migration functions should be able to superseed/override the root SO type. This could also be dangerous as it may cause developer errors trying to push changes/properties not defined in the schema.

Do you have any usages where you would need to use type override on the whole SavedObjectUnsanitizedDoc type?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least in version 7.0.0 references were added. And we should somehow handle it.
Also who knows what will be added in future?

From the other hand if someone wants to add something into doc we have at least 2 options: use JS without typings, just cast object to needed type =) So, validation should be implemented in a different way

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently there's an index signature so anything goes, but we should definitely remove that

kibana/src/core/server/saved_objects/serialization/types.ts

Line 63 in 97d1685

[rootProp: string]: any;

Root properties cannot be added by plugins, so if we ever add them in Core, we can update this type at the same time like we recently did for the new namespaces root property.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexwizp could I get your review on #63943? Any argument on why the PR wouldn't answer the need in a better/safer way is welcome.

TOutputDoc extends SavedObjectUnsanitizedDoc | SavedObjectSanitizedDoc = SavedObjectUnsanitizedDoc
> = (doc: TInputDoc, context: SavedObjectMigrationContext) => TOutputDoc;

/**
* Migration context provided when invoking a {@link SavedObjectMigrationFn | migration handler}
Expand Down
2 changes: 1 addition & 1 deletion src/core/server/server.api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1680,7 +1680,7 @@ export interface SavedObjectMigrationContext {
}

// @public
export type SavedObjectMigrationFn = (doc: SavedObjectUnsanitizedDoc, context: SavedObjectMigrationContext) => SavedObjectUnsanitizedDoc;
export type SavedObjectMigrationFn<TInputDoc extends SavedObjectUnsanitizedDoc | SavedObjectSanitizedDoc = SavedObjectUnsanitizedDoc, TOutputDoc extends SavedObjectUnsanitizedDoc | SavedObjectSanitizedDoc = SavedObjectUnsanitizedDoc> = (doc: TInputDoc, context: SavedObjectMigrationContext) => TOutputDoc;

// @public
export interface SavedObjectMigrationMap {
Expand Down
19 changes: 9 additions & 10 deletions src/plugins/data/server/saved_objects/search_migrations.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
*/

import { flow, get } from 'lodash';
import { SavedObjectMigrationFn } from 'kibana/server';
import { SavedObjectMigrationFn, SavedObjectSanitizedDoc } from 'kibana/server';
import { DEFAULT_QUERY_LANGUAGE } from '../../common';

const migrateMatchAllQuery: SavedObjectMigrationFn = doc => {
Expand Down Expand Up @@ -55,7 +55,7 @@ const migrateMatchAllQuery: SavedObjectMigrationFn = doc => {
return doc;
};

const migrateIndexPattern: SavedObjectMigrationFn = doc => {
const migrateIndexPattern: SavedObjectMigrationFn<SavedObjectSanitizedDoc> = doc => {
const searchSourceJSON = get(doc, 'attributes.kibanaSavedObjectMeta.searchSourceJSON');
if (typeof searchSourceJSON !== 'string') {
return doc;
Expand All @@ -68,7 +68,7 @@ const migrateIndexPattern: SavedObjectMigrationFn = doc => {
return doc;
}

if (searchSource.index && Array.isArray(doc.references)) {
if (searchSource.index) {
searchSource.indexRefName = 'kibanaSavedObjectMeta.searchSourceJSON.index';
doc.references.push({
name: searchSource.indexRefName,
Expand Down Expand Up @@ -97,13 +97,12 @@ const migrateIndexPattern: SavedObjectMigrationFn = doc => {
return doc;
};

const setNewReferences: SavedObjectMigrationFn = (doc, context) => {
doc.references = doc.references || [];
// Migrate index pattern
return migrateIndexPattern(doc, context);
};
const setNewReferences: SavedObjectMigrationFn = (doc, context) => ({
...doc,
references: doc.references || [],
});

const migrateSearchSortToNestedArray: SavedObjectMigrationFn = doc => {
const migrateSearchSortToNestedArray: SavedObjectMigrationFn<SavedObjectSanitizedDoc> = doc => {
const sort = get(doc, 'attributes.sort');
if (!sort) return doc;

Expand All @@ -123,6 +122,6 @@ const migrateSearchSortToNestedArray: SavedObjectMigrationFn = doc => {

export const searchSavedObjectTypeMigrations = {
'6.7.2': flow<SavedObjectMigrationFn>(migrateMatchAllQuery),
'7.0.0': flow<SavedObjectMigrationFn>(setNewReferences),
'7.0.0': flow<SavedObjectMigrationFn>(setNewReferences, migrateIndexPattern),
'7.4.0': flow<SavedObjectMigrationFn>(migrateSearchSortToNestedArray),
};