Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add generic typings for SavedObjectMigrationFn #63943

Merged
merged 13 commits into from
Apr 30, 2020
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,14 @@ A migration function for a [saved object type](./kibana-plugin-core-server.saved
<b>Signature:</b>

```typescript
export declare type SavedObjectMigrationFn = (doc: SavedObjectUnsanitizedDoc, context: SavedObjectMigrationContext) => SavedObjectUnsanitizedDoc;
export declare type SavedObjectMigrationFn<InputAttributes = unknown, MigratedAttributes = unknown> = (doc: SavedObjectUnsanitizedDoc<InputAttributes>, context: SavedObjectMigrationContext) => SavedObjectUnsanitizedDoc<MigratedAttributes>;
```

## Example


```typescript
const migrateProperty: SavedObjectMigrationFn = (doc, { log }) => {
const migrateProperty: SavedObjectMigrationFn<MyUnmigratedAttributes, MyMigratedAttributes> = (doc, { log }) => {
if(doc.attributes.someProp === null) {
log.warn('Skipping migration');
} else {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@ Describes Saved Object documents that have passed through the migration framewor
<b>Signature:</b>

```typescript
export declare type SavedObjectSanitizedDoc = SavedObjectDoc & Referencable;
export declare type SavedObjectSanitizedDoc<T = unknown> = SavedObjectDoc<T> & Referencable;
```
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@ Describes Saved Object documents from Kibana &lt; 7.0.0 which don't have a `refe
<b>Signature:</b>

```typescript
export declare type SavedObjectUnsanitizedDoc = SavedObjectDoc & Partial<Referencable>;
export declare type SavedObjectUnsanitizedDoc<T = unknown> = SavedObjectDoc<T> & Partial<Referencable>;
```
2 changes: 1 addition & 1 deletion src/core/server/saved_objects/migrations/core/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,5 @@ export { DocumentMigrator } from './document_migrator';
export { IndexMigrator } from './index_migrator';
export { buildActiveMappings } from './build_active_mappings';
export { CallCluster } from './call_cluster';
export { LogFn } from './migration_logger';
export { LogFn, SavedObjectsMigrationLogger } from './migration_logger';
export { MigrationResult, MigrationStatus } from './migration_coordinator';
43 changes: 43 additions & 0 deletions src/core/server/saved_objects/migrations/mocks.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

import { SavedObjectMigrationContext } from './types';
import { SavedObjectsMigrationLogger } from './core';

const createLoggerMock = (): jest.Mocked<SavedObjectsMigrationLogger> => {
const mock = {
debug: jest.fn(),
info: jest.fn(),
warning: jest.fn(),
warn: jest.fn(),
};

return mock;
};

const createContextMock = (): jest.Mocked<SavedObjectMigrationContext> => {
const mock = {
log: createLoggerMock(),
};
return mock;
};

export const migrationMocks = {
createContext: createContextMock,
};
8 changes: 4 additions & 4 deletions src/core/server/saved_objects/migrations/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import { SavedObjectsMigrationLogger } from './core/migration_logger';
*
* @example
* ```typescript
* const migrateProperty: SavedObjectMigrationFn = (doc, { log }) => {
* const migrateProperty: SavedObjectMigrationFn<MyUnmigratedAttributes, MyMigratedAttributes> = (doc, { log }) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should update the example to show how to construct a copy so that the output is typed different to the input. Since this is only for type safety we don't need to make a deep copy, so it might be worth adding a comment to the example to highlight this.

* if(doc.attributes.someProp === null) {
* log.warn('Skipping migration');
* } else {
Expand All @@ -39,10 +39,10 @@ import { SavedObjectsMigrationLogger } from './core/migration_logger';
*
* @public
*/
export type SavedObjectMigrationFn = (
doc: SavedObjectUnsanitizedDoc,
export type SavedObjectMigrationFn<InputAttributes = unknown, MigratedAttributes = unknown> = (
doc: SavedObjectUnsanitizedDoc<InputAttributes>,
context: SavedObjectMigrationContext
) => SavedObjectUnsanitizedDoc;
) => SavedObjectUnsanitizedDoc<MigratedAttributes>;
Comment on lines +56 to +59
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, my only interrogation around that is that atm migrations are all mutating and returning the input doc object, causing usage of these introduced generics difficult in existing migrations.

I.E

export const migrateToKibana660: SavedObjectMigrationFn<any, any> = doc => {
  if (!doc.attributes.hasOwnProperty('disabledFeatures')) {
    doc.attributes.disabledFeatures = [];
  }
  return doc;
};

Meaning that

type InputAttrs = {
}

type OutputAttrs = {
    disabledFeatures: SomeType[];
}

export const migrateToKibana660: SavedObjectMigrationFn<InputAttrs, OutputAttrs> = doc => {
  if (!doc.attributes.hasOwnProperty('disabledFeatures')) {
    doc.attributes.disabledFeatures = [];
  }
  return doc;
};

Would fail to compile with something like property disabledFeatures does not exists on type InputAttrs on the doc.attributes.disabledFeatures = []; line.

Which is why I'm wondering if we should add the output properties on the input doc:

export type SavedObjectMigrationFn<InputAttributes = unknown, MigratedAttributes = unknown> = (
  doc: SavedObjectUnsanitizedDoc<InputAttributes & MigratedAttributes>,
  context: SavedObjectMigrationContext
) => SavedObjectUnsanitizedDoc<MigratedAttributes>;

Ideally I would have been using Partial, doc: SavedObjectUnsanitizedDoc<InputAttributes & Partial<MigratedAttributes>>, however the given example still fails with disabledFeatures is optional on type SavedObjectUnsanitizedDoc<InputAttributes & Partial<MigratedAttributes>> but required on type SavedObjectUnsanitizedDoc<MigratedAttributes> when returning the doc.

The other option being to decide that migration function should properly spread / construct the resulting migrated object without returning directly the input doc. @rudolf wdyt?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other option being to decide that migration function should properly spread / construct the resulting migrated object without returning directly the input doc.

Given the high impact of even a small cannot read property x of undefined bug in a migration I'd say it validates the bit of extra effort and memory to construct a new output object 👍

We can leave existing migrations with any as they are, but then encourage new migrations to be written in this way. We should update our documentation to show this pattern and probably add this to the developer release notes to create some more visibility so that teams don't stumble on this.

Copy link
Contributor

@rudolf rudolf Apr 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a slightly more type-safe solution than any, teams can still use InputAttributes & MigratedAttributes as their InputAttributes generic paramater.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM, I'll keep the definition as it is right now then.


/**
* Migration context provided when invoking a {@link SavedObjectMigrationFn | migration handler}
Expand Down
2 changes: 2 additions & 0 deletions src/core/server/saved_objects/saved_objects_service.mock.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ import { savedObjectsClientProviderMock } from './service/lib/scoped_client_prov
import { savedObjectsRepositoryMock } from './service/lib/repository.mock';
import { savedObjectsClientMock } from './service/saved_objects_client.mock';
import { typeRegistryMock } from './saved_objects_type_registry.mock';
import { migrationMocks } from './migrations/mocks';
import { ServiceStatusLevels } from '../status';

type SavedObjectsServiceContract = PublicMethodsOf<SavedObjectsService>;
Expand Down Expand Up @@ -105,4 +106,5 @@ export const savedObjectsServiceMock = {
createSetupContract: createSetupContractMock,
createInternalStartContract: createInternalStartContractMock,
createStartContract: createStartContractMock,
createMigrationContext: migrationMocks.createContext,
};
8 changes: 4 additions & 4 deletions src/core/server/saved_objects/serialization/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@ export interface SavedObjectsRawDocSource {
/**
* Saved Object base document
*/
interface SavedObjectDoc {
attributes: any;
interface SavedObjectDoc<T = unknown> {
attributes: T;
id?: string; // NOTE: SavedObjectDoc is used for uncreated objects where `id` is optional
type: string;
namespace?: string;
Expand All @@ -69,12 +69,12 @@ interface Referencable {
*
* @public
*/
export type SavedObjectUnsanitizedDoc = SavedObjectDoc & Partial<Referencable>;
export type SavedObjectUnsanitizedDoc<T = unknown> = SavedObjectDoc<T> & Partial<Referencable>;

/**
* Describes Saved Object documents that have passed through the migration
* framework and are guaranteed to have a `references` root property.
*
* @public
*/
export type SavedObjectSanitizedDoc = SavedObjectDoc & Referencable;
export type SavedObjectSanitizedDoc<T = unknown> = SavedObjectDoc<T> & Referencable;
6 changes: 3 additions & 3 deletions src/core/server/server.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -1680,7 +1680,7 @@ export interface SavedObjectMigrationContext {
}

// @public
export type SavedObjectMigrationFn = (doc: SavedObjectUnsanitizedDoc, context: SavedObjectMigrationContext) => SavedObjectUnsanitizedDoc;
export type SavedObjectMigrationFn<InputAttributes = unknown, MigratedAttributes = unknown> = (doc: SavedObjectUnsanitizedDoc<InputAttributes>, context: SavedObjectMigrationContext) => SavedObjectUnsanitizedDoc<MigratedAttributes>;

// @public
export interface SavedObjectMigrationMap {
Expand Down Expand Up @@ -1708,7 +1708,7 @@ export interface SavedObjectsAddToNamespacesOptions extends SavedObjectsBaseOpti
// Warning: (ae-forgotten-export) The symbol "Referencable" needs to be exported by the entry point index.d.ts
//
// @public
export type SavedObjectSanitizedDoc = SavedObjectDoc & Referencable;
export type SavedObjectSanitizedDoc<T = unknown> = SavedObjectDoc<T> & Referencable;

// @public (undocumented)
export interface SavedObjectsBaseOptions {
Expand Down Expand Up @@ -2309,7 +2309,7 @@ export class SavedObjectTypeRegistry {
}

// @public
export type SavedObjectUnsanitizedDoc = SavedObjectDoc & Partial<Referencable>;
export type SavedObjectUnsanitizedDoc<T = unknown> = SavedObjectDoc<T> & Partial<Referencable>;

// @public
export type ScopeableRequest = KibanaRequest | LegacyRequest | FakeRequest;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,17 @@
*/

import { SavedObjectUnsanitizedDoc } from 'kibana/server';
import { savedObjectsServiceMock } from '../../../../core/server/mocks';
import { dashboardSavedObjectTypeMigrations as migrations } from './dashboard_migrations';

const contextMock = savedObjectsServiceMock.createMigrationContext();

describe('dashboard', () => {
describe('7.0.0', () => {
const migration = migrations['7.0.0'];

test('skips error on empty object', () => {
expect(migration({} as SavedObjectUnsanitizedDoc)).toMatchInlineSnapshot(`
expect(migration({} as SavedObjectUnsanitizedDoc, contextMock)).toMatchInlineSnapshot(`
Object {
"references": Array [],
}
Expand All @@ -44,7 +47,7 @@ Object {
'[{"id":"1","type":"visualization","foo":true},{"id":"2","type":"visualization","bar":true}]',
},
};
const migratedDoc = migration(doc);
const migratedDoc = migration(doc, contextMock);
expect(migratedDoc).toMatchInlineSnapshot(`
Object {
"attributes": Object {
Expand Down Expand Up @@ -83,7 +86,7 @@ Object {
'[{"id":"1","type":"visualization","foo":true},{"id":"2","type":"visualization","bar":true}]',
},
};
const migratedDoc = migration(doc);
const migratedDoc = migration(doc, contextMock);
expect(migratedDoc).toMatchInlineSnapshot(`
Object {
"attributes": Object {
Expand Down Expand Up @@ -122,7 +125,7 @@ Object {
'[{"id":"1","type":"visualization","foo":true},{"id":"2","type":"visualization","bar":true}]',
},
};
expect(migration(doc)).toMatchInlineSnapshot(`
expect(migration(doc, contextMock)).toMatchInlineSnapshot(`
Object {
"attributes": Object {
"kibanaSavedObjectMeta": Object {
Expand Down Expand Up @@ -160,7 +163,7 @@ Object {
'[{"id":"1","type":"visualization","foo":true},{"id":"2","type":"visualization","bar":true}]',
},
};
expect(migration(doc)).toMatchInlineSnapshot(`
expect(migration(doc, contextMock)).toMatchInlineSnapshot(`
Object {
"attributes": Object {
"kibanaSavedObjectMeta": Object {
Expand Down Expand Up @@ -198,7 +201,7 @@ Object {
'[{"id":"1","type":"visualization","foo":true},{"id":"2","type":"visualization","bar":true}]',
},
};
const migratedDoc = migration(doc);
const migratedDoc = migration(doc, contextMock);
expect(migratedDoc).toMatchInlineSnapshot(`
Object {
"attributes": Object {
Expand Down Expand Up @@ -237,7 +240,7 @@ Object {
'[{"id":"1","type":"visualization","foo":true},{"id":"2","type":"visualization","bar":true}]',
},
};
const migratedDoc = migration(doc);
const migratedDoc = migration(doc, contextMock);
expect(migratedDoc).toMatchInlineSnapshot(`
Object {
"attributes": Object {
Expand Down Expand Up @@ -291,7 +294,7 @@ Object {
'[{"id":"1","type":"visualization","foo":true},{"id":"2","type":"visualization","bar":true}]',
},
};
const migratedDoc = migration(doc);
const migratedDoc = migration(doc, contextMock);

expect(migratedDoc).toMatchInlineSnapshot(`
Object {
Expand Down Expand Up @@ -331,7 +334,7 @@ Object {
panelsJSON: 123,
},
} as SavedObjectUnsanitizedDoc;
expect(migration(doc)).toMatchInlineSnapshot(`
expect(migration(doc, contextMock)).toMatchInlineSnapshot(`
Object {
"attributes": Object {
"panelsJSON": 123,
Expand All @@ -349,7 +352,7 @@ Object {
panelsJSON: '{123abc}',
},
} as SavedObjectUnsanitizedDoc;
expect(migration(doc)).toMatchInlineSnapshot(`
expect(migration(doc, contextMock)).toMatchInlineSnapshot(`
Object {
"attributes": Object {
"panelsJSON": "{123abc}",
Expand All @@ -367,7 +370,7 @@ Object {
panelsJSON: '{}',
},
} as SavedObjectUnsanitizedDoc;
expect(migration(doc)).toMatchInlineSnapshot(`
expect(migration(doc, contextMock)).toMatchInlineSnapshot(`
Object {
"attributes": Object {
"panelsJSON": "{}",
Expand All @@ -385,7 +388,7 @@ Object {
panelsJSON: '[{"id":"123"}]',
},
} as SavedObjectUnsanitizedDoc;
expect(migration(doc)).toMatchInlineSnapshot(`
expect(migration(doc, contextMock)).toMatchInlineSnapshot(`
Object {
"attributes": Object {
"panelsJSON": "[{\\"id\\":\\"123\\"}]",
Expand All @@ -403,7 +406,7 @@ Object {
panelsJSON: '[{"type":"visualization"}]',
},
} as SavedObjectUnsanitizedDoc;
expect(migration(doc)).toMatchInlineSnapshot(`
expect(migration(doc, contextMock)).toMatchInlineSnapshot(`
Object {
"attributes": Object {
"panelsJSON": "[{\\"type\\":\\"visualization\\"}]",
Expand All @@ -422,7 +425,7 @@ Object {
'[{"id":"1","type":"visualization","foo":true},{"id":"2","type":"visualization","bar":true}]',
},
} as SavedObjectUnsanitizedDoc;
const migratedDoc = migration(doc);
const migratedDoc = migration(doc, contextMock);
expect(migratedDoc).toMatchInlineSnapshot(`
Object {
"attributes": Object {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

import { get, flow } from 'lodash';

import { SavedObjectMigrationFn, SavedObjectUnsanitizedDoc } from 'kibana/server';
import { SavedObjectMigrationFn } from 'kibana/server';
import { migrations730 } from './migrations_730';
import { migrateMatchAllQuery } from './migrate_match_all_query';
import { DashboardDoc700To720 } from '../../common';
Expand Down Expand Up @@ -62,7 +62,7 @@ function migrateIndexPattern(doc: DashboardDoc700To720) {
doc.attributes.kibanaSavedObjectMeta.searchSourceJSON = JSON.stringify(searchSource);
}

const migrations700: SavedObjectMigrationFn = (doc): DashboardDoc700To720 => {
const migrations700: SavedObjectMigrationFn<any, any> = (doc): DashboardDoc700To720 => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const migrations700: SavedObjectMigrationFn<any, any> = (doc): DashboardDoc700To720 => {
const migrations700: SavedObjectMigrationFn<DashboardDoc700To720['attributes'], DashboardDoc700To720['attributes']> = doc => {

// Set new "references" attribute
doc.references = doc.references || [];

Expand Down Expand Up @@ -111,7 +111,7 @@ export const dashboardSavedObjectTypeMigrations = {
* in that version. So we apply this twice, once with 6.7.2 and once with 7.0.1 while the backport to 6.7
* only contained the 6.7.2 migration and not the 7.0.1 migration.
*/
'6.7.2': flow<SavedObjectMigrationFn>(migrateMatchAllQuery),
'7.0.0': flow<(doc: SavedObjectUnsanitizedDoc) => DashboardDoc700To720>(migrations700),
'7.3.0': flow<SavedObjectMigrationFn>(migrations730),
'6.7.2': flow<SavedObjectMigrationFn<any, any>>(migrateMatchAllQuery),
'7.0.0': flow<SavedObjectMigrationFn<any, DashboardDoc700To720['attributes']>>(migrations700),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elastic/kibana-app DashboardAttributesTo720 exists, but is not exported from bwc/types. I used DashboardDoc700To720['attributes'] instead.

'7.3.0': flow<SavedObjectMigrationFn<any, any>>(migrations730),
Comment on lines +114 to +116
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
'6.7.2': flow<SavedObjectMigrationFn<any, any>>(migrateMatchAllQuery),
'7.0.0': flow<SavedObjectMigrationFn<any, DashboardDoc700To720['attributes']>>(migrations700),
'7.3.0': flow<SavedObjectMigrationFn<any, any>>(migrations730),
'6.7.2': flow<typeof migrateMatchAllQuery>(migrateMatchAllQuery),
'7.0.0': flow<typeof migrations700>(migrations700),
'7.3.0': flow<SavedObjectMigrationFn<any, any>>(migrations730),

We can't follow the same pattern with migrations730 because it's not typed as a SavedObjectMigrationFn.

};
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import { SavedObjectMigrationFn } from 'kibana/server';
import { get } from 'lodash';
import { DEFAULT_QUERY_LANGUAGE } from '../../../data/common';

export const migrateMatchAllQuery: SavedObjectMigrationFn = doc => {
export const migrateMatchAllQuery: SavedObjectMigrationFn<any, any> = doc => {
const searchSourceJSON = get<string>(doc, 'attributes.kibanaSavedObjectMeta.searchSourceJSON');

if (searchSourceJSON) {
Expand Down
14 changes: 4 additions & 10 deletions src/plugins/dashboard/server/saved_objects/migrations_730.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,13 @@
* under the License.
*/

import { savedObjectsServiceMock } from '../../../../core/server/mocks';
import { dashboardSavedObjectTypeMigrations as migrations } from './dashboard_migrations';
import { migrations730 } from './migrations_730';
import { DashboardDoc700To720, DashboardDoc730ToLatest, DashboardDocPre700 } from '../../common';
import { RawSavedDashboardPanel730ToLatest } from '../../common';

const mockContext = {
log: {
warning: () => {},
warn: () => {},
debug: () => {},
info: () => {},
},
};
const mockContext = savedObjectsServiceMock.createMigrationContext();

test('dashboard migration 7.3.0 migrates filters to query on search source', () => {
const doc: DashboardDoc700To720 = {
Expand Down Expand Up @@ -95,7 +89,7 @@ test('dashboard migration 7.3.0 migrates filters to query on search source when
},
};

const doc700: DashboardDoc700To720 = migrations['7.0.0'](doc);
const doc700 = migrations['7.0.0'](doc, mockContext);
Comment on lines -98 to +92
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This migration is now an explicit SavedObjectMigrationFn so the second parameter is now needed in the signature, so I had to adapt the tests.

const newDoc = migrations['7.3.0'](doc700, mockContext);

const parsedSearchSource = JSON.parse(newDoc.attributes.kibanaSavedObjectMeta.searchSourceJSON);
Expand Down Expand Up @@ -127,7 +121,7 @@ test('dashboard migration works when panelsJSON is missing panelIndex', () => {
},
};

const doc700: DashboardDoc700To720 = migrations['7.0.0'](doc);
const doc700 = migrations['7.0.0'](doc, mockContext);
const newDoc = migrations['7.3.0'](doc700, mockContext);

const parsedSearchSource = JSON.parse(newDoc.attributes.kibanaSavedObjectMeta.searchSourceJSON);
Expand Down
Loading