Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add generic typings for SavedObjectMigrationFn #63943

Merged
merged 13 commits into from
Apr 30, 2020
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,14 @@ A migration function for a [saved object type](./kibana-plugin-core-server.saved
<b>Signature:</b>

```typescript
export declare type SavedObjectMigrationFn = (doc: SavedObjectUnsanitizedDoc, context: SavedObjectMigrationContext) => SavedObjectUnsanitizedDoc;
export declare type SavedObjectMigrationFn<InputAttributes = unknown, MigratedAttributes = unknown> = (doc: SavedObjectUnsanitizedDoc<InputAttributes>, context: SavedObjectMigrationContext) => SavedObjectUnsanitizedDoc<MigratedAttributes>;
```

## Example


```typescript
const migrateProperty: SavedObjectMigrationFn = (doc, { log }) => {
const migrateProperty: SavedObjectMigrationFn<MyUnmigratedAttributes, MyMigratedAttributes> = (doc, { log }) => {
if(doc.attributes.someProp === null) {
log.warn('Skipping migration');
} else {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@
<b>Signature:</b>

```typescript
export declare type SavedObjectSanitizedDoc = SavedObjectDoc & Referencable;
export declare type SavedObjectSanitizedDoc<T = unknown> = SavedObjectDoc<T> & Referencable;
```
8 changes: 4 additions & 4 deletions src/core/server/saved_objects/migrations/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import { SavedObjectsMigrationLogger } from './core/migration_logger';
*
* @example
* ```typescript
* const migrateProperty: SavedObjectMigrationFn = (doc, { log }) => {
* const migrateProperty: SavedObjectMigrationFn<MyUnmigratedAttributes, MyMigratedAttributes> = (doc, { log }) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should update the example to show how to construct a copy so that the output is typed different to the input. Since this is only for type safety we don't need to make a deep copy, so it might be worth adding a comment to the example to highlight this.

* if(doc.attributes.someProp === null) {
* log.warn('Skipping migration');
* } else {
Expand All @@ -39,10 +39,10 @@ import { SavedObjectsMigrationLogger } from './core/migration_logger';
*
* @public
*/
export type SavedObjectMigrationFn = (
doc: SavedObjectUnsanitizedDoc,
export type SavedObjectMigrationFn<InputAttributes = unknown, MigratedAttributes = unknown> = (
doc: SavedObjectUnsanitizedDoc<InputAttributes>,
context: SavedObjectMigrationContext
) => SavedObjectUnsanitizedDoc;
) => SavedObjectUnsanitizedDoc<MigratedAttributes>;
Comment on lines +56 to +59
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, my only interrogation around that is that atm migrations are all mutating and returning the input doc object, causing usage of these introduced generics difficult in existing migrations.

I.E

export const migrateToKibana660: SavedObjectMigrationFn<any, any> = doc => {
  if (!doc.attributes.hasOwnProperty('disabledFeatures')) {
    doc.attributes.disabledFeatures = [];
  }
  return doc;
};

Meaning that

type InputAttrs = {
}

type OutputAttrs = {
    disabledFeatures: SomeType[];
}

export const migrateToKibana660: SavedObjectMigrationFn<InputAttrs, OutputAttrs> = doc => {
  if (!doc.attributes.hasOwnProperty('disabledFeatures')) {
    doc.attributes.disabledFeatures = [];
  }
  return doc;
};

Would fail to compile with something like property disabledFeatures does not exists on type InputAttrs on the doc.attributes.disabledFeatures = []; line.

Which is why I'm wondering if we should add the output properties on the input doc:

export type SavedObjectMigrationFn<InputAttributes = unknown, MigratedAttributes = unknown> = (
  doc: SavedObjectUnsanitizedDoc<InputAttributes & MigratedAttributes>,
  context: SavedObjectMigrationContext
) => SavedObjectUnsanitizedDoc<MigratedAttributes>;

Ideally I would have been using Partial, doc: SavedObjectUnsanitizedDoc<InputAttributes & Partial<MigratedAttributes>>, however the given example still fails with disabledFeatures is optional on type SavedObjectUnsanitizedDoc<InputAttributes & Partial<MigratedAttributes>> but required on type SavedObjectUnsanitizedDoc<MigratedAttributes> when returning the doc.

The other option being to decide that migration function should properly spread / construct the resulting migrated object without returning directly the input doc. @rudolf wdyt?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other option being to decide that migration function should properly spread / construct the resulting migrated object without returning directly the input doc.

Given the high impact of even a small cannot read property x of undefined bug in a migration I'd say it validates the bit of extra effort and memory to construct a new output object 👍

We can leave existing migrations with any as they are, but then encourage new migrations to be written in this way. We should update our documentation to show this pattern and probably add this to the developer release notes to create some more visibility so that teams don't stumble on this.

Copy link
Contributor

@rudolf rudolf Apr 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a slightly more type-safe solution than any, teams can still use InputAttributes & MigratedAttributes as their InputAttributes generic paramater.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM, I'll keep the definition as it is right now then.


/**
* Migration context provided when invoking a {@link SavedObjectMigrationFn | migration handler}
Expand Down
8 changes: 4 additions & 4 deletions src/core/server/saved_objects/serialization/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@ export interface SavedObjectsRawDocSource {
* that future props are likely to be added. Migrations support this
* scenario out of the box.
*/
interface SavedObjectDoc {
attributes: any;
interface SavedObjectDoc<T = unknown> {
attributes: T;
id?: string; // NOTE: SavedObjectDoc is used for uncreated objects where `id` is optional
type: string;
namespace?: string;
Expand All @@ -75,7 +75,7 @@ interface Referencable {
*
* @public
*/
export type SavedObjectUnsanitizedDoc = SavedObjectDoc & Partial<Referencable>;
export type SavedObjectUnsanitizedDoc<T = unknown> = SavedObjectDoc<T> & Partial<Referencable>;

/** @public */
export type SavedObjectSanitizedDoc = SavedObjectDoc & Referencable;
export type SavedObjectSanitizedDoc<T = unknown> = SavedObjectDoc<T> & Referencable;
4 changes: 2 additions & 2 deletions src/core/server/server.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -1682,7 +1682,7 @@ export interface SavedObjectMigrationContext {
// Warning: (ae-forgotten-export) The symbol "SavedObjectUnsanitizedDoc" needs to be exported by the entry point index.d.ts
//
// @public
export type SavedObjectMigrationFn = (doc: SavedObjectUnsanitizedDoc, context: SavedObjectMigrationContext) => SavedObjectUnsanitizedDoc;
export type SavedObjectMigrationFn<InputAttributes = unknown, MigratedAttributes = unknown> = (doc: SavedObjectUnsanitizedDoc<InputAttributes>, context: SavedObjectMigrationContext) => SavedObjectUnsanitizedDoc<MigratedAttributes>;

// @public
export interface SavedObjectMigrationMap {
Expand Down Expand Up @@ -1710,7 +1710,7 @@ export interface SavedObjectsAddToNamespacesOptions extends SavedObjectsBaseOpti
// Warning: (ae-forgotten-export) The symbol "Referencable" needs to be exported by the entry point index.d.ts
//
// @public (undocumented)
export type SavedObjectSanitizedDoc = SavedObjectDoc & Referencable;
export type SavedObjectSanitizedDoc<T = unknown> = SavedObjectDoc<T> & Referencable;

// @public (undocumented)
export interface SavedObjectsBaseOptions {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import { SavedObjectMigrationFn } from 'kibana/server';
import { get } from 'lodash';
import { DEFAULT_QUERY_LANGUAGE } from '../../../../../../plugins/data/common';

export const migrateMatchAllQuery: SavedObjectMigrationFn = doc => {
export const migrateMatchAllQuery: SavedObjectMigrationFn<any, any> = doc => {
const searchSourceJSON = get<string>(doc, 'attributes.kibanaSavedObjectMeta.searchSourceJSON');

if (searchSourceJSON) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
import { flow, omit } from 'lodash';
import { SavedObjectMigrationFn } from 'kibana/server';

const migrateAttributeTypeAndAttributeTypeMeta: SavedObjectMigrationFn = doc => ({
const migrateAttributeTypeAndAttributeTypeMeta: SavedObjectMigrationFn<any, any> = doc => ({
...doc,
attributes: {
...doc.attributes,
Expand All @@ -29,7 +29,7 @@ const migrateAttributeTypeAndAttributeTypeMeta: SavedObjectMigrationFn = doc =>
},
});

const migrateSubTypeAndParentFieldProperties: SavedObjectMigrationFn = doc => {
const migrateSubTypeAndParentFieldProperties: SavedObjectMigrationFn<any, any> = doc => {
if (!doc.attributes.fields) return doc;

const fieldsString = doc.attributes.fields;
Expand Down
14 changes: 7 additions & 7 deletions src/plugins/data/server/saved_objects/search_migrations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import { flow, get } from 'lodash';
import { SavedObjectMigrationFn } from 'kibana/server';
import { DEFAULT_QUERY_LANGUAGE } from '../../common';

const migrateMatchAllQuery: SavedObjectMigrationFn = doc => {
const migrateMatchAllQuery: SavedObjectMigrationFn<any, any> = doc => {
const searchSourceJSON = get<string>(doc, 'attributes.kibanaSavedObjectMeta.searchSourceJSON');

if (searchSourceJSON) {
Expand Down Expand Up @@ -55,7 +55,7 @@ const migrateMatchAllQuery: SavedObjectMigrationFn = doc => {
return doc;
};

const migrateIndexPattern: SavedObjectMigrationFn = doc => {
const migrateIndexPattern: SavedObjectMigrationFn<any, any> = doc => {
const searchSourceJSON = get(doc, 'attributes.kibanaSavedObjectMeta.searchSourceJSON');
if (typeof searchSourceJSON !== 'string') {
return doc;
Expand Down Expand Up @@ -97,13 +97,13 @@ const migrateIndexPattern: SavedObjectMigrationFn = doc => {
return doc;
};

const setNewReferences: SavedObjectMigrationFn = (doc, context) => {
const setNewReferences: SavedObjectMigrationFn<any, any> = (doc, context) => {
doc.references = doc.references || [];
// Migrate index pattern
return migrateIndexPattern(doc, context);
};

const migrateSearchSortToNestedArray: SavedObjectMigrationFn = doc => {
const migrateSearchSortToNestedArray: SavedObjectMigrationFn<any, any> = doc => {
const sort = get(doc, 'attributes.sort');
if (!sort) return doc;

Expand All @@ -122,7 +122,7 @@ const migrateSearchSortToNestedArray: SavedObjectMigrationFn = doc => {
};

export const searchSavedObjectTypeMigrations = {
'6.7.2': flow<SavedObjectMigrationFn>(migrateMatchAllQuery),
'7.0.0': flow<SavedObjectMigrationFn>(setNewReferences),
'7.4.0': flow<SavedObjectMigrationFn>(migrateSearchSortToNestedArray),
'6.7.2': flow<SavedObjectMigrationFn<any, any>>(migrateMatchAllQuery),
'7.0.0': flow<SavedObjectMigrationFn<any, any>>(setNewReferences),
'7.4.0': flow<SavedObjectMigrationFn<any, any>>(migrateSearchSortToNestedArray),
};
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
import { flow } from 'lodash';
import { SavedObjectMigrationFn, SavedObjectsType } from 'kibana/server';

const resetCount: SavedObjectMigrationFn = doc => ({
const resetCount: SavedObjectMigrationFn<any, any> = doc => ({
...doc,
attributes: {
...doc.attributes,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import { SavedObjectMigrationFn } from 'kibana/server';
import { cloneDeep, get, omit, has, flow } from 'lodash';
import { DEFAULT_QUERY_LANGUAGE } from '../../../data/common';

const migrateIndexPattern: SavedObjectMigrationFn = doc => {
const migrateIndexPattern: SavedObjectMigrationFn<any, any> = doc => {
const searchSourceJSON = get(doc, 'attributes.kibanaSavedObjectMeta.searchSourceJSON');
if (typeof searchSourceJSON !== 'string') {
return doc;
Expand Down Expand Up @@ -64,7 +64,7 @@ const migrateIndexPattern: SavedObjectMigrationFn = doc => {
};

// [TSVB] Migrate percentile-rank aggregation (value -> values)
const migratePercentileRankAggregation: SavedObjectMigrationFn = doc => {
const migratePercentileRankAggregation: SavedObjectMigrationFn<any, any> = doc => {
const visStateJSON = get<string>(doc, 'attributes.visState');
let visState;

Expand Down Expand Up @@ -100,7 +100,7 @@ const migratePercentileRankAggregation: SavedObjectMigrationFn = doc => {
};

// [TSVB] Remove stale opperator key
const migrateOperatorKeyTypo: SavedObjectMigrationFn = doc => {
const migrateOperatorKeyTypo: SavedObjectMigrationFn<any, any> = doc => {
const visStateJSON = get<string>(doc, 'attributes.visState');
let visState;

Expand Down Expand Up @@ -132,7 +132,7 @@ const migrateOperatorKeyTypo: SavedObjectMigrationFn = doc => {
};

// Migrate date histogram aggregation (remove customInterval)
const migrateDateHistogramAggregation: SavedObjectMigrationFn = doc => {
const migrateDateHistogramAggregation: SavedObjectMigrationFn<any, any> = doc => {
const visStateJSON = get<string>(doc, 'attributes.visState');
let visState;

Expand Down Expand Up @@ -174,7 +174,7 @@ const migrateDateHistogramAggregation: SavedObjectMigrationFn = doc => {
return doc;
};

const removeDateHistogramTimeZones: SavedObjectMigrationFn = doc => {
const removeDateHistogramTimeZones: SavedObjectMigrationFn<any, any> = doc => {
const visStateJSON = get<string>(doc, 'attributes.visState');
if (visStateJSON) {
let visState;
Expand Down Expand Up @@ -206,7 +206,7 @@ const removeDateHistogramTimeZones: SavedObjectMigrationFn = doc => {

// migrate gauge verticalSplit to alignment
// https://github.com/elastic/kibana/issues/34636
const migrateGaugeVerticalSplitToAlignment: SavedObjectMigrationFn = (doc, logger) => {
const migrateGaugeVerticalSplitToAlignment: SavedObjectMigrationFn<any, any> = (doc, logger) => {
const visStateJSON = get<string>(doc, 'attributes.visState');

if (visStateJSON) {
Expand Down Expand Up @@ -241,7 +241,7 @@ const migrateGaugeVerticalSplitToAlignment: SavedObjectMigrationFn = (doc, logge
Path to the series array is thus:
attributes.visState.
*/
const transformFilterStringToQueryObject: SavedObjectMigrationFn = (doc, logger) => {
const transformFilterStringToQueryObject: SavedObjectMigrationFn<any, any> = (doc, logger) => {
// Migrate filters
// If any filters exist and they are a string, we assume it to be lucene and transform the filter into an object accordingly
const newDoc = cloneDeep(doc);
Expand Down Expand Up @@ -325,7 +325,7 @@ const transformFilterStringToQueryObject: SavedObjectMigrationFn = (doc, logger)
return newDoc;
};

const transformSplitFiltersStringToQueryObject: SavedObjectMigrationFn = doc => {
const transformSplitFiltersStringToQueryObject: SavedObjectMigrationFn<any, any> = doc => {
// Migrate split_filters in TSVB objects that weren't migrated in 7.3
// If any filters exist and they are a string, we assume them to be lucene syntax and transform the filter into an object accordingly
const newDoc = cloneDeep(doc);
Expand Down Expand Up @@ -370,7 +370,7 @@ const transformSplitFiltersStringToQueryObject: SavedObjectMigrationFn = doc =>
return newDoc;
};

const migrateFiltersAggQuery: SavedObjectMigrationFn = doc => {
const migrateFiltersAggQuery: SavedObjectMigrationFn<any, any> = doc => {
const visStateJSON = get<string>(doc, 'attributes.visState');

if (visStateJSON) {
Expand Down Expand Up @@ -402,7 +402,7 @@ const migrateFiltersAggQuery: SavedObjectMigrationFn = doc => {
return doc;
};

const replaceMovAvgToMovFn: SavedObjectMigrationFn = (doc, logger) => {
const replaceMovAvgToMovFn: SavedObjectMigrationFn<any, any> = (doc, logger) => {
const visStateJSON = get<string>(doc, 'attributes.visState');
let visState;

Expand Down Expand Up @@ -450,7 +450,7 @@ const replaceMovAvgToMovFn: SavedObjectMigrationFn = (doc, logger) => {
return doc;
};

const migrateFiltersAggQueryStringQueries: SavedObjectMigrationFn = (doc, logger) => {
const migrateFiltersAggQueryStringQueries: SavedObjectMigrationFn<any, any> = (doc, logger) => {
const visStateJSON = get<string>(doc, 'attributes.visState');

if (visStateJSON) {
Expand Down Expand Up @@ -483,12 +483,12 @@ const migrateFiltersAggQueryStringQueries: SavedObjectMigrationFn = (doc, logger
return doc;
};

const addDocReferences: SavedObjectMigrationFn = doc => ({
const addDocReferences: SavedObjectMigrationFn<any, any> = doc => ({
...doc,
references: doc.references || [],
});

const migrateSavedSearch: SavedObjectMigrationFn = doc => {
const migrateSavedSearch: SavedObjectMigrationFn<any, any> = doc => {
const savedSearchId = get<string>(doc, 'attributes.savedSearchId');

if (savedSearchId && doc.references) {
Expand All @@ -505,7 +505,7 @@ const migrateSavedSearch: SavedObjectMigrationFn = doc => {
return doc;
};

const migrateControls: SavedObjectMigrationFn = doc => {
const migrateControls: SavedObjectMigrationFn<any, any> = doc => {
const visStateJSON = get<string>(doc, 'attributes.visState');

if (visStateJSON) {
Expand Down Expand Up @@ -536,7 +536,7 @@ const migrateControls: SavedObjectMigrationFn = doc => {
return doc;
};

const migrateTableSplits: SavedObjectMigrationFn = doc => {
const migrateTableSplits: SavedObjectMigrationFn<any, any> = doc => {
try {
const visState = JSON.parse(doc.attributes.visState);
if (get(visState, 'type') !== 'table') {
Expand Down Expand Up @@ -572,7 +572,7 @@ const migrateTableSplits: SavedObjectMigrationFn = doc => {
}
};

const migrateMatchAllQuery: SavedObjectMigrationFn = doc => {
const migrateMatchAllQuery: SavedObjectMigrationFn<any, any> = doc => {
const searchSourceJSON = get<string>(doc, 'attributes.kibanaSavedObjectMeta.searchSourceJSON');

if (searchSourceJSON) {
Expand Down Expand Up @@ -606,7 +606,7 @@ const migrateMatchAllQuery: SavedObjectMigrationFn = doc => {
};

// [TSVB] Default color palette is changing, keep the default for older viz
const migrateTsvbDefaultColorPalettes: SavedObjectMigrationFn = doc => {
const migrateTsvbDefaultColorPalettes: SavedObjectMigrationFn<any, any> = doc => {
const visStateJSON = get<string>(doc, 'attributes.visState');
let visState;

Expand Down Expand Up @@ -649,27 +649,30 @@ export const visualizationSavedObjectTypeMigrations = {
* in that version. So we apply this twice, once with 6.7.2 and once with 7.0.1 while the backport to 6.7
* only contained the 6.7.2 migration and not the 7.0.1 migration.
*/
'6.7.2': flow<SavedObjectMigrationFn>(migrateMatchAllQuery, removeDateHistogramTimeZones),
'7.0.0': flow<SavedObjectMigrationFn>(
'6.7.2': flow<SavedObjectMigrationFn<any, any>>(
migrateMatchAllQuery,
removeDateHistogramTimeZones
),
'7.0.0': flow<SavedObjectMigrationFn<any, any>>(
addDocReferences,
migrateIndexPattern,
migrateSavedSearch,
migrateControls,
migrateTableSplits
),
'7.0.1': flow<SavedObjectMigrationFn>(removeDateHistogramTimeZones),
'7.2.0': flow<SavedObjectMigrationFn>(
'7.0.1': flow<SavedObjectMigrationFn<any, any>>(removeDateHistogramTimeZones),
'7.2.0': flow<SavedObjectMigrationFn<any, any>>(
migratePercentileRankAggregation,
migrateDateHistogramAggregation
),
'7.3.0': flow<SavedObjectMigrationFn>(
'7.3.0': flow<SavedObjectMigrationFn<any, any>>(
migrateGaugeVerticalSplitToAlignment,
transformFilterStringToQueryObject,
migrateFiltersAggQuery,
replaceMovAvgToMovFn
),
'7.3.1': flow<SavedObjectMigrationFn>(migrateFiltersAggQueryStringQueries),
'7.4.2': flow<SavedObjectMigrationFn>(transformSplitFiltersStringToQueryObject),
'7.7.0': flow<SavedObjectMigrationFn>(migrateOperatorKeyTypo),
'7.8.0': flow<SavedObjectMigrationFn>(migrateTsvbDefaultColorPalettes),
'7.3.1': flow<SavedObjectMigrationFn<any, any>>(migrateFiltersAggQueryStringQueries),
'7.4.2': flow<SavedObjectMigrationFn<any, any>>(transformSplitFiltersStringToQueryObject),
'7.7.0': flow<SavedObjectMigrationFn<any, any>>(migrateOperatorKeyTypo),
'7.8.0': flow<SavedObjectMigrationFn<any, any>>(migrateTsvbDefaultColorPalettes),
};
3 changes: 2 additions & 1 deletion x-pack/plugins/lens/server/migrations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ interface XYLayerPre77 {
accessors: string[];
}

export const migrations: Record<string, SavedObjectMigrationFn> = {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
export const migrations: Record<string, SavedObjectMigrationFn<any, any>> = {
Comment on lines -134 to +137
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caused by

kibana/.eslintrc.js

Lines 736 to 741 in 40f8222

{
files: ['x-pack/plugins/lens/**/*.{ts,tsx}'],
rules: {
'@typescript-eslint/no-explicit-any': 'error',
},
},

'7.7.0': doc => {
const newDoc = cloneDeep(doc);
if (newDoc.attributes?.visualizationType === 'lnsXY') {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

import { SavedObjectMigrationFn } from 'src/core/server';

export const migrateToKibana660: SavedObjectMigrationFn = doc => {
export const migrateToKibana660: SavedObjectMigrationFn<any, any> = doc => {
if (!doc.attributes.hasOwnProperty('disabledFeatures')) {
doc.attributes.disabledFeatures = [];
}
Expand Down