Skip to content

Commit

Permalink
Add KQL functionality in the find function of the saved objects (elas…
Browse files Browse the repository at this point in the history
…tic#41136)

* Add KQL functionality in the find function of the saved objects

wip

rename variable from KQL to filter, fix unit test + add new ones

miss security pluggins

review I

fix api changes

refactor after reviewing with Rudolf

fix type

review III

review IV

for security put back allowed logic back to return empty results

remove StaticIndexPattern

review V

fix core_api_changes

fix type

* validate filter to match requirement type.attributes.key or type.savedObjectKey

* Fix types

* fix a bug + add more api integration test

* fix types in test until we create package @kbn/types

* fix type issue

* fix api integration test

* export nodeTypes from packages @kbn/es-query instead of the function buildNodeKuery

* throw 400- bad request when validation error in find

* fix type issue

* accept api change

* renove _ to represent private

* fix unit test + add doc

* add comment to explain why we removed the private
  • Loading branch information
XavierM committed Oct 3, 2019
1 parent d5595eb commit cad047b
Show file tree
Hide file tree
Showing 42 changed files with 2,459 additions and 152 deletions.
5 changes: 5 additions & 0 deletions docs/api/saved-objects/find.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,11 @@ experimental[] Retrieve a paginated set of {kib} saved objects by various condit
`has_reference`::
(Optional, object) Filters to objects that have a relationship with the type and ID combination.

`filter`::
(Optional, string) The filter is a KQL string with the caveat that if you filter with an attribute from your type saved object.
It should look like that savedObjectType.attributes.title: "myTitle". However, If you used a direct attribute of a saved object like `updatedAt`,
you will have to define your filter like that savedObjectType.updatedAt > 2018-12-22.

NOTE: As objects change in {kib}, the results on each page of the response also
change. Use the find API for traditional paginated results, but avoid using it to export large amounts of data.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@ Search for objects
<b>Signature:</b>

```typescript
find: <T extends SavedObjectAttributes>(options: Pick<SavedObjectFindOptionsServer, "search" | "type" | "defaultSearchOperator" | "searchFields" | "sortField" | "hasReference" | "page" | "perPage" | "fields">) => Promise<SavedObjectsFindResponsePublic<T>>;
find: <T extends SavedObjectAttributes>(options: Pick<SavedObjectFindOptionsServer, "search" | "filter" | "type" | "page" | "perPage" | "sortField" | "fields" | "searchFields" | "hasReference" | "defaultSearchOperator">) => Promise<SavedObjectsFindResponsePublic<T>>;
```
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ export declare class SavedObjectsClient
| [bulkGet](./kibana-plugin-public.savedobjectsclient.bulkget.md) | | <code>(objects?: {</code><br/><code> id: string;</code><br/><code> type: string;</code><br/><code> }[]) =&gt; Promise&lt;SavedObjectsBatchResponse&lt;SavedObjectAttributes&gt;&gt;</code> | Returns an array of objects by id |
| [create](./kibana-plugin-public.savedobjectsclient.create.md) | | <code>&lt;T extends SavedObjectAttributes&gt;(type: string, attributes: T, options?: SavedObjectsCreateOptions) =&gt; Promise&lt;SimpleSavedObject&lt;T&gt;&gt;</code> | Persists an object |
| [delete](./kibana-plugin-public.savedobjectsclient.delete.md) | | <code>(type: string, id: string) =&gt; Promise&lt;{}&gt;</code> | Deletes an object |
| [find](./kibana-plugin-public.savedobjectsclient.find.md) | | <code>&lt;T extends SavedObjectAttributes&gt;(options: Pick&lt;SavedObjectFindOptionsServer, &quot;search&quot; &#124; &quot;type&quot; &#124; &quot;defaultSearchOperator&quot; &#124; &quot;searchFields&quot; &#124; &quot;sortField&quot; &#124; &quot;hasReference&quot; &#124; &quot;page&quot; &#124; &quot;perPage&quot; &#124; &quot;fields&quot;&gt;) =&gt; Promise&lt;SavedObjectsFindResponsePublic&lt;T&gt;&gt;</code> | Search for objects |
| [find](./kibana-plugin-public.savedobjectsclient.find.md) | | <code>&lt;T extends SavedObjectAttributes&gt;(options: Pick&lt;SavedObjectFindOptionsServer, &quot;search&quot; &#124; &quot;filter&quot; &#124; &quot;type&quot; &#124; &quot;page&quot; &#124; &quot;perPage&quot; &#124; &quot;sortField&quot; &#124; &quot;fields&quot; &#124; &quot;searchFields&quot; &#124; &quot;hasReference&quot; &#124; &quot;defaultSearchOperator&quot;&gt;) =&gt; Promise&lt;SavedObjectsFindResponsePublic&lt;T&gt;&gt;</code> | Search for objects |
| [get](./kibana-plugin-public.savedobjectsclient.get.md) | | <code>&lt;T extends SavedObjectAttributes&gt;(type: string, id: string) =&gt; Promise&lt;SimpleSavedObject&lt;T&gt;&gt;</code> | Fetches a single object |

## Methods
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-public](./kibana-plugin-public.md) &gt; [SavedObjectsFindOptions](./kibana-plugin-public.savedobjectsfindoptions.md) &gt; [filter](./kibana-plugin-public.savedobjectsfindoptions.filter.md)

## SavedObjectsFindOptions.filter property

<b>Signature:</b>

```typescript
filter?: string;
```
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ export interface SavedObjectsFindOptions extends SavedObjectsBaseOptions
| --- | --- | --- |
| [defaultSearchOperator](./kibana-plugin-public.savedobjectsfindoptions.defaultsearchoperator.md) | <code>'AND' &#124; 'OR'</code> | |
| [fields](./kibana-plugin-public.savedobjectsfindoptions.fields.md) | <code>string[]</code> | An array of fields to include in the results |
| [filter](./kibana-plugin-public.savedobjectsfindoptions.filter.md) | <code>string</code> | |
| [hasReference](./kibana-plugin-public.savedobjectsfindoptions.hasreference.md) | <code>{</code><br/><code> type: string;</code><br/><code> id: string;</code><br/><code> }</code> | |
| [page](./kibana-plugin-public.savedobjectsfindoptions.page.md) | <code>number</code> | |
| [perPage](./kibana-plugin-public.savedobjectsfindoptions.perpage.md) | <code>number</code> | |
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-server](./kibana-plugin-server.md) &gt; [SavedObjectsFindOptions](./kibana-plugin-server.savedobjectsfindoptions.md) &gt; [filter](./kibana-plugin-server.savedobjectsfindoptions.filter.md)

## SavedObjectsFindOptions.filter property

<b>Signature:</b>

```typescript
filter?: string;
```
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ export interface SavedObjectsFindOptions extends SavedObjectsBaseOptions
| --- | --- | --- |
| [defaultSearchOperator](./kibana-plugin-server.savedobjectsfindoptions.defaultsearchoperator.md) | <code>'AND' &#124; 'OR'</code> | |
| [fields](./kibana-plugin-server.savedobjectsfindoptions.fields.md) | <code>string[]</code> | An array of fields to include in the results |
| [filter](./kibana-plugin-server.savedobjectsfindoptions.filter.md) | <code>string</code> | |
| [hasReference](./kibana-plugin-server.savedobjectsfindoptions.hasreference.md) | <code>{</code><br/><code> type: string;</code><br/><code> id: string;</code><br/><code> }</code> | |
| [page](./kibana-plugin-server.savedobjectsfindoptions.page.md) | <code>number</code> | |
| [perPage](./kibana-plugin-server.savedobjectsfindoptions.perpage.md) | <code>number</code> | |
Expand Down
11 changes: 2 additions & 9 deletions packages/kbn-es-query/src/kuery/ast/ast.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
* under the License.
*/

import { JsonObject } from '..';

/**
* WARNING: these typings are incomplete
*/
Expand All @@ -30,15 +32,6 @@ export interface KueryParseOptions {
startRule: string;
}

type JsonValue = null | boolean | number | string | JsonObject | JsonArray;

interface JsonObject {
[key: string]: JsonValue;
}

// eslint-disable-next-line @typescript-eslint/no-empty-interface
interface JsonArray extends Array<JsonValue> {}

export function fromKueryExpression(
expression: string,
parseOptions?: KueryParseOptions
Expand Down
3 changes: 1 addition & 2 deletions packages/kbn-es-query/src/kuery/functions/is.js
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ export function buildNodeParams(fieldName, value, isPhrase = false) {
if (_.isUndefined(value)) {
throw new Error('value is a required argument');
}

const fieldNode = typeof fieldName === 'string' ? ast.fromLiteralExpression(fieldName) : literal.buildNode(fieldName);
const valueNode = typeof value === 'string' ? ast.fromLiteralExpression(value) : literal.buildNode(value);
const isPhraseNode = literal.buildNode(isPhrase);
Expand All @@ -42,7 +41,7 @@ export function buildNodeParams(fieldName, value, isPhrase = false) {
}

export function toElasticsearchQuery(node, indexPattern = null, config = {}) {
const { arguments: [ fieldNameArg, valueArg, isPhraseArg ] } = node;
const { arguments: [fieldNameArg, valueArg, isPhraseArg] } = node;
const fieldName = ast.toElasticsearchQuery(fieldNameArg);
const value = !_.isUndefined(valueArg) ? ast.toElasticsearchQuery(valueArg) : valueArg;
const type = isPhraseArg.value ? 'phrase' : 'best_fields';
Expand Down
10 changes: 10 additions & 0 deletions packages/kbn-es-query/src/kuery/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,13 @@
*/

export * from './ast';
export { nodeTypes } from './node_types';

export type JsonValue = null | boolean | number | string | JsonObject | JsonArray;

export interface JsonObject {
[key: string]: JsonValue;
}

// eslint-disable-next-line @typescript-eslint/no-empty-interface
export interface JsonArray extends Array<JsonValue> {}
2 changes: 1 addition & 1 deletion packages/kbn-es-query/src/kuery/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,5 @@

export * from './ast';
export * from './filter_migration';
export * from './node_types';
export { nodeTypes } from './node_types';
export * from './errors';
76 changes: 76 additions & 0 deletions packages/kbn-es-query/src/kuery/node_types/index.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

/**
* WARNING: these typings are incomplete
*/

import { JsonObject, JsonValue } from '..';

type FunctionName =
| 'is'
| 'and'
| 'or'
| 'not'
| 'range'
| 'exists'
| 'geoBoundingBox'
| 'geoPolygon';

interface FunctionTypeBuildNode {
type: 'function';
function: FunctionName;
// TODO -> Need to define a better type for DSL query
arguments: any[];
}

interface FunctionType {
buildNode: (functionName: FunctionName, ...args: any[]) => FunctionTypeBuildNode;
buildNodeWithArgumentNodes: (functionName: FunctionName, ...args: any[]) => FunctionTypeBuildNode;
toElasticsearchQuery: (node: any, indexPattern: any, config: JsonObject) => JsonValue;
}

interface LiteralType {
buildNode: (
value: null | boolean | number | string
) => { type: 'literal'; value: null | boolean | number | string };
toElasticsearchQuery: (node: any) => null | boolean | number | string;
}

interface NamedArgType {
buildNode: (name: string, value: any) => { type: 'namedArg'; name: string; value: any };
toElasticsearchQuery: (node: any) => string;
}

interface WildcardType {
buildNode: (value: string) => { type: 'wildcard'; value: string };
test: (node: any, string: string) => boolean;
toElasticsearchQuery: (node: any) => string;
toQueryStringQuery: (node: any) => string;
hasLeadingWildcard: (node: any) => boolean;
}

interface NodeTypes {
function: FunctionType;
literal: LiteralType;
namedArg: NamedArgType;
wildcard: WildcardType;
}

export const nodeTypes: NodeTypes;
2 changes: 1 addition & 1 deletion src/core/public/notifications/notifications_service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ export class NotificationsService {
public setup({ uiSettings }: SetupDeps): NotificationsSetup {
const notificationSetup = { toasts: this.toasts.setup({ uiSettings }) };

this.uiSettingsErrorSubscription = uiSettings.getUpdateErrors$().subscribe(error => {
this.uiSettingsErrorSubscription = uiSettings.getUpdateErrors$().subscribe((error: Error) => {
notificationSetup.toasts.addDanger({
title: i18n.translate('core.notifications.unableUpdateUISettingNotificationMessageTitle', {
defaultMessage: 'Unable to update UI setting',
Expand Down
4 changes: 3 additions & 1 deletion src/core/public/public.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -752,7 +752,7 @@ export class SavedObjectsClient {
}[]) => Promise<SavedObjectsBatchResponse<SavedObjectAttributes>>;
create: <T extends SavedObjectAttributes>(type: string, attributes: T, options?: SavedObjectsCreateOptions) => Promise<SimpleSavedObject<T>>;
delete: (type: string, id: string) => Promise<{}>;
find: <T extends SavedObjectAttributes>(options: Pick<SavedObjectsFindOptions, "search" | "type" | "defaultSearchOperator" | "searchFields" | "sortField" | "hasReference" | "page" | "perPage" | "fields">) => Promise<SavedObjectsFindResponsePublic<T>>;
find: <T extends SavedObjectAttributes>(options: Pick<SavedObjectsFindOptions, "search" | "filter" | "type" | "page" | "perPage" | "sortField" | "fields" | "searchFields" | "hasReference" | "defaultSearchOperator">) => Promise<SavedObjectsFindResponsePublic<T>>;
get: <T extends SavedObjectAttributes>(type: string, id: string) => Promise<SimpleSavedObject<T>>;
update<T extends SavedObjectAttributes>(type: string, id: string, attributes: T, { version, migrationVersion, references }?: SavedObjectsUpdateOptions): Promise<SimpleSavedObject<T>>;
}
Expand All @@ -775,6 +775,8 @@ export interface SavedObjectsFindOptions extends SavedObjectsBaseOptions {
defaultSearchOperator?: 'AND' | 'OR';
fields?: string[];
// (undocumented)
filter?: string;
// (undocumented)
hasReference?: {
type: string;
id: string;
Expand Down
1 change: 1 addition & 0 deletions src/core/public/saved_objects/saved_objects_client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,7 @@ export class SavedObjectsClient {
searchFields: 'search_fields',
sortField: 'sort_field',
type: 'type',
filter: 'filter',
};

const renamedQuery = renameKeys<SavedObjectsFindOptions, any>(renameMap, options);
Expand Down
1 change: 1 addition & 0 deletions src/core/server/saved_objects/service/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ export {
SavedObjectsClientWrapperFactory,
SavedObjectsClientWrapperOptions,
SavedObjectsErrorHelpers,
SavedObjectsCacheIndexPatterns,
} from './lib';

export * from './saved_objects_client';
108 changes: 108 additions & 0 deletions src/core/server/saved_objects/service/lib/cache_index_patterns.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

import { SavedObjectsCacheIndexPatterns } from './cache_index_patterns';

const mockGetFieldsForWildcard = jest.fn();
const mockIndexPatternsService: jest.Mock = jest.fn().mockImplementation(() => ({
getFieldsForWildcard: mockGetFieldsForWildcard,
getFieldsForTimePattern: jest.fn(),
}));

describe('SavedObjectsRepository', () => {
let cacheIndexPatterns: SavedObjectsCacheIndexPatterns;

const fields = [
{
aggregatable: true,
name: 'config.type',
searchable: true,
type: 'string',
},
{
aggregatable: true,
name: 'foo.type',
searchable: true,
type: 'string',
},
{
aggregatable: true,
name: 'bar.type',
searchable: true,
type: 'string',
},
{
aggregatable: true,
name: 'baz.type',
searchable: true,
type: 'string',
},
{
aggregatable: true,
name: 'dashboard.otherField',
searchable: true,
type: 'string',
},
{
aggregatable: true,
name: 'hiddenType.someField',
searchable: true,
type: 'string',
},
];

beforeEach(() => {
cacheIndexPatterns = new SavedObjectsCacheIndexPatterns();
jest.clearAllMocks();
});

it('setIndexPatterns should return an error object when indexPatternsService is undefined', async () => {
try {
await cacheIndexPatterns.setIndexPatterns('test-index');
} catch (error) {
expect(error.message).toMatch('indexPatternsService is not defined');
}
});

it('setIndexPatterns should return an error object if getFieldsForWildcard is not defined', async () => {
mockGetFieldsForWildcard.mockImplementation(() => {
throw new Error('something happen');
});
try {
cacheIndexPatterns.setIndexPatternsService(new mockIndexPatternsService());
await cacheIndexPatterns.setIndexPatterns('test-index');
} catch (error) {
expect(error.message).toMatch('Index Pattern Error - something happen');
}
});

it('setIndexPatterns should return empty array when getFieldsForWildcard is returning null or undefined', async () => {
mockGetFieldsForWildcard.mockImplementation(() => null);
cacheIndexPatterns.setIndexPatternsService(new mockIndexPatternsService());
await cacheIndexPatterns.setIndexPatterns('test-index');
expect(cacheIndexPatterns.getIndexPatterns()).toEqual(undefined);
});

it('setIndexPatterns should return index pattern when getFieldsForWildcard is returning fields', async () => {
mockGetFieldsForWildcard.mockImplementation(() => fields);
cacheIndexPatterns.setIndexPatternsService(new mockIndexPatternsService());
await cacheIndexPatterns.setIndexPatterns('test-index');
expect(cacheIndexPatterns.getIndexPatterns()).toEqual({ fields, title: 'test-index' });
});
});
Loading

0 comments on commit cad047b

Please sign in to comment.