Add KQL functionality in the find function of the saved objects

[XavierM]

Summary

The SIEM app is migrating to using SavedObjects which also means migrating all their queries. Since these are written in KQL rewriting them as Simple Query Strings will be time-consuming and error-prone. In addition, Simple Query String doesn't support range queries.

Since KQL is designed as a simple filter language it doesn't replace searching with Simple Query Strings in Saved Objects but rather can be used in addition to constructing more powerful queries.

Dev Docs

SavedObjectsClient.find now supports filtering using a KQL string, with the caveat that if you filter with an attribute from your type saved object, it should look like this: savedObjectType.attributes.name: "SayMyName". However, If you use a direct attribute of a saved object like updatedAt, you will have to define your filter like this: savedObjectType.updatedAt > 2018-12-22.

savedObjectsClient.find({
      type: 'savedObjectType',
      sortField: '@timestamp',
      sortOrder: 'desc',
      search: '',
      searchFields:'',
      fields: ['id', 'name', '@created', '@timestamp'],
      filter:
        'savedObjectType.attributes.name: "SayMyName" and savedObjectType.updatedAt > 2018-12-22'
    });

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

• [ ] This was checked for cross-browser compatibility, including a check against IE11
• [ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials~~
• Unit or functional tests were updated or added to match the most common scenarios
• [ ] This was checked for keyboard-only and screenreader accessibility

For maintainers

• [ ] This was checked for breaking API changes and was labeled appropriately
• [ ] This includes a feature addition or change that requires a release note and was labeled appropriately

[rudolf]

Mike Cote brought up another use case that could be solved by having KQL: https://github.com/elastic/kibana/pull/39829/files#diff-5598f367eaa6aef5c6bbe4654d764a9bR130

[elasticmachine]

Pinging @elastic/kibana-platform

[rudolf]

Left some initial comments

[rudolf]

Edit: added a 3rd option.

@elastic/kibana-platform In order to evaluate KQL expressions the KQL parser requires a StaticIndexPattern which has the following shape:

{
  fields: [
    name: string;
    type: string;
    aggregatable: boolean;
    searchable: boolean;
  ];
  title: string;
}

There are two ways to collect this information:

1. IndexPatternsService src/legacy/server/index_patterns.js which uses ES field capabilities api
2. Construct it from the SavedObject mappings
3. Use the ES field capabilities api directly inside core/saved_objects without moving the whole IndexPatternsService to Core.

If we use (1) it means we'll have to move this service to Core, I haven't seen this discussed, but I assume the current plan is for it to live in the data plugin. (2) is a bit more complex and it means maintaining a field type -> {searchable, aggregatable} mapping instead of leveraging the ES API which is guaranteed to be up to date.

The way @XavierM implemented it now the SavedObjects Repository only requires the compiled static index patterns, so it's easy to swap around how we construct these when we move everything to Core.

[rudolf]

Github stopped letting me add comments 🚫 so submitting what I have.

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 2f2e266

[elasticmachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request
• Commit: 2f2e266