Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Lens] Clear out all attribute properties before updating #74483

Merged
merged 5 commits into from
Aug 10, 2020

Conversation

flash1293
Copy link
Contributor

@flash1293 flash1293 commented Aug 6, 2020

Fixes #72855

Lens uses an unindexed object type to store its attributes. This has some advantages, but becomes a problem when updating, because when doing an update, Elasticsearch is merging objects instead of replacing them (https://discuss.elastic.co/t/updating-an-object-field/110735). This means if there are objects used as maps with unknown keys, they will stay in the saved object even though they were deleted from the update document.

E.g. if the following document is stored:

{
  "attributes": {
    "myMap": {
      "a": 1,
      "b": 2,
    }
  }
}

and

savedObjectsClient.update('mydoc', '1234', {
  myMap: {
    b: 5
  }
})

then the resulting document will look like this:

{
  "attributes": {
    "myMap": {
      "a": 1,
      "b": 5,
    }
  }
}

This PR fixes the problem for Lens by doing a two-phased update setting all keys to null before updating them with the actual new attributes using the bulk update API. The order of these updates is guaranteed (https://discuss.elastic.co/t/order-of--bulk-request-operations/98124).

This PR also removes some local types which were in place because the core service wasn't typed when the code was written.

@flash1293 flash1293 marked this pull request as ready for review August 6, 2020 14:38
@flash1293 flash1293 requested a review from a team August 6, 2020 14:38
@flash1293 flash1293 added this to In progress in Lens via automation Aug 6, 2020
update: jest.fn((_type: string, id: string) => Promise.resolve({ id })),
bulkUpdate: jest.fn(([{ id }]: SavedObjectsBulkUpdateObject[]) =>
Promise.resolve({ savedObjects: [{ id }, { id }] })
),
Copy link
Contributor

@wylieconlon wylieconlon Aug 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using bulkUpdate, is it more correct to use index? I'm basing this off the docs which say that "update" will perform a merge, but "index" rewrites the whole thing. https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html

It also appears that index is a method supported in the repository_es_client.ts but not necessarily exported by the types

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes a lot of sense, thanks. I would like to get a fix out without waiting for changed core apis, do you think this approach is fine as a stopgap measure while working with the platform team to get a better solution?

Copy link
Contributor

@wylieconlon wylieconlon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested in safari, and this change definitely fixes the behavior. This is a good fix.

@flash1293
Copy link
Contributor Author

@elasticmachine merge upstream

@flash1293
Copy link
Contributor Author

@elasticmachine merge upstream

1 similar comment
@flash1293
Copy link
Contributor Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Build metrics

page load bundle size

id value diff baseline
lens 843.9KB +717.0B 843.2KB

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@flash1293 flash1293 merged commit c1b55d5 into elastic:master Aug 10, 2020
Lens automation moved this from In progress to Done Aug 10, 2020
flash1293 added a commit to flash1293/kibana that referenced this pull request Aug 10, 2020
gmmorris added a commit to gmmorris/kibana that referenced this pull request Aug 11, 2020
…-task

* master: (42 commits)
  Allow any hostname for chromium proxy bypass (elastic#74693)
  [ML] ML on Kibana Management: Add ability to pass a group ID filter to job management page (elastic#74533)
  [Metrics UI] Fix No Data preview pluralization (elastic#74399)
  [Bug][Security_Solution][Telemetry] Capitalize S in macOS (elastic#74688)
  Remove karma tests  from legacy maps (elastic#74668)
  [Ingest Manager] stop creating events-* index pattern and placeholder index (elastic#74683)
  [Enterprise Search] Update the browser/document title on plugin navigation (elastic#74392)
  [visualizations] Add i18n translation for 'No results found' (elastic#74619)
  [maps] convert vector style properties to TS (elastic#74553)
  bump geckodriver binary to 0.27 (elastic#74638)
  fix: update apm agents to catch abort requests (elastic#74658)
  [Security Solution] Resolver children pagination (elastic#74603)
  add memoryStatus to df analytics page and analytics table in management (elastic#74570)
  [Ingest Manager] Allow prerelease in package version (elastic#74452)
  [App Arch]: remove legacy karma tests (elastic#74599)
  [i18n] revert reverted changes (elastic#74633)
  [Lens] Clear out all attribute properties before updating (elastic#74483)
  [Uptime] Fix full reloads while navigating to alert/ml (elastic#73796)
  Index pattern field class refactor (elastic#73180)
  [ML] Functional tests - stabilize DFA job type check (elastic#74631)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Lens] Indexpattern datasource persistence issue layer not cleaned up
4 participants