Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SIEM] Adds performance enhancements such by removing wasted renderers and adding incremental DOM rendering #43157

Merged
merged 13 commits into from
Aug 17, 2019

Conversation

FrankHassanabad
Copy link
Contributor

@FrankHassanabad FrankHassanabad commented Aug 12, 2019

Summary

Whenever you wiggle the "resize" bar on the timeline or any other action within the UI which alters the redux store, all the components mapStateProps will and redux will re-render if it sees any of the properties have a shallow compare that returns false.

Ref: https://react-redux.js.org/using-react-redux/connect-mapstate

This PR fixes Super Date Selector which unnecessarily causes re-renders due to not using selectors or memoizations per guidance from the react-redux project.


Super Date Selector will re-render on every single change to the redux store regardless of if it needs to render or not.

Screen Shot 2019-08-12 at 11 49 13 AM

You can see that with something as simple as a slight wiggle of the resizer of timeline causes around 23'ish re-renders

Screen Shot 2019-08-12 at 11 52 54 AM

After the fixes it no longer shows it being re-rendered on a redux change and the number or renders drop significantly when resizing timeline.

It renders only once:
Screen Shot 2019-08-12 at 3 51 20 PM


Some parts of the application are expensive to render and are wasted renders by something such as anonymous functions. By swapping out the pure for React.memo and using the property compare we can cut out those renders and speed up the application considerably.

Before with all the flyout sections causing re-renders when state did not change:
Screen Shot 2019-08-12 at 4 46 29 PM

After with the fix to where the ms is really cheap and the expensive pieces are not rendered unless needed now:
Screen Shot 2019-08-13 at 10 32 55 AM


For the timeline and its properties section it contained a large volume of JSX logic called "PropertiesLeft" and "PropertiesRight" in which both were having re-renders involving expensive calculations such as width changes to the date time when you did something as simple as type in a new title or description into the timeline. I broke those two out into PropertiesLeft and PropertiesRight subsections utilizing React.Memo for performance improvements.

Before when typing a title of timeline, PropertiesRight renders 49+ times:
properties-right-before

Now it renders only 2 times and most of the time it is skipped with something as simple as a title changing:
properties-right-after


When rending feature rich renderers on the timeline it was having a difficult time rendering them all within a reasonable amount of time. This introduces a simple scheduler of incremental loading using requestIdleCallback when it is available from the browser, otherwise this will fall back on a setTimeout as a polyfill/shim.

Ref:
https://developer.mozilla.org/en-US/docs/Web/API/Window/requestIdleCallback
https://developers.google.com/web/updates/2015/08/using-requestidlecallback
https://www.w3.org/TR/requestidlecallback/

Tested on Safari, Firefox, and Chrome. IE-11 looked like it didn't crash :-)

Gif of each of the browsers incrementally rendering the rows:

Chrome:
incremental-load-1
incremental-load-2

Firefox:
firefox-incremental-1
firefox-incremental-2

Safari (polyfil/shim only working):
safari-polyfill-1
safari-polyfill-2

Before Performance picture of the full load. You can see that it will render all rows of 25+ and that causes a lag of up to 2 seconds depending on how complex the renderers are:
Screen Shot 2019-08-13 at 4 24 56 PM

Afterwards you can see it chops it more evenly up between the rendering and tries to get in 5 at a time and maintain a 60 FPS:

Screen Shot 2019-08-14 at 3 59 31 PM

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

- [ ] This was checked for cross-browser compatibility, including a check against IE11

- [ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support

- [ ] Documentation was added for features that require explanation or tutorials

- [ ] This was checked for keyboard-only and screenreader accessibility

For maintainers

@FrankHassanabad FrankHassanabad self-assigned this Aug 12, 2019
@FrankHassanabad FrankHassanabad changed the title Removed extra renderers for super date picker [SIEM] Removes wasted renderers from super date picker for more performance gains Aug 12, 2019
@FrankHassanabad FrankHassanabad marked this pull request as ready for review August 12, 2019 21:54
@elasticmachine
Copy link
Contributor

Pinging @elastic/siem

@elasticmachine

This comment has been minimized.

@elasticmachine

This comment has been minimized.

@elasticmachine

This comment has been minimized.

@FrankHassanabad FrankHassanabad changed the title [SIEM] Removes wasted renderers from super date picker for more performance gains [SIEM] Adds performance enhancements such as removing wasted renderers and incremental DOM rendering Aug 14, 2019
@FrankHassanabad FrankHassanabad changed the title [SIEM] Adds performance enhancements such as removing wasted renderers and incremental DOM rendering [SIEM] Adds performance enhancements such by removing wasted renderers and adding incremental DOM rendering Aug 14, 2019
@elasticmachine

This comment has been minimized.

@elasticmachine

This comment has been minimized.

isFavorite={isFavorite}
timelineId={timelineId}
updateIsFavorite={updateIsFavorite}
showDescription={width >= showDescriptionThreshold}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i appreciate how moving the width threshold check up to this component eliminates the need to propagate width to children, since width updates when the timeline is resized 👍

* 100 milliseconds through the range of .range([100, 2000]).
*
* NOTE: ScaleLog cannot be given a zero, so the domain starting at 1
* like so: domain([1, 100]) is intentional.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@andrew-goldstein andrew-goldstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @FrankHassanabad for all the profiling and time you’re spending in the chrome & react dev tools to improve performance, and especially for the incremental loading you implemented to break up the rendering of events in this PR.

Tested locally in Chrome, FF, and Safari

LGTM 🚀

@elasticmachine

This comment has been minimized.

@spalger
Copy link
Contributor

spalger commented Aug 16, 2019

Please merge master to fix percy failure

@elasticmachine

This comment has been minimized.

@elasticmachine

This comment has been minimized.

@elasticmachine

This comment has been minimized.

@FrankHassanabad
Copy link
Contributor Author

retest this

getUpdatedAt={getUpdatedAt}
width={width}
/>
<TimelineContext.Provider value={{ isLoading: loading }}>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great catch here! ⚾️

@@ -106,7 +104,6 @@ class StatefulBodyComponent extends React.PureComponent<StatefulBodyComponentPro
addNoteToEvent={this.onAddNoteToEvent}
browserFields={browserFields}
id={id}
isLoading={isLoading}
columnHeaders={columnHeaders || []}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional since I don't think we saw an impact while profiling, but perhaps we should replace || [] with a stable reference?

columnHeaders={columnHeaders}
columnRenderers={columnRenderers}
expanded={expanded}
data={data}
Copy link
Contributor

@andrew-goldstein andrew-goldstein Aug 17, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional: move this one line up, and since _id is now id, it can also optionally move down

Copy link
Contributor

@andrew-goldstein andrew-goldstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FrankHassanabad went above and beyond after the initial PR review to add even more performance gains, which required a lot of detective work 🕵️‍♀️ , profiling, and finding perf issues in code that seemed innocuous 🥂

LGTM X2 🚀 🚀

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@FrankHassanabad FrankHassanabad merged commit 9b92f84 into elastic:master Aug 17, 2019
@FrankHassanabad FrankHassanabad deleted the remove-extra-renderers branch August 17, 2019 01:35
FrankHassanabad added a commit to FrankHassanabad/kibana that referenced this pull request Aug 17, 2019
…s and adding incremental DOM rendering (elastic#43157)

## Summary

Whenever you wiggle the "resize" bar on the timeline or any other action within the UI which alters the redux store, all the components `mapStateProps` will and redux will re-render if it sees any of the properties have a shallow compare that returns false.

Ref: https://react-redux.js.org/using-react-redux/connect-mapstate

This PR fixes Super Date Selector which unnecessarily causes re-renders due to not using selectors or memoizations per guidance from the react-redux project.

---

Super Date Selector will re-render on every single change to the redux store regardless of if it needs to render or not.

<img width="2072" alt="Screen Shot 2019-08-12 at 11 49 13 AM" src="https://user-images.githubusercontent.com/1151048/62900648-eaa39f80-bd17-11e9-91fd-ed4b1e81e27b.png">

You can see that with something as simple as a slight wiggle of the resizer of timeline causes around 23'ish re-renders
 
<img width="2338" alt="Screen Shot 2019-08-12 at 11 52 54 AM" src="https://user-images.githubusercontent.com/1151048/62900666-f7c08e80-bd17-11e9-8ddd-77093553aaf9.png">

After the fixes it no longer shows it being re-rendered on a redux change and the number or renders drop significantly when resizing timeline.

It renders only once: 
<img width="1857" alt="Screen Shot 2019-08-12 at 3 51 20 PM" src="https://user-images.githubusercontent.com/1151048/62901080-1c693600-bd19-11e9-8f8e-f19df029dd31.png">

---
Some parts of the application are expensive to render and are wasted renders by something such as anonymous functions. By swapping out the `pure` for `React.memo` and using the property compare we can cut out those renders and speed up the application considerably.

Before with all the flyout sections causing re-renders when state did not change:
<img width="1718" alt="Screen Shot 2019-08-12 at 4 46 29 PM" src="https://user-images.githubusercontent.com/1151048/62959566-589ea400-bdb6-11e9-89f1-047c268e74fd.png">

After with the fix to where the ms is really cheap and the expensive pieces are not rendered unless needed now:
<img width="1682" alt="Screen Shot 2019-08-13 at 10 32 55 AM" src="https://user-images.githubusercontent.com/1151048/62959667-82f06180-bdb6-11e9-9cc9-c9c609a0fb71.png">

---
For the timeline and its properties section it contained a large volume of JSX logic called "PropertiesLeft" and "PropertiesRight" in which both were having re-renders involving expensive calculations such as width changes to the date time when you did something as simple as type in a new title or description into the timeline. I broke those two out into PropertiesLeft and PropertiesRight subsections utilizing `React.Memo` for performance improvements.

Before when typing a title of timeline, PropertiesRight renders 49+ times:
<img width="2030" alt="properties-right-before" src="https://user-images.githubusercontent.com/1151048/63056285-f7093300-bea4-11e9-8f33-a2e63d0468ac.png">

Now it renders only 2 times and most of the time it is skipped with something as simple as a title changing:
<img width="2020" alt="properties-right-after" src="https://user-images.githubusercontent.com/1151048/63056339-10aa7a80-bea5-11e9-9c0c-ae12be5b5851.png">

---

When rending feature rich renderers on the timeline it was having a difficult time rendering them all within a reasonable amount of time. This introduces a simple scheduler of incremental loading using `requestIdleCallback` when it is available from the browser, otherwise this will fall back on a `setTimeout` as a polyfill/shim.

Ref:
https://developer.mozilla.org/en-US/docs/Web/API/Window/requestIdleCallback
https://developers.google.com/web/updates/2015/08/using-requestidlecallback
https://www.w3.org/TR/requestidlecallback/

Tested on Safari, Firefox, and Chrome. IE-11 looked like it didn't crash :-)

Gif of each of the browsers incrementally rendering the rows:

Chrome:
![incremental-load-1](https://user-images.githubusercontent.com/1151048/63057036-8fec7e00-bea6-11e9-83f9-7bad3f79746f.gif)
![incremental-load-2](https://user-images.githubusercontent.com/1151048/63057043-92e76e80-bea6-11e9-8f0d-95ea2559b6c7.gif)

Firefox:
![firefox-incremental-1](https://user-images.githubusercontent.com/1151048/63057052-97138c00-bea6-11e9-8e15-19f5b7cb0a5e.gif)
![firefox-incremental-2](https://user-images.githubusercontent.com/1151048/63057060-98dd4f80-bea6-11e9-9bf3-c8ae798db9e4.gif)

Safari (polyfil/shim only working):
![safari-polyfill-1](https://user-images.githubusercontent.com/1151048/63057099-adb9e300-bea6-11e9-9b94-32a08b5905ad.gif)
![safari-polyfill-2](https://user-images.githubusercontent.com/1151048/63057101-b01c3d00-bea6-11e9-9261-77444272992d.gif)


Before Performance picture of the full load. You can see that it will render all rows of 25+ and that causes a lag of up to 2 seconds depending on how complex the renderers are:
<img width="1346" alt="Screen Shot 2019-08-13 at 4 24 56 PM" src="https://user-images.githubusercontent.com/1151048/63059739-77339680-bead-11e9-818a-ee19df596855.png">


Afterwards you can see it chops it more evenly up between the rendering and tries to get in 5 at a time and maintain a 60 FPS:

<img width="691" alt="Screen Shot 2019-08-14 at 3 59 31 PM" src="https://user-images.githubusercontent.com/1151048/63059764-83b7ef00-bead-11e9-84da-bf66de8a35e6.png">



 
### Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~

~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~

~~- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~

### For maintainers

- [x] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
- [x] This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
FrankHassanabad added a commit that referenced this pull request Aug 17, 2019
…s and adding incremental DOM rendering (#43157) (#43500)

## Summary

Whenever you wiggle the "resize" bar on the timeline or any other action within the UI which alters the redux store, all the components `mapStateProps` will and redux will re-render if it sees any of the properties have a shallow compare that returns false.

Ref: https://react-redux.js.org/using-react-redux/connect-mapstate

This PR fixes Super Date Selector which unnecessarily causes re-renders due to not using selectors or memoizations per guidance from the react-redux project.

---

Super Date Selector will re-render on every single change to the redux store regardless of if it needs to render or not.

<img width="2072" alt="Screen Shot 2019-08-12 at 11 49 13 AM" src="https://user-images.githubusercontent.com/1151048/62900648-eaa39f80-bd17-11e9-91fd-ed4b1e81e27b.png">

You can see that with something as simple as a slight wiggle of the resizer of timeline causes around 23'ish re-renders
 
<img width="2338" alt="Screen Shot 2019-08-12 at 11 52 54 AM" src="https://user-images.githubusercontent.com/1151048/62900666-f7c08e80-bd17-11e9-8ddd-77093553aaf9.png">

After the fixes it no longer shows it being re-rendered on a redux change and the number or renders drop significantly when resizing timeline.

It renders only once: 
<img width="1857" alt="Screen Shot 2019-08-12 at 3 51 20 PM" src="https://user-images.githubusercontent.com/1151048/62901080-1c693600-bd19-11e9-8f8e-f19df029dd31.png">

---
Some parts of the application are expensive to render and are wasted renders by something such as anonymous functions. By swapping out the `pure` for `React.memo` and using the property compare we can cut out those renders and speed up the application considerably.

Before with all the flyout sections causing re-renders when state did not change:
<img width="1718" alt="Screen Shot 2019-08-12 at 4 46 29 PM" src="https://user-images.githubusercontent.com/1151048/62959566-589ea400-bdb6-11e9-89f1-047c268e74fd.png">

After with the fix to where the ms is really cheap and the expensive pieces are not rendered unless needed now:
<img width="1682" alt="Screen Shot 2019-08-13 at 10 32 55 AM" src="https://user-images.githubusercontent.com/1151048/62959667-82f06180-bdb6-11e9-9cc9-c9c609a0fb71.png">

---
For the timeline and its properties section it contained a large volume of JSX logic called "PropertiesLeft" and "PropertiesRight" in which both were having re-renders involving expensive calculations such as width changes to the date time when you did something as simple as type in a new title or description into the timeline. I broke those two out into PropertiesLeft and PropertiesRight subsections utilizing `React.Memo` for performance improvements.

Before when typing a title of timeline, PropertiesRight renders 49+ times:
<img width="2030" alt="properties-right-before" src="https://user-images.githubusercontent.com/1151048/63056285-f7093300-bea4-11e9-8f33-a2e63d0468ac.png">

Now it renders only 2 times and most of the time it is skipped with something as simple as a title changing:
<img width="2020" alt="properties-right-after" src="https://user-images.githubusercontent.com/1151048/63056339-10aa7a80-bea5-11e9-9c0c-ae12be5b5851.png">

---

When rending feature rich renderers on the timeline it was having a difficult time rendering them all within a reasonable amount of time. This introduces a simple scheduler of incremental loading using `requestIdleCallback` when it is available from the browser, otherwise this will fall back on a `setTimeout` as a polyfill/shim.

Ref:
https://developer.mozilla.org/en-US/docs/Web/API/Window/requestIdleCallback
https://developers.google.com/web/updates/2015/08/using-requestidlecallback
https://www.w3.org/TR/requestidlecallback/

Tested on Safari, Firefox, and Chrome. IE-11 looked like it didn't crash :-)

Gif of each of the browsers incrementally rendering the rows:

Chrome:
![incremental-load-1](https://user-images.githubusercontent.com/1151048/63057036-8fec7e00-bea6-11e9-83f9-7bad3f79746f.gif)
![incremental-load-2](https://user-images.githubusercontent.com/1151048/63057043-92e76e80-bea6-11e9-8f0d-95ea2559b6c7.gif)

Firefox:
![firefox-incremental-1](https://user-images.githubusercontent.com/1151048/63057052-97138c00-bea6-11e9-8e15-19f5b7cb0a5e.gif)
![firefox-incremental-2](https://user-images.githubusercontent.com/1151048/63057060-98dd4f80-bea6-11e9-9bf3-c8ae798db9e4.gif)

Safari (polyfil/shim only working):
![safari-polyfill-1](https://user-images.githubusercontent.com/1151048/63057099-adb9e300-bea6-11e9-9b94-32a08b5905ad.gif)
![safari-polyfill-2](https://user-images.githubusercontent.com/1151048/63057101-b01c3d00-bea6-11e9-9261-77444272992d.gif)


Before Performance picture of the full load. You can see that it will render all rows of 25+ and that causes a lag of up to 2 seconds depending on how complex the renderers are:
<img width="1346" alt="Screen Shot 2019-08-13 at 4 24 56 PM" src="https://user-images.githubusercontent.com/1151048/63059739-77339680-bead-11e9-818a-ee19df596855.png">


Afterwards you can see it chops it more evenly up between the rendering and tries to get in 5 at a time and maintain a 60 FPS:

<img width="691" alt="Screen Shot 2019-08-14 at 3 59 31 PM" src="https://user-images.githubusercontent.com/1151048/63059764-83b7ef00-bead-11e9-84da-bf66de8a35e6.png">



 
### Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~

~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~

~~- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~

### For maintainers

- [x] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
- [x] This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
jloleysens added a commit to jloleysens/kibana that referenced this pull request Aug 19, 2019
…_update_json_spec

* 'master' of github.com:elastic/kibana: (35 commits)
  fix: 🐛 pass whole action context to isCompatible() method (elastic#43457)
  Deleted old kbn-top-nav directive (elastic#43168)
  [ML] Fixing cloning of single metric distinct count job (elastic#43435)
  Update @elastic/charts version 8.1.6 > 9.1.1 (elastic#43516)
  [Inspector Views] [Request View] - Migrate inspector_views to new platform (elastic#43191)
  [ML] Adding loading indicators to all wizard charts (elastic#43382)
  disable flaky test (elastic#43492)
  feature(code/frontend): cancel file blob and directory commits request if outdated (elastic#43348)
  fix(code/frontend): button group url should have previous query string (elastic#43428)
  [SIEM] Fixes index substring incorrectly matching configured indices and failing to install ML job (elastic#43409)
  [SIEM] Adds performance enhancements such by removing wasted renderers and adding incremental DOM rendering (elastic#43157)
  disable flaky test (elastic#37859)
  Added sass lint to Canvas (elastic#43410)
  [Maps] add indicator when layer is filtered by search bar (elastic#43283)
  Properly validate current user password during password change. (elastic#43447)
  Spaces - allow for hex color codes that include uppercase characters (elastic#43470)
  [Reporting] Add a bit more logging and a few more logging level promotions (elastic#43415)
  Partially convert index pattern server to typescript (elastic#43291)
  [Infra UI] Use sum for aggregating AWS metrics. (elastic#43293)
  [SIEM] Format bytes columns in timeline (elastic#43147)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants