Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[intepreter][Canvas] Dedupe server functions in batched requests #32712

Conversation

clintandrewhall
Copy link
Contributor

Summary

Currently, if a set of batched function calls are identical, they are all called. For example, if a page calls for demodata in a number of elements, demodata is requested and returned for each element, (ballooning the size of the response).

This PR dedupes the server calls, greatly reducing the payload size.

Before:

screen shot 2019-03-07 at 4 00 00 pm

screen shot 2019-03-07 at 4 00 07 pm

After:

screen shot 2019-03-07 at 3 57 26 pm

screen shot 2019-03-07 at 3 57 34 pm

@clintandrewhall clintandrewhall added review v7.0.0 Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas v8.0.0 canvasGA_0 v6.7.0 labels Mar 7, 2019
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-canvas

@elasticmachine
Copy link
Contributor

💔 Build Failed

Copy link
Contributor

@w33ble w33ble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems right. Adding some tests would certainly be nice. Curious if you're seeing much of a performance gain here... it will probably help with the network side of things even if not.


// Check to see if this is a duplicate server function.
const duplicate = Object.values(batch).find(batchedRequest =>
_.isMatch(batchedRequest.request, request)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like you're using isMatch here instead of isEqual because you're trying to ignore the id param. Is that right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's right.

@w33ble w33ble added the v7.2.0 label Mar 7, 2019
@clintandrewhall
Copy link
Contributor Author

clintandrewhall commented Mar 8, 2019

@w33ble Here's the difference with YDYW: about 600k savings overall. Seems like a good fix... it would be amazing if we could batch that across requests. Had we not decided to start loading pages separately, we probably could have done that.

I'll add some tests.

mar-07-2019 17-59-02

@clintandrewhall
Copy link
Contributor Author

Batching the pages saved another 600k. But the interface got clunky. So caching similar requests from other pages would have the same sized savings as this fix.

screen shot 2019-03-07 at 6 07 00 pm

@w33ble
Copy link
Contributor

w33ble commented Mar 8, 2019

Batching the pages saved another 600k. But the interface got clunky.

You might be able to find a sweet spot by tweaking that timeout in the batched fetch code. 10ms is pretty small, maybe something like 25, 50, or 100 would let you de-dupe more without a noticable lag. It still wouldn't help across multiple pages, but it may speed up inital loading for each one. Just a guess though.

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Copy link
Contributor

@w33ble w33ble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat change, LGTM!

@clintandrewhall clintandrewhall merged commit 41e68eb into elastic:master Mar 8, 2019
clintandrewhall added a commit to clintandrewhall/kibana that referenced this pull request Mar 8, 2019
…stic#32712)

* [intepreter][Canvas] Dedupe server functions in batched requests

* Add and correct tests
clintandrewhall added a commit to clintandrewhall/kibana that referenced this pull request Mar 8, 2019
…stic#32712)

* [intepreter][Canvas] Dedupe server functions in batched requests

* Add and correct tests
clintandrewhall added a commit to clintandrewhall/kibana that referenced this pull request Mar 8, 2019
…stic#32712)

* [intepreter][Canvas] Dedupe server functions in batched requests

* Add and correct tests
clintandrewhall added a commit that referenced this pull request Mar 9, 2019
#32712) (#32833)

* [intepreter][Canvas] Dedupe server functions in batched requests (#32712)

* [intepreter][Canvas] Dedupe server functions in batched requests

* Add and correct tests

* Update interpreter.test.js

Fix merge error

* Delete batched_fetch.test.js

This file was not present in the branch and is failing.
clintandrewhall added a commit that referenced this pull request Mar 9, 2019
#32712) (#32832)

* [intepreter][Canvas] Dedupe server functions in batched requests (#32712)

* [intepreter][Canvas] Dedupe server functions in batched requests

* Add and correct tests

* Delete batched_fetch.test.js

This file was not present in the branch and is failing.
clintandrewhall added a commit that referenced this pull request Mar 9, 2019
) (#32830)

* [intepreter][Canvas] Dedupe server functions in batched requests

* Add and correct tests
clintandrewhall added a commit to clintandrewhall/kibana that referenced this pull request Mar 9, 2019
…stic#32712)

* [intepreter][Canvas] Dedupe server functions in batched requests

* Add and correct tests
clintandrewhall added a commit to clintandrewhall/kibana that referenced this pull request Mar 10, 2019
…stic#32712)

* [intepreter][Canvas] Dedupe server functions in batched requests

* Add and correct tests
clintandrewhall added a commit to clintandrewhall/kibana that referenced this pull request Mar 10, 2019
…stic#32712)

* [intepreter][Canvas] Dedupe server functions in batched requests

* Add and correct tests
@clintandrewhall clintandrewhall deleted the fix-duplicated-batched-requests branch June 6, 2019 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Canvas review Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas v6.7.0 v7.0.0 v7.2.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants