Switch JS runtime caching to sync.Pool #70
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
September 30, 2020 17:56
Just realized a problem, currently we use a loading cache for goja.Runtime caching keyed by ctx. However that means the caching is per transform. We had scenarios in the past where a service handles many concurrent requests, each of which contains a tiny payload that needs to be parsed and transformed. In that situation, the goja.Runtime caching is completely ineffective. Instead, we want to share goja.Runtime across different transforms, as long as javascripts are not called at the same time. Thus the introduction of resource pool. We will adaptively create a number of goja.Runtime in the resource pool, and each of the transforms, concurrent or not, requests a runtime from the pool. We don't want to block, so if there is no available runtime in the pool, while the pool will create a new runtime, it will immediately fail and return. The new execProgram will fallback and create a new runtime. The hope is the resource pool eventually grow to certain size that any concurrent requests would be satisfied. Extensive benchmarks created and verified the new design. With this, now javascript is truly performant enough to fully replace eval.
Codecov Report
@@ Coverage Diff @@
## master #70 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 29 29
Lines 1257 1266 +9
=========================================
+ Hits 1257 1266 +9
Continue to review full report at Codecov.
|
…o set BlockWhenExhausted to false.
jf-tech
changed the title
jf-tech
changed the title
liangxibing
reviewed
Sep 30, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Just realized a problem, currently we use a loading cache for
goja.Runtimecaching keyed byctx. However that means the caching is pertransform. We had scenarios in the past where a service handles many concurrent requests, each of which contains a tiny payload that needs to be parsed and transformed. In that situation, thegoja.Runtimecaching is completely ineffective.Instead, we want to share
goja.Runtimeacross different transforms, as long asjavascript()are not called at the same time. Thus the introduction of resource pool. We will adaptively create a number ofgoja.Runtimein the resource pool, and each of the transforms, concurrent or not, requests a runtime from the pool. The hope is the resource pool eventually grow to certain size that any concurrent requests would be satisfied. The pool is also configured to have a minimum idling runtimes, so that when eviction kicks in, it won't completely clear up the cache.Extensive benchmarks created and verified the new design. With this, now javascript is truly performant enough to fully replace eval.
[update] - given my simple use case of the resource pool: no max limit, no blocking, etc. there is simply no reason to use this full featured resource pool - instead use the much simpler and more performant
sync.Pool. And benchmark proved it's much faster and nowjavascriptis nearly on par witheval; and concurrent performance is much better than the full-fledged resource pool usage.