Cherry-pick #13857 to 7.4: Make script processor Run method concurrency safe #13899
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This should make the script processor safe to be used across multiple goroutines. It compiles the script once. And loads it into a pool of VMs. Each invocation of Run() grabs a VM instance from the pool and returns it when complete. Fixes elastic#13690 ``` $ benchcmp master.txt pool.txt benchmark old ns/op new ns/op delta BenchmarkBeatEventV0/Put-12 2271 2597 +14.35% BenchmarkBeatEventV0/timeout_Put-12 2449 2783 +13.64% BenchmarkBeatEventV0/Object_Put_Key-12 2155 2374 +10.16% BenchmarkBeatEventV0/timeout_Object_Put_Key-12 2365 2602 +10.02% BenchmarkBeatEventV0/Get-12 2315 2534 +9.46% BenchmarkBeatEventV0/timeout_Get-12 2480 2830 +14.11% BenchmarkBeatEventV0/Get_Object-12 2439 2872 +17.75% BenchmarkBeatEventV0/timeout_Get_Object-12 2669 3079 +15.36% BenchmarkBeatEventV0/Get_Undefined_Key-12 2588 2919 +12.79% BenchmarkBeatEventV0/timeout_Get_Undefined_Key-12 2789 3128 +12.15% BenchmarkBeatEventV0/fields_get_key-12 2382 2722 +14.27% BenchmarkBeatEventV0/timeout_fields_get_key-12 2663 2933 +10.14% BenchmarkBeatEventV0/Get_@metadata-12 2291 2460 +7.38% BenchmarkBeatEventV0/timeout_Get_@metadata-12 2515 2697 +7.24% BenchmarkBeatEventV0/Put_@metadata-12 2662 2492 -6.39% BenchmarkBeatEventV0/timeout_Put_@metadata-12 3038 2722 -10.40% BenchmarkBeatEventV0/Delete_@metadata-12 2229 2420 +8.57% BenchmarkBeatEventV0/timeout_Delete_@metadata-12 2452 2653 +8.20% BenchmarkBeatEventV0/Cancel-12 2274 2319 +1.98% BenchmarkBeatEventV0/timeout_Cancel-12 3198 2576 -19.45% BenchmarkBeatEventV0/Tag-12 2925 2668 -8.79% BenchmarkBeatEventV0/timeout_Tag-12 3107 2877 -7.40% BenchmarkBeatEventV0/AppendTo-12 2496 2428 -2.72% BenchmarkBeatEventV0/timeout_AppendTo-12 2570 2721 +5.88% benchmark old allocs new allocs delta BenchmarkBeatEventV0/Put-12 30 30 +0.00% BenchmarkBeatEventV0/timeout_Put-12 32 32 +0.00% BenchmarkBeatEventV0/Object_Put_Key-12 29 29 +0.00% BenchmarkBeatEventV0/timeout_Object_Put_Key-12 31 31 +0.00% BenchmarkBeatEventV0/Get-12 29 29 +0.00% BenchmarkBeatEventV0/timeout_Get-12 31 31 +0.00% BenchmarkBeatEventV0/Get_Object-12 32 32 +0.00% BenchmarkBeatEventV0/timeout_Get_Object-12 34 34 +0.00% BenchmarkBeatEventV0/Get_Undefined_Key-12 35 35 +0.00% BenchmarkBeatEventV0/timeout_Get_Undefined_Key-12 37 37 +0.00% BenchmarkBeatEventV0/fields_get_key-12 32 32 +0.00% BenchmarkBeatEventV0/timeout_fields_get_key-12 34 34 +0.00% BenchmarkBeatEventV0/Get_@metadata-12 29 29 +0.00% BenchmarkBeatEventV0/timeout_Get_@metadata-12 31 31 +0.00% BenchmarkBeatEventV0/Put_@metadata-12 30 30 +0.00% BenchmarkBeatEventV0/timeout_Put_@metadata-12 32 32 +0.00% BenchmarkBeatEventV0/Delete_@metadata-12 28 28 +0.00% BenchmarkBeatEventV0/timeout_Delete_@metadata-12 30 30 +0.00% BenchmarkBeatEventV0/Cancel-12 28 28 +0.00% BenchmarkBeatEventV0/timeout_Cancel-12 30 30 +0.00% BenchmarkBeatEventV0/Tag-12 28 28 +0.00% BenchmarkBeatEventV0/timeout_Tag-12 30 30 +0.00% BenchmarkBeatEventV0/AppendTo-12 28 28 +0.00% BenchmarkBeatEventV0/timeout_AppendTo-12 30 30 +0.00% benchmark old bytes new bytes delta BenchmarkBeatEventV0/Put-12 2112 2114 +0.09% BenchmarkBeatEventV0/timeout_Put-12 2208 2210 +0.09% BenchmarkBeatEventV0/Object_Put_Key-12 2096 2098 +0.10% BenchmarkBeatEventV0/timeout_Object_Put_Key-12 2192 2194 +0.09% BenchmarkBeatEventV0/Get-12 2096 2098 +0.10% BenchmarkBeatEventV0/timeout_Get-12 2192 2194 +0.09% BenchmarkBeatEventV0/Get_Object-12 2256 2258 +0.09% BenchmarkBeatEventV0/timeout_Get_Object-12 2352 2354 +0.09% BenchmarkBeatEventV0/Get_Undefined_Key-12 2416 2419 +0.12% BenchmarkBeatEventV0/timeout_Get_Undefined_Key-12 2512 2514 +0.08% BenchmarkBeatEventV0/fields_get_key-12 2256 2258 +0.09% BenchmarkBeatEventV0/timeout_fields_get_key-12 2352 2354 +0.09% BenchmarkBeatEventV0/Get_@metadata-12 2096 2098 +0.10% BenchmarkBeatEventV0/timeout_Get_@metadata-12 2192 2194 +0.09% BenchmarkBeatEventV0/Put_@metadata-12 2112 2114 +0.09% BenchmarkBeatEventV0/timeout_Put_@metadata-12 2208 2210 +0.09% BenchmarkBeatEventV0/Delete_@metadata-12 2080 2082 +0.10% BenchmarkBeatEventV0/timeout_Delete_@metadata-12 2176 2178 +0.09% BenchmarkBeatEventV0/Cancel-12 2080 2082 +0.10% BenchmarkBeatEventV0/timeout_Cancel-12 2176 2178 +0.09% BenchmarkBeatEventV0/Tag-12 2080 2082 +0.10% BenchmarkBeatEventV0/timeout_Tag-12 2176 2178 +0.09% BenchmarkBeatEventV0/AppendTo-12 2080 2082 +0.10% BenchmarkBeatEventV0/timeout_AppendTo-12 2176 2178 +0.09% ``` (cherry picked from commit 0338e66)
andrewkroh
force-pushed
the
backport_13857_7.4
branch
from
76a893a
to
b0c77ca
Compare
adriansr
approved these changes
Oct 7, 2019
leweafan
pushed a commit
to leweafan/beats
that referenced
this pull request
Apr 28, 2023
…astic#13899) This should make the script processor safe to be used across multiple goroutines. It compiles the script once. And loads it into a pool of VMs. Each invocation of Run() grabs a VM instance from the pool and returns it when complete. Fixes elastic#13690 ``` $ benchcmp master.txt pool.txt benchmark old ns/op new ns/op delta BenchmarkBeatEventV0/Put-12 2271 2597 +14.35% BenchmarkBeatEventV0/timeout_Put-12 2449 2783 +13.64% BenchmarkBeatEventV0/Object_Put_Key-12 2155 2374 +10.16% BenchmarkBeatEventV0/timeout_Object_Put_Key-12 2365 2602 +10.02% BenchmarkBeatEventV0/Get-12 2315 2534 +9.46% BenchmarkBeatEventV0/timeout_Get-12 2480 2830 +14.11% BenchmarkBeatEventV0/Get_Object-12 2439 2872 +17.75% BenchmarkBeatEventV0/timeout_Get_Object-12 2669 3079 +15.36% BenchmarkBeatEventV0/Get_Undefined_Key-12 2588 2919 +12.79% BenchmarkBeatEventV0/timeout_Get_Undefined_Key-12 2789 3128 +12.15% BenchmarkBeatEventV0/fields_get_key-12 2382 2722 +14.27% BenchmarkBeatEventV0/timeout_fields_get_key-12 2663 2933 +10.14% BenchmarkBeatEventV0/Get_@metadata-12 2291 2460 +7.38% BenchmarkBeatEventV0/timeout_Get_@metadata-12 2515 2697 +7.24% BenchmarkBeatEventV0/Put_@metadata-12 2662 2492 -6.39% BenchmarkBeatEventV0/timeout_Put_@metadata-12 3038 2722 -10.40% BenchmarkBeatEventV0/Delete_@metadata-12 2229 2420 +8.57% BenchmarkBeatEventV0/timeout_Delete_@metadata-12 2452 2653 +8.20% BenchmarkBeatEventV0/Cancel-12 2274 2319 +1.98% BenchmarkBeatEventV0/timeout_Cancel-12 3198 2576 -19.45% BenchmarkBeatEventV0/Tag-12 2925 2668 -8.79% BenchmarkBeatEventV0/timeout_Tag-12 3107 2877 -7.40% BenchmarkBeatEventV0/AppendTo-12 2496 2428 -2.72% BenchmarkBeatEventV0/timeout_AppendTo-12 2570 2721 +5.88% benchmark old allocs new allocs delta BenchmarkBeatEventV0/Put-12 30 30 +0.00% BenchmarkBeatEventV0/timeout_Put-12 32 32 +0.00% BenchmarkBeatEventV0/Object_Put_Key-12 29 29 +0.00% BenchmarkBeatEventV0/timeout_Object_Put_Key-12 31 31 +0.00% BenchmarkBeatEventV0/Get-12 29 29 +0.00% BenchmarkBeatEventV0/timeout_Get-12 31 31 +0.00% BenchmarkBeatEventV0/Get_Object-12 32 32 +0.00% BenchmarkBeatEventV0/timeout_Get_Object-12 34 34 +0.00% BenchmarkBeatEventV0/Get_Undefined_Key-12 35 35 +0.00% BenchmarkBeatEventV0/timeout_Get_Undefined_Key-12 37 37 +0.00% BenchmarkBeatEventV0/fields_get_key-12 32 32 +0.00% BenchmarkBeatEventV0/timeout_fields_get_key-12 34 34 +0.00% BenchmarkBeatEventV0/Get_@metadata-12 29 29 +0.00% BenchmarkBeatEventV0/timeout_Get_@metadata-12 31 31 +0.00% BenchmarkBeatEventV0/Put_@metadata-12 30 30 +0.00% BenchmarkBeatEventV0/timeout_Put_@metadata-12 32 32 +0.00% BenchmarkBeatEventV0/Delete_@metadata-12 28 28 +0.00% BenchmarkBeatEventV0/timeout_Delete_@metadata-12 30 30 +0.00% BenchmarkBeatEventV0/Cancel-12 28 28 +0.00% BenchmarkBeatEventV0/timeout_Cancel-12 30 30 +0.00% BenchmarkBeatEventV0/Tag-12 28 28 +0.00% BenchmarkBeatEventV0/timeout_Tag-12 30 30 +0.00% BenchmarkBeatEventV0/AppendTo-12 28 28 +0.00% BenchmarkBeatEventV0/timeout_AppendTo-12 30 30 +0.00% benchmark old bytes new bytes delta BenchmarkBeatEventV0/Put-12 2112 2114 +0.09% BenchmarkBeatEventV0/timeout_Put-12 2208 2210 +0.09% BenchmarkBeatEventV0/Object_Put_Key-12 2096 2098 +0.10% BenchmarkBeatEventV0/timeout_Object_Put_Key-12 2192 2194 +0.09% BenchmarkBeatEventV0/Get-12 2096 2098 +0.10% BenchmarkBeatEventV0/timeout_Get-12 2192 2194 +0.09% BenchmarkBeatEventV0/Get_Object-12 2256 2258 +0.09% BenchmarkBeatEventV0/timeout_Get_Object-12 2352 2354 +0.09% BenchmarkBeatEventV0/Get_Undefined_Key-12 2416 2419 +0.12% BenchmarkBeatEventV0/timeout_Get_Undefined_Key-12 2512 2514 +0.08% BenchmarkBeatEventV0/fields_get_key-12 2256 2258 +0.09% BenchmarkBeatEventV0/timeout_fields_get_key-12 2352 2354 +0.09% BenchmarkBeatEventV0/Get_@metadata-12 2096 2098 +0.10% BenchmarkBeatEventV0/timeout_Get_@metadata-12 2192 2194 +0.09% BenchmarkBeatEventV0/Put_@metadata-12 2112 2114 +0.09% BenchmarkBeatEventV0/timeout_Put_@metadata-12 2208 2210 +0.09% BenchmarkBeatEventV0/Delete_@metadata-12 2080 2082 +0.10% BenchmarkBeatEventV0/timeout_Delete_@metadata-12 2176 2178 +0.09% BenchmarkBeatEventV0/Cancel-12 2080 2082 +0.10% BenchmarkBeatEventV0/timeout_Cancel-12 2176 2178 +0.09% BenchmarkBeatEventV0/Tag-12 2080 2082 +0.10% BenchmarkBeatEventV0/timeout_Tag-12 2176 2178 +0.09% BenchmarkBeatEventV0/AppendTo-12 2080 2082 +0.10% BenchmarkBeatEventV0/timeout_AppendTo-12 2176 2178 +0.09% ``` (cherry picked from commit a690020)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry-pick of PR #13857 to 7.4 branch. Original message:
This should make the script processor safe to be used across multiple goroutines.
It compiles the script once. And loads it into a pool of VMs. Each invocation of
Run()grabs a VM instance from the pool and returns it when complete.