Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: introduce ast.Value to support read and write concurrently #579

Closed
wants to merge 6 commits into from

Conversation

AsterDY
Copy link
Collaborator

@AsterDY AsterDY commented Jan 24, 2024

Background

sonic/ast.Node provide completed and efficient APIs to manipulate JSON. However, due to its transversely-lazy-load design, it CANNOT be read concurrently, result in limited usage in practice -- or abuse usage of LoadAll().
Therefore, we consider introduce a new type ast.Value, whose APIs are all based on JSON skipping-and-searching algorithm, can naturally support concurrently read

Thread Safty

  • GetXX(): All read APIs are implemented by searching-and-skipping raw JSON, and returned a slice of the origin -- Thus it is thread-safe
  • SetXX(): All write APIs are implemented based on GetXX(), and returned a new copy to the origin -- Thus it is also thread-safe, while the correctness of currently-writing need user to ensure by itself

Performance

Value's performance is close to Node, and 3x~10x times faster than the same API of gjson|sjson!!!

goos: darwin
goarch: amd64
pkg: github.com/bytedance/sonic/external_jsonlib_test/benchmark_test
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkGet/gjson-16  	   24603	     43020 ns/op	       1 B/op	       0 allocs/op
BenchmarkGet/Node-16   	  289396	      4206 ns/op	     333 B/op	       1 allocs/op
BenchmarkGet/Value-16  	  298384	      4442 ns/op	     342 B/op	       1 allocs/op
BenchmarkSubGet/gjson-16         	   10000	    107682 ns/op	       2 B/op	       0 allocs/op
BenchmarkSubGet/Node-16          	   20682	     60565 ns/op	    5976 B/op	       5 allocs/op
BenchmarkSubGet/Value-16         	   19473	     59041 ns/op	    1563 B/op	       0 allocs/op
BenchmarkGetMany/gjson-16        	    7738	    155706 ns/op	     242 B/op	       1 allocs/op
BenchmarkGetMany/Node-16         	   22348	     51654 ns/op	    3697 B/op	       5 allocs/op
BenchmarkGetMany/Value-16        	   26848	     49827 ns/op	    1493 B/op	       0 allocs/op
BenchmarkSetMany/sjson-16        	    4611	    253986 ns/op	   58056 B/op	      18 allocs/op
BenchmarkSetMany/Node-16         	   24410	     50759 ns/op	    3096 B/op	       6 allocs/op
BenchmarkSetMany/Value-16        	   16446	     72621 ns/op	   70596 B/op	      12 allocs/op
BenchmarkSet/sjson-16            	    7494	    179970 ns/op	   53008 B/op	      11 allocs/op
BenchmarkSet/Node-16             	   42609	     28077 ns/op	    2704 B/op	       6 allocs/op
BenchmarkSet/Value-16            	   60609	     17365 ns/op	   29228 B/op	       5 allocs/op

@codecov-commenter
Copy link

codecov-commenter commented Jan 25, 2024

Codecov Report

Attention: Patch coverage is 58.84808% with 493 lines in your changes are missing coverage. Please review.

Project coverage is 76.63%. Comparing base (3739ffe) to head (bbf63a6).

Files Patch % Lines
ast/raw.go 64.67% 194 Missing and 102 partials ⚠️
ast/decode.go 0.00% 52 Missing ⚠️
ast/parser.go 66.01% 43 Missing and 9 partials ⚠️
ast/search.go 25.37% 44 Missing and 6 partials ⚠️
ast/api_amd64.go 57.44% 16 Missing and 4 partials ⚠️
api.go 14.28% 18 Missing ⚠️
internal/rt/fastmem.go 0.00% 4 Missing ⚠️
ast/visitor.go 50.00% 1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #579      +/-   ##
==========================================
- Coverage   77.56%   76.63%   -0.93%     
==========================================
  Files          83       84       +1     
  Lines       11542    12581    +1039     
==========================================
+ Hits         8952     9642     +690     
- Misses       2192     2408     +216     
- Partials      398      531     +133     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@AsterDY AsterDY closed this Mar 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants