Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat:(ast) introduce Value to support concurrent operations #573

Closed
wants to merge 67 commits into from

Conversation

AsterDY
Copy link
Collaborator

@AsterDY AsterDY commented Jan 16, 2024

Background

sonic/ast.Node provide completed and efficient APIs to manipulate JSON. However, due to its transversely-lazy-load design, it CANNOT be read concurrently, result in limited usage in practice -- or abuse usage of LoadAll().
Therefore, we consider introduce a new type ast.Value, whose APIs are all based on JSON skipping-and-searching algorithm, can naturally support concurrently read

Thread Safty

  • GetXX(): All read APIs are implemented by searching-and-skipping raw JSON, and returned a slice of the origin -- Thus it is thread-safe
  • SetXX(): All write APIs are implemented based on GetXX(), and returned a new copy to the origin -- Thus it is false thread-safe, while the correctness of currently-writing need user to ensure by itself

Performance

Value's performance is close to Node, and 3x~10x times faster than the same API of gjson|sjson!!!

goos: darwin
goarch: amd64
pkg: github.com/bytedance/sonic/external_jsonlib_test/benchmark_test
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkGet/gjson-16  	   24603	     43020 ns/op	       1 B/op	       0 allocs/op
BenchmarkGet/Node-16   	  289396	      4206 ns/op	     333 B/op	       1 allocs/op
BenchmarkGet/Value-16  	  298384	      4442 ns/op	     342 B/op	       1 allocs/op
BenchmarkSubGet/gjson-16         	   10000	    107682 ns/op	       2 B/op	       0 allocs/op
BenchmarkSubGet/Node-16          	   20682	     60565 ns/op	    5976 B/op	       5 allocs/op
BenchmarkSubGet/Value-16         	   19473	     59041 ns/op	    1563 B/op	       0 allocs/op
BenchmarkGetMany/gjson-16        	    7738	    155706 ns/op	     242 B/op	       1 allocs/op
BenchmarkGetMany/Node-16         	   22348	     51654 ns/op	    3697 B/op	       5 allocs/op
BenchmarkGetMany/Value-16        	   26848	     49827 ns/op	    1493 B/op	       0 allocs/op
BenchmarkSetMany/sjson-16        	    4611	    253986 ns/op	   58056 B/op	      18 allocs/op
BenchmarkSetMany/Node-16         	   24410	     50759 ns/op	    3096 B/op	       6 allocs/op
BenchmarkSetMany/Value-16        	   16446	     72621 ns/op	   70596 B/op	      12 allocs/op
BenchmarkSet/sjson-16            	    7494	    179970 ns/op	   53008 B/op	      11 allocs/op
BenchmarkSet/Node-16             	   42609	     28077 ns/op	    2704 B/op	       6 allocs/op
BenchmarkSet/Value-16            	   60609	     17365 ns/op	   29228 B/op	       5 allocs/op

@codecov-commenter
Copy link

codecov-commenter commented Jan 16, 2024

Codecov Report

Attention: 575 lines in your changes are missing coverage. Please review.

Comparison is base (63f4c07) 78.57% compared to head (583741e) 77.60%.

Files Patch % Lines
ast/raw.go 65.05% 186 Missing and 103 partials ⚠️
ast/parser.go 63.78% 93 Missing and 62 partials ⚠️
ast/search.go 33.33% 45 Missing and 9 partials ⚠️
ast/decode.go 0.00% 52 Missing ⚠️
api.go 5.26% 18 Missing ⚠️
ast/api_amd64.go 78.12% 4 Missing and 3 partials ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #573      +/-   ##
==========================================
- Coverage   78.57%   77.60%   -0.98%     
==========================================
  Files          69       70       +1     
  Lines       10823    11841    +1018     
==========================================
+ Hits         8504     9189     +685     
- Misses       1942     2138     +196     
- Partials      377      514     +137     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@AsterDY AsterDY changed the title feat:(ast) introduce Value to support concurrently read feat:(ast) introduce Value to support concurrently operations Jan 22, 2024
@AsterDY AsterDY changed the title feat:(ast) introduce Value to support concurrently operations feat:(ast) introduce Value to support concurrent operations Jan 22, 2024
@AsterDY AsterDY closed this Jan 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants