Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

json2 perf: encode faster than cJSON #20052

Merged
merged 49 commits into from Dec 29, 2023

Conversation

enghitalo
Copy link
Contributor

@enghitalo enghitalo commented Dec 1, 2023

json2 became 2x or 3x faster in -prod. In some cases, more faster than cJSON.

Even after array_push_many fix PR

Before / Now
image

MAX_ITERATIONS=100_000 ./v -prod crun vlib/v/tests/bench/bench_json_vs_json2.v
image

🤖[deprecated] Generated by Copilot at d32ffe3

Refactor the JSON encoder in vlib/x/json2 to use byte buffers and unsafe operations for faster and more efficient encoding. Extract the encoder code to a separate file encoder.v.

🤖[deprecated] Generated by Copilot at d32ffe3

  • Remove unused imports of io and strings modules from vlib/x/json2/encoder.v (link)
  • Change constants for JSON tokens and characters from byte arrays to strings or single bytes, to simplify encoding and avoid conversions and allocations (link)
  • Refactor encoder methods to use byte buffers ([]u8) instead of writers (io.Writer) as arguments, to improve performance and memory efficiency by using pre-allocated buffers instead of writers that may allocate memory on each write operation (link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link)
  • Use unsafe operations to copy bytes from string representations of values to byte buffers, instead of converting them to byte arrays and writing them, to avoid unnecessary allocations and conversions (link, link, link, link, link, link, link, link)
  • Use string literals instead of calling str() method for booleans, to avoid memory allocation (link, link, link, link, link, link, link)
  • Change encoding logic for unicode escape sequences to use a for loop instead of an unsafe operation, to avoid potential buffer overflow issues (link)
  • Remove unused import of strings module from vlib/x/json2/json2.v (link)
  • Change json_str and encode_array methods to use byte buffers instead of string builders, and to free buffers instead of builders, for the same reasons as above (link, link)
  • Change file name from vlib/x/json2/json2.v to vlib/x/json2/encoder.v to reflect the refactoring of the encoder module (link, link)

@enghitalo enghitalo marked this pull request as draft December 2, 2023 02:51
@enghitalo enghitalo marked this pull request as ready for review December 3, 2023 20:28
cmd/tools/vls.v Outdated Show resolved Hide resolved
cmd/tools/vls.v Outdated Show resolved Hide resolved
@enghitalo enghitalo marked this pull request as ready for review December 28, 2023 06:44
@enghitalo enghitalo changed the title json2 perf: encode as faster as cJSON json2 perf: encode faster than cJSON Dec 28, 2023
@enghitalo
Copy link
Contributor Author

I think the fail is not related
image

vlib/x/json2/encoder.v Outdated Show resolved Hide resolved
vlib/v/gen/js/js.v Outdated Show resolved Hide resolved
vlib/x/json2/encoder.v Outdated Show resolved Hide resolved
vlib/x/json2/encoder.v Outdated Show resolved Hide resolved
vlib/x/json2/encoder.v Outdated Show resolved Hide resolved
vlib/x/json2/encoder.v Outdated Show resolved Hide resolved
vlib/x/json2/encoder.v Outdated Show resolved Hide resolved
vlib/x/json2/encoder.v Outdated Show resolved Hide resolved
vlib/x/json2/encoder.v Outdated Show resolved Hide resolved
vlib/x/json2/encoder.v Outdated Show resolved Hide resolved
vlib/x/json2/encoder.v Outdated Show resolved Hide resolved
vlib/x/json2/encoder.v Outdated Show resolved Hide resolved
@spytheman spytheman removed the Needs Rebase The code of the PR must be rebased over current master before it can be approved. label Dec 29, 2023
@spytheman
Copy link
Member

Excellent work.

@spytheman spytheman merged commit 7fc3159 into vlang:master Dec 29, 2023
54 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants