Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate better binary data support #29

Closed
Zulu-Inuoe opened this issue Feb 17, 2023 · 1 comment · Fixed by #31
Closed

Investigate better binary data support #29

Zulu-Inuoe opened this issue Feb 17, 2023 · 1 comment · Fixed by #31
Labels
enhancement New feature or request

Comments

@Zulu-Inuoe
Copy link
Owner

Zulu-Inuoe commented Feb 17, 2023

jzon performs great when given strings, but when given binary data it falls back on flexi-streams to provide a translation layer.
Unfortunately, this causes jzon to slow down tremendously.

I'd hope there's more performant ways to do this parsing that don't involve a mountain of effort. Off the top of my head, we may ditch flexi-streams and rely on babel functions to iterate over a UB8 vector and/or stream.

While a decrease of performance is tolerable, here are some benchmarks on my system when using a simple-string - jzon, and (simple-array (unsigned-byte 8) (*)) - jzon-ub8:

Testing 'countries'
Repeat = '40'

JZON
Evaluation took:
  0.862 seconds of real time
  0.859375 seconds of total run time (0.796875 user, 0.062500 system)
  99.65% CPU
  3,017,972,224 processor cycles
  312,920,896 bytes consed


JZON-UB8
Evaluation took:
  21.695 seconds of real time
  21.625000 seconds of total run time (21.312500 user, 0.312500 system)
  [ Run times consist of 0.015 seconds GC time, and 21.610 seconds non-GC time. ]
  99.68% CPU
  52 lambdas converted
  75,912,864,352 processor cycles
  1,960,107,360 bytes consed

Testing 'small-file'
Repeat = '2000'

JZON
Evaluation took:
  0.017 seconds of real time
  0.031250 seconds of total run time (0.031250 user, 0.000000 system)
  182.35% CPU
  61,817,016 processor cycles
  9,214,640 bytes consed


JZON-UB8
Evaluation took:
  0.411 seconds of real time
  0.406250 seconds of total run time (0.390625 user, 0.015625 system)
  98.78% CPU
  1,440,014,925 processor cycles
  39,246,112 bytes consed

Testing 'large-file'
Repeat = '1'

JZON
Evaluation took:
  0.463 seconds of real time
  0.437500 seconds of total run time (0.421875 user, 0.015625 system)
  94.60% CPU
  1,622,016,973 processor cycles
  135,756,384 bytes consed


JZON-UB8
Evaluation took:
  16.775 seconds of real time
  16.718750 seconds of total run time (16.484375 user, 0.234375 system)
  [ Run times consist of 0.093 seconds GC time, and 16.626 seconds non-GC time. ]
  99.67% CPU
  58,697,360,841 processor cycles
  1,224,720,768 bytes consed

Testing 'numbers'
Repeat = '1'

JZON
Evaluation took:
  0.272 seconds of real time
  0.265625 seconds of total run time (0.265625 user, 0.000000 system)
  97.79% CPU
  951,784,533 processor cycles
  39,986,704 bytes consed


JZON-UB8
Evaluation took:
  9.844 seconds of real time
  9.843750 seconds of total run time (9.625000 user, 0.218750 system)
  [ Run times consist of 0.062 seconds GC time, and 9.782 seconds non-GC time. ]
  100.00% CPU
  34,446,031,815 processor cycles
  749,442,064 bytes consed
@Zulu-Inuoe Zulu-Inuoe added the enhancement New feature or request label Feb 17, 2023
@Zulu-Inuoe
Copy link
Owner Author

See discussion here - cl-babel/babel#39

it appears babel doesn't have a way to do this at the moment

Zulu-Inuoe added a commit that referenced this issue Feb 24, 2023
Embed a UTF-8 Decoder to avoid going through flexi-streams or babel

Fixes #29
Zulu-Inuoe added a commit that referenced this issue Feb 24, 2023
Embed a UTF-8 Decoder to avoid going through flexi-streams or babel

Fixes #29
Zulu-Inuoe added a commit that referenced this issue Feb 25, 2023
Embed a UTF-8 Decoder to avoid going through flexi-streams or babel

* Consolidate pos calculation to `%calc-pos`

Fixes #29
@Zulu-Inuoe Zulu-Inuoe linked a pull request Mar 1, 2023 that will close this issue
@Zulu-Inuoe Zulu-Inuoe added the pending-release Issue is resolved, but not yet released label Mar 15, 2023
@Zulu-Inuoe Zulu-Inuoe removed the pending-release Issue is resolved, but not yet released label Nov 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant