-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test(bench): improve go-faster/jx benchmark #1
Conversation
ernado
commented
Jan 10, 2022
•
edited
Loading
edited
- Use ObjBytes to avoid key allocations
- Use Skip instead of parsing
- Avoid unrelated allocation (add strToBytes)
* Use ObjBytes to avoid key allocations * Use Skip instead of parsing * Avoid unrelated allocation (add strToBytes)
@@ -232,15 +234,20 @@ func CalcStatsJsoniterWithPath(str string) (s Stats) { | |||
return | |||
} | |||
|
|||
// strToBytes returns byte slice from string without allocations. | |||
func strToBytes(s string) []byte { | |||
return unsafe.Slice((*byte)(unsafe.Pointer((*reflect.StringHeader)(unsafe.Pointer(&s)).Data)), len(s)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I totally get your point and it sure is a bit unfair to have the extra overhead because of a different input type but I'm still not sure how I feel about introducing unsafe
to the party since usually it's not welcome in production code 😕
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this library is focusing on processing strings - it's fair to compare only legal and safe methods of use for the context of this library only. If any user of a library requires performance at the cost of the risks of using unsafe
then it's his/her responsibility to do so.
However, we can let it pass under the assumption that the user of go-faster/jx is willing to pay that price (which many folks often do) just in order to eliminate the overhead for a fair comparison's sake.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A bit of a background:
I actually started out with unsafe
but quickly encountered a problem on my M1 Max macOS 12.1 Go 1.17.6. Seemingly random bugs started to appear after converting string
to []byte
using the unsafe
method from valyala/fasthttp. I couldn't reproduce them neither on another Intel i7 macOS 12.1 Go 1.17.6 MBP, nor on go.dev/play. Only later did I realize that it's me who did the stupid mistake of using an unsafe []byte -> string
method for string -> []byte
conversion, which, IMHO, only proves the dangers and cost of utilizing unsafe
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that it is unsafe, but not sure how to compare libraries fairly, because string -> byte allocation is unrelated.
If you will find another way I'm ok with it!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it's unrelated and slightly falsifies the results.
The solution is to just provide a new function:
func Scan(o Options, s []byte, fn func(*IteratorBytes) (err bool)) Error
This will be the most widely used function anyway.
@@ -294,15 +298,15 @@ func CalcStatsGofasterJx(str string) (s Stats) { | |||
} | |||
case gofasterjx.Object: | |||
s.TotalObjects++ | |||
if err := d.Obj(func(d *gofasterjx.Decoder, key string) error { | |||
if err := d.ObjBytes(func(d *gofasterjx.Decoder, key []byte) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the tip!