-
Notifications
You must be signed in to change notification settings - Fork 17.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
encoding/json: make Scanner public #6499
Labels
Milestone
Comments
In particular, I used this in my jsonpointer implementation as it allowed me to get the information out *significantly* faster and with less memory overhead (i.e. didn't have to construct a big map[string]interface{} of all the parts and walk it). This is what I get out of it, in practice: Using only what's in encoding/json (blob of JSON to map): BenchmarkLargeMap 20 103041502 ns/op 18.83 MB/s 8139008 B/op 220394 allocs/op Using the scanner approach so I can throw away all the parts that aren't interesting and stop when I get to the part I'm seeking. BenchmarkLarge3Key 100000 24502 ns/op 79194.52 MB/s 7258 B/op 89 allocs/op BenchmarkLargeShallow 2000 876675 ns/op 2213.44 MB/s 33155 B/op 2718 allocs/op BenchmarkLargeMissing 50 35058577 ns/op 55.35 MB/s 922645 B/op 101772 allocs/op BenchmarkLargeIdentity 5000000 711 ns/op 2726464.30 MB/s 566 B/op 5 allocs/op BenchmarkLargeBest 500000 3893 ns/op 498379.24 MB/s 1445 B/op 24 allocs/op I've got another project that is needing the incremental scan right now which is why this just came up. In the meantime, I'm occasionally pulling in go's json improvements and exposing the scanner. |
I can understand why exposing, documenting, etc... this stuff could be a longer term burden. My particular uses don't involve structs, though. The data processing is more dynamic and keys being accessed are user-supplied. I used the scanner as part of evaluating an expression to dig data out of, in some cases, large JSON objects. The first version (Map above) was necessary using only encoding/json where I have to parse the entire thing into a map[string]interface{} and then traverse that structure to find the key I wanted. The rest are how I do it with scanner. I also needed to do some streaming transforms of JSON for which I *almost* used the scanner I'd exposed, but since I was only doing simple substitution in strings, I used a more simple state machine. |
This is related to issue #6050 ("encoding/json: provide tokenizer"). |
CL https://golang.org/cl/11651 mentions this issue. |
CL https://golang.org/cl/9073 mentions this issue. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
by michael.schurter:
The text was updated successfully, but these errors were encountered: