Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: make Scanner public #6499

Closed
gopherbot opened this issue Sep 27, 2013 · 9 comments

Comments

Projects
None yet
5 participants
@gopherbot
Copy link

commented Sep 27, 2013

by michael.schurter:

The json.Scanner struct is generally useful and should be exposed as part of the public
JSON API as someone has done here:

https://github.com/dustin/gojson
@dustin

This comment has been minimized.

Copy link

commented Sep 27, 2013

Comment 1:

In particular, I used this in my jsonpointer implementation as it allowed me to get the
information out *significantly* faster and with less memory overhead (i.e. didn't have
to construct a big map[string]interface{} of all the parts and walk it).
This is what I get out of it, in practice:
Using only what's in encoding/json (blob of JSON to map):
BenchmarkLargeMap         20     103041502 ns/op      18.83 MB/s     8139008 B/op     220394 allocs/op
Using the scanner approach so I can throw away all the parts that aren't interesting and
stop when I get to the part I'm seeking.
BenchmarkLarge3Key    100000         24502 ns/op    79194.52 MB/s       7258 B/op         89
allocs/op
BenchmarkLargeShallow       2000        876675 ns/op    2213.44 MB/s       33155 B/op       2718
allocs/op
BenchmarkLargeMissing         50      35058577 ns/op      55.35 MB/s      922645 B/op     101772
allocs/op
BenchmarkLargeIdentity   5000000           711 ns/op    2726464.30 MB/s      566 B/op          5
allocs/op
BenchmarkLargeBest    500000          3893 ns/op    498379.24 MB/s      1445 B/op         24
allocs/op
I've got another project that is needing the incremental scan right now which is why
this just came up.
In the meantime, I'm occasionally pulling in go's json improvements and exposing the
scanner.
@rsc

This comment has been minimized.

Copy link
Contributor

commented Oct 18, 2013

Comment 2:

I am skeptical that we want to commit to implementation details like the Scanner.
If you are decoding into a struct it will already throw away the parts that aren't
interesting.

Labels changed: added priority-later, removed priority-triage.

Status changed to Thinking.

@dustin

This comment has been minimized.

Copy link

commented Oct 18, 2013

Comment 3:

I can understand why exposing, documenting, etc... this stuff could be a longer term
burden.
My particular uses don't involve structs, though.  The data processing is more dynamic
and keys being accessed are user-supplied.  I used the scanner as part of evaluating an
expression to dig data out of, in some cases, large JSON objects.  The first version
(Map above) was necessary using only encoding/json where I have to parse the entire
thing into a map[string]interface{} and then traverse that structure to find the key I
wanted.  The rest are how I do it with scanner.
I also needed to do some streaming transforms of JSON for which I *almost* used the
scanner I'd exposed, but since I was only doing simple substitution in strings, I used a
more simple state machine.
@rsc

This comment has been minimized.

Copy link
Contributor

commented Nov 27, 2013

Comment 4:

Labels changed: added go1.3maybe.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Dec 4, 2013

Comment 5:

Labels changed: added release-none, removed go1.3maybe.

@rsc

This comment has been minimized.

Copy link
Contributor

commented Dec 4, 2013

Comment 6:

Labels changed: added repo-main.

@extemporalgenome

This comment has been minimized.

Copy link

commented Apr 26, 2014

Comment 7:

This is related to issue #6050 ("encoding/json: provide tokenizer").

@rsc rsc added this to the Unplanned milestone Apr 10, 2015

@gopherbot

This comment has been minimized.

Copy link
Author

commented Jun 27, 2015

CL https://golang.org/cl/11651 mentions this issue.

@gopherbot

This comment has been minimized.

Copy link
Author

commented Jul 1, 2015

CL https://golang.org/cl/9073 mentions this issue.

@rsc rsc closed this in 0cf48b4 Jul 27, 2015

@mikioh mikioh modified the milestones: Go1.5, Unplanned Jul 28, 2015

@golang golang locked and limited conversation to collaborators Aug 5, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.