Provide streaming API for reading

[ralfstx]

It should be possible to

• register a handler that receives callbacks during parsing
• avoid creating an object representation of the entire input in this mode
• interrupt the parsing at any point

[ralfstx]

Use cases for a streaming API

It seems that there are different use cases to take into account

reading without creating data

One example is syntax highlighting in an editor. For this case you don't care about the content, only about the text positions where syntactical elements begin and end. Another example could be analyzing/verifying the structure of a JSON. In this case, the content would matter (e.g. object names and value types), but you would not build up a structure.

creating a custom data structure

Another case would be creating a custom structure while reading, without intermediate JsonValues. This could be useful because

• the application requires a custom data types type anyway and it's faster to create the right structure right away
• a custom data structure could be more lightweight by immediately transforming or even skipping values

For example, a custom document handler would know that the value for an object member named "time" is a string representing a time. Instead of storing the string in a JsonValue, it could store it in a Date field of the object being created. There should be no JsonValues involved unless this type is part of the custom data structure.

skipping parts of the inputs

The application wants to skip parts of a long document. This could be useful because:

• the document is too large to fit in memory
• skipping parts of the document could improve reading performance

The parser still needs to read every single character, but by telling it to skip a part, it could avoid filling the capture buffer and save CPU cycles.

[angelozerr]

@ralfstx for your information, I try to contribute to Eclipse IDE with Eclipse WTP JSON. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=471820

I would like to support JSON Schema validation by using minimal-json to report error with location. To do that I need that JsonValue provide a getLocation kind.

The streaming API could help me a lot to avoid copying/pasting and updating source of minimal-json to add location of JsonValue.

Have you an idea when streaming API will be available? I could give you feedback when I will try it.

Thanks!

[ralfstx]

@angelozerr the streaming feature is high on my list and there were a lot of good ideas in #49 that will need some time to consolidate. It's out of scope for 0.9.4, but I hope to include it in the release after that. Maybe late summer, but no promises.

BTW, I also used a modified version of minimal-json in eclipsesource/jshint-eclipse to be able to report errors with location info. Maybe you could use a similar hack in the meantime?

[angelozerr]

It's out of scope for 0.9.4, but I hope to include it in the release after that. Maybe late summer, but no promises.

Ok that's cool. For the moment I use the copy of minimal-json that I adapt it.

BTW, I also used a modified version of minimal-json in eclipsesource/jshint-eclipse to be able to report errors with location info. Maybe you could use a similar hack in the meantime?

Yes I have seen that, but in my case it's for a JSON Schema validator. In this case the JSON to validate is valid (none exception). When JSON Schema processor doens't match a rule (eg : a JsonValue doesn't match a type, I need to have the location of this node). In other words I need the whole location of the Json nodes.

[ralfstx]

With location, you mean start and end offset, right? I'll let you know when I have a draft proposal to make sure that it fits for your needs.

[angelozerr]

With location, you mean start and end offset, right?

Exactly, and I think I will need for JsonObject :

• start/end location for "names" . Ex : when a property name doesn't exists in the schema.
• start/end location for "values". Ex : when a value has not the well type.

[ralfstx]

@angelozerr A first version of the streaming API has landed in the repository and a snapshot build is available at oss.sonatype.org. Some details may still change until the release.

[angelozerr]

Many thanks @ralfstx! I will play with your work and give you feedback ASAP.