A streaming JsonPath processor in Java
Java ANTLR
Latest commit 4a63186 Feb 16, 2017 @wanglingsong wanglingsong Update copy right
Permalink
Failed to load latest commit information.
jsurfer-benchmark Update copy right Feb 16, 2017
jsurfer-core Update copy right Feb 16, 2017
.gitignore Initial project Mar 19, 2015
.travis.yml 1. Add travis file Mar 27, 2015
README.md Fixing typo Sep 26, 2016
pom.xml Update dependency and bump version Sep 5, 2016

README.md

JsonSurfer - Let's surf on Json!

Join the chat at https://gitter.im/jsurfer/JsonSurfer

Coverage Status Code Advisor On Demand Status

Why JsonSurfer

Jsonsurfer is dedicated in processing big and complicated json data with three major features.

  • Streaming

    No need to deserialize entire json into memory

  • JsonPath

    Selectively extract json data by the power of JsonPath

  • Stoppable

    JsonSurfer is built on stoppable SAX-like interface that allows the processor to stop itself if necessary.

Getting started

What is JsonPath?

  • JsonSurfer supports incomplete JsonPath feature at current version:
Operator Description Supported
$ root YES
@ current node Not yet
* wildcard YES
.. recursive descent YES
.<name> child YES
['<name>' (, '<name>')] child/children YES
[<number> (, <number>)] index/indices YES
[start:end] array slice YES
[?(<expression>)] expression Not yet
  • JsonSurfer is available in cetral maven repository.
<dependency>
    <groupId>com.github.jsurfer</groupId>
    <artifactId>jsurfer-core</artifactId>
    <version>1.2.8</version>
</dependency>
  • Optional dependencies

JsonSurfer has driver for Gson, Jackson and JsonSimple but does not transitively include Gson and Jackson library. Please declare them in POM as needed.

<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>2.7</version>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.8.2</version>
</dependency>

Surfing API:

"Surfing" in Json DOM tree collecting matched value in the listeners

        JsonSurfer surfer = new JsonSurfer(GsonParser.INSTANCE, JavaCollectionProvider.INSTANCE);
        surfer.configBuilder()
                .bind("$.store.book[*]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) throws Exception {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Repeated surfing with same binding

        JsonSurfer surfer = new JsonSurfer(GsonParser.INSTANCE, JavaCollectionProvider.INSTANCE);
        SurfingConfiguration config = surfer.configBuilder()
                .bind("$.store.book[*]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) throws Exception {
                        System.out.println(value);
                    }
                })
                .build();        
        surfer.surf(sample1, config);
        surfer.surf(sample2, config);

Collect the first matched value

        JsonSurfer jsonSurfer = JsonSurfer.gson();
        Object singleResult = jsonSurfer.collectOne(sample, "$.store.book[0]");

Colllect every matched value

        JsonSurfer jsonSurfer = JsonSurfer.gson();
        Collection<Object> multipleResults = jsonSurfer.collectAll(sample, "$.store.book[*]");

Resolver API:

  • As of 1.2.6, JsonSurfer provides another way of processing json. You can directly resolve value with JsonPath from a well-built DOM like HashMap or even POJO:
        Book book = new Book();
        book.setAuthor("Leo");
        book.setCategory("Fiction");
        book.setPrice(100.0d);
        book.setTitle("JsonSurfer is great!");
        System.out.print(compile("$.author").resolve(book, new PoJoResolver()));

which prints "Leo".

        List<String> list = Arrays.asList("foo", "bar");
        HashMap<String, Object> map = new HashMap<String, Object>();
        map.put("list", list);
        System.out.println(compile("$.list[1]").resolve(map, JavaCollectionProvider.INSTANCE));

which prints "bar".

Other API:

  • JsonSurfer provides flexible plug-in interface, you can choose your library for parsing and modeling. For example:
        // use json-simple parser (Json-Simple dependency is included by default)
        // transform json into json-simple model i.e.org.json.simple.JSONObject or org.json.simple.JSONArray
        JsonSurfer surfer = new JsonSurfer(JsonSimpleParser.INSTANCE, JsonSimpleProvider.INSTANCE);
        // or JsonSurfer surfer = JsonSurfer.simple();
        // use gson parser (You need to explicitly declare Gson dependency in you pom)
        // transform json into gson model i.e.com.google.gson.JsonElement
        JsonSurfer surfer = new JsonSurfer(GsonParser.INSTANCE, GsonProvider.INSTANCE);
        // or JsonSurfer surfer = JsonSurfer.gson();

More code Examples

Sample Json:

{
    "store": {
        "book": [
            {
                "category": "reference",
                "author": "Nigel Rees",
                "title": "Sayings of the Century",
                "price": 8.95
            },
            {
                "category": "fiction",
                "author": "Evelyn Waugh",
                "title": "Sword of Honour",
                "price": 12.99
            },
            {
                "category": "fiction",
                "author": "Herman Melville",
                "title": "Moby Dick",
                "isbn": "0-553-21311-3",
                "price": 8.99
            },
            {
                "category": "fiction",
                "author": "J. R. R. Tolkien",
                "title": "The Lord of the Rings",
                "isbn": "0-395-19395-8",
                "price": 22.99
            }
        ],
        "bicycle": {
            "color": "red",
            "price": 19.95
        }
    },
    "expensive": 10
}

Find the authors of all books:

$.store.book[*].author
        JsonSurfer surfer = JsonSurfer.gson();
        surfer.configBuilder()
                .bind("$.store.book[*].author", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

"Nigel Rees"
"Evelyn Waugh"
"Herman Melville"
"J. R. R. Tolkien"

All authors

$..author
        JsonSurfer surfer = JsonSurfer.gson();
        surfer.configBuilder()
                .bind("$..author", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

"Nigel Rees"
"Evelyn Waugh"
"Herman Melville"
"J. R. R. Tolkien"

All things in store

$.store.*
        JsonSurfer surfer = JsonSurfer.gson();
        surfer.configBuilder()
                .bind("$.store.*", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

[{"category":"reference","author":"Nigel Rees","title":"Sayings of the Century","price":8.95},{"category":"fiction","author":"Evelyn Waugh","title":"Sword of Honour","price":12.99},{"category":"fiction","author":"Herman Melville","title":"Moby Dick","isbn":"0-553-21311-3","price":8.99},{"category":"fiction","author":"J. R. R. Tolkien","title":"The Lord of the Rings","isbn":"0-395-19395-8","price":22.99}]
{"color":"red","price":19.95}

The price of everything in the store

$.store..price
        JsonSurfer surfer = JsonSurfer.gson();
        surfer.configBuilder()
                .bind("$.store..price", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

8.95
12.99
8.99
22.99
19.95

The thrid book

$..book[2]
        JsonSurfer surfer = JsonSurfer.gson();
        surfer.configBuilder()
                .bind("$..book[2]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

{"category":"fiction","author":"Herman Melville","title":"Moby Dick","isbn":"0-553-21311-3","price":8.99}

The first two books

$..book[0,1]
        JsonSurfer surfer = JsonSurfer.gson();
        surfer.configBuilder()
                .bind("$..book[0,1]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {
                        System.out.println(value);
                    }
                })
                .buildAndSurf(sample);

Output

{"category":"reference","author":"Nigel Rees","title":"Sayings of the Century","price":8.95}
{"category":"fiction","author":"Evelyn Waugh","title":"Sword of Honour","price":12.99}

Stoppable parsing

The parsing is stopped when the first book found and printed.

$..book[0,1]
        JsonSurfer surfer = JsonSurfer.gson();
        surfer.configBuilder()
                .bind("$..book[0,1]", new JsonPathListener() {
                    @Override
                    public void onValue(Object value, ParsingContext context) {                        
                        System.out.println(value);
                        context.stopParsing();
                    }
                })
                .buildAndSurf(sample);

Output

{"category":"reference","author":"Nigel Rees","title":"Sayings of the Century","price":8.95}

Benchmark

  • JsonSurfer is fast !!! The benchmark is powered by JMH
Benchmark                                            Mode  Cnt       Score      Error  Units
BenchmarkCollectSingleValue.benchmarkGson           thrpt   10  176764.763 ± 1104.124  ops/s
BenchmarkCollectSingleValue.benchmarkGsonSurfer     thrpt   10  740780.577 ± 5342.493  ops/s
BenchmarkCollectSingleValue.benchmarkJackson        thrpt   10  190154.310 ±  929.849  ops/s
BenchmarkCollectSingleValue.benchmarkJacksonSurfer  thrpt   10  521919.374 ± 2685.533  ops/s
BenchmarkCollectSingleValue.benchmarkSimpleSurfer   thrpt   10  219732.696 ± 3476.744  ops/s