Universal approach for Jackson and Play JSON libraries

[chermenin]

The project was separated into several modules:

• jsonpath-core - some base classes for the library and test cases
• jsonpath-play - an implementation of JsonPath for Play JSON
• jsonpath-jackson - an implementation of JsonPath for Jackson

To add an implementation for another JSON library it's needed to extend the following classes:

• BaseJsonPath for a concrete implementation
• BaseJsonPathSpec for test cases

[slandelle]

I'm sorry, as you've spent some effort into this PR, but we won't merge this.

As explained in the README, this project is intended for Gatling's internal usage.
Been there, done that, providing general purpose libraries is too much of a burden with us:

• we can't afford to maintain code we don't use
• we want to be able to change things any time we want, without having people complaining because we've broken something on their side.
• we want to optimize as much as we can, without indirection layers or abstractions we don't use and that cause extra allocations (eg the new AST you've introduced)

If you need support for play-json support, I recommend you fork (make sure to keep license headers on our code, and to not publish with an artifactId starting with "gatling").

[plokhotnyuk]

@chermenin All that is happening is for better!

I would like to invite you to add JSONPath-like support for jsoniter-scala instead. Currently, this library provides the safest and most efficient way to work with JSON in Scala. But its ability to skim through JSON fast by efficiently skipping the unwanted part and extracting only required values is underestimated for the moment. Just look results of ExtractFields benchmark here to see that speed can exceed 1Gb per second in one thread easily.

[slandelle]

@plokhotnyuk Do you have any benchmark where your parser competes with Jackson? IMO, any « fastest JSON Scala parser » competition is pointless as long as they are 3 times slower than Jackson.

Then, really, don’t go with JsonPath. The total lack of a proper spec or TCK makes it a nightmare. Your users will craft their expressions with the help of a JavaScript online evaluator, and then complain your implementation behaves differently.
Gatling 3.2 will introduce JMESPath support.

[plokhotnyuk]

@slandelle There are numbers of benchmarks which compares jsoniter-scala vs jackson-module-scala. Here are results of a couple of them:

1. https://plokhotnyuk.github.io/jsoniter-scala/ - this benchmark is focused on different aspects of parsing and serialization of JSON using real-world samples and synthetically generated input, it is also a test lab for validation of parsers and checking of their safety
2. Add benchmarks for jsoniter-scala sirthias/borer#30 - that is focused on real-world samples from APIs or files.

Also, thanks for your suggestions about JsonPath and JMESPath.

[slandelle]

@plokhotnyuk thanks for the pointers.

I’ll adapt and run those benchmarks on my side later today. In my case, I want to compare with raw Jackson without the Scala module.
For my use case, I don’t care about projecting into Scala case classes, I want an AST I can quickly traverse, meaning without having to match node type with something that boils down to instanceof which is expensive.

[plokhotnyuk]

@slandelle JFYK: Yet another benchmark with results where both parsers are compared vs. best binary Java/Scala parsers

[slandelle]

@plokhotnyuk Does jsoniter-scala provide any means to project into a generic AST (eg Map[String, Any] + Seq[Any])? I can't find anything like this in the documentation and the samples?

[plokhotnyuk]

@slandelle You can create a custom codec with support of any JSON AST, but it is not so safe and efficient to be used on production at scale... Please, see here an example and results for AST of circe.

[slandelle]

but it is not so safe

Well, can't say this is an issue for us in the context of a load test tool :)

and efficient to be used on production at scale

For most users, probably. For us, as we don't serialize value or domain objects, this is not an issue. The AST wouldn't be an intermediate byproduct but the final result.

You can create a custom codec with support of any JSON AST

Thanks. I was hoping there was such support out of the box. Sadly, I don't think I'll have time to dig any time soon.

[plokhotnyuk]

@slandelle I suppose that you are interested to make the load efficiently and jsoniter-scala can serialize to the output stream or preallocated buffer with zero allocation for almost all supported types...

Have you considered to use code generation tool like this to emit case classes from JSON Schemas in compile-time?