JSON-LD 1.1 interest

[fils]

I am curious if JSON-LD 1.1 is on the roadmap for this package.

https://json-ld.org/spec/latest/json-ld/

json-gold has been so wonderful to have and I'm just curious as I see 1.1 addressing some of the issues of multiple types and also being able to integrate things like GeoJSON elements into the JSON-LD object.

Thanks!
Doug

[kazarena]

@fils the short answer is yes, absolutely. I've already put in some minor provisions to support it.

The long answer is: I haven't been following the latest development of this standard (since Dec 2017) and not 100% sure how stable it is. Also, the usefulness of this package (at least for us) depends on how up to date the other implementations are (JavaScript, Python and Java in particular).

Do you think it's a good time to start supporting 1.1?

[BigBlueHat]

Do you think it's a good time to start supporting 1.1?

Definitely! Full disclosure: I'm one of the Co-Chairs of the @w3c JSON-LD Working Group 😸

It would be great to have you join us as an implementer/tester as well as involved in the discussions around JSON-LD's future.

You can get involved via our various GitHub repos or join the W3C and then join the JSON-LD WG to get involved in our weekly phone discussions.

Regardless of what involvement you have time for, we'd appreciate the feedback you can contribute!

Thanks!
🎩

[kazarena]

@BigBlueHat, I would be delighted help. We are actively using JSON-LD for the last 3 years and further development of the standard is of the utmost importance for us.

I'll take a look at the WG repos you mentioned. Any hints where my input would be most useful are welcome.

[BigBlueHat]

The syntax and api repos are likely of the greatest interest at this point. You can see more about what we plan to accomplish in our charter.

The other big area we could use help with is testing. We'll be continuing to use the current JSON-LD testing suite and at some point there will be an official W3C testing environment and process to get a "gold star" 🥇 as a conforming implementation--JSON-goLD is already listed there in fact! 😁

Lastly, there's a public mailing list where you can see some discussions and the posting of the call agenda and minutes each week. If you dig in deep enough, we'd also love to have you on the calls where final decisions are made by the group.

Thanks for all you're doing, and it's great to hear you've got so much experience with JSON-LD!

[kazarena]

@BigBlueHat thanks for all the details.

It's our goal to always have 100% pass rate for the relevant test suite. By the way, providing this suite is a fantastic way to help JSON-LD developers with new implementations, it can't go any better!

[kazarena]

@fils @BigBlueHat, we have implemented the initial support for JSON-LD 1.1 spec, see #15. All 1.1 tests from the current official test suite pass, with an exception of 5 advanced framing tests.

We used pyLD as a reference implementation, with some insights from jsonld.js.

We don't support omitGraph option yet.

I'll try to fix a couple of remaining tests, but this implementation is good enough to go live, in my opinion.

Having looked at the 1.1 spec in great detail I must say it's very good!

[kazarena]

Added omitGraph support and fixed 2 framing tests. The only outstanding broken framing test is tg010, see #13. I thought it's related to a different behaviour in 1.1, but it appears to have a different issue with document base. Will describe in #13.

[fils]

@kazarena Thanks for this! I am going to be in D.C. next week for some coding time with others in my group. I'll definitely put this on our agenda. I'm particularly interested in seeing if this makes some spatial data (as in GeoJSON) integration easier.

We got some more funding to continue our work with schema.org integration with federal earth science facilities. So this is well timed...

great news!

[kazarena]

@fils I'm looking forward to hearing about your experience.

In the meantime, we got a new version out (0.2.0) that includes 1.1 support.

[BigBlueHat]

Great to hear about the progress @kazarena! Another great reference implementation is @gkellogg's Ruby-based implementation for RDF.rb. Gregg is the primary editor for the three JSON-LD specs, so his implementation tends to stay in sync more than most. 😄

Thanks for the great work you're doing here!

[gkellogg]

The json-ld.org version of the test suite is frozen from work done by the community group. The working group spec references the up-to-date test suite at w3c.github.io/json-ld-api/tests. Framing has its own test suite.

Both python and JavaScript mplementations are somewhat behind the latest version of the specs.

[kazarena]

@gkellogg many thanks for the pointer. I've taken the tests from the repos mentioned above and raised a new PR. There are a lot of additions, and the pass rate dropped to 87% (see the conformance report here). I implemented some of the smaller updates and marked the remaining unsupported/failing tests as to-be-skipped for now. The largest outstanding areas are HTML based tests (not sure when I'll look at these ones), list-of-lists, sealed contexts, stricter validation. The rest are various edge cases across all the operations. I'll turn it into issues and will deal with them when time permits.

There were several surprises. For instance, three toRdf tests which were passing before, started failing after the test file update. It's related to the order of quads:

Old output (toRdf-0106-out.nq)

<http://example.com/people/dave> <http://xmlns.com/foaf/0.1/knows> <http://example.com/people/markus> .
<http://example.com/people/dave> <http://xmlns.com/foaf/0.1/name> "Dave Longley" .
<http://example.com/people/markus> <http://xmlns.com/foaf/0.1/name> "Markus Lanthaler" .
<https://json-ld.org/test-suite/tests/relative-node> <http://example.com/vocab/noTerm> <http://example.com/people/markus> .
<https://json-ld.org/test-suite/tests/relative-node> <http://xmlns.com/foaf/0.1/name> "Compact keys using @vocab" .

New output (toRdf-0106-out.nq)

<http://example.com/people/markus> <http://xmlns.com/foaf/0.1/name> "Markus Lanthaler" .
<http://example.com/people/dave> <http://xmlns.com/foaf/0.1/name> "Dave Longley" .
<http://example.com/people/dave> <http://xmlns.com/foaf/0.1/knows> <http://example.com/people/markus> .
<https://w3c.github.io/json-ld-api/tests/toRdf/relative-node> <http://xmlns.com/foaf/0.1/name> "Compact keys using @vocab" .
<https://w3c.github.io/json-ld-api/tests/toRdf/relative-node> <http://example.com/vocab/noTerm> <http://example.com/people/markus> .

The only difference is the order of quads. My implementation (in line with Python and Java versions) sorts the quads as the last step. The spec doesn't mention ordering, and the test suite docs explicitly recommend comparing the outputs of ToRDF operation semantically, so no reliance on order. Sorting the quads was always a major performance hit and a limiting factor for streaming. I would love to get rid of it. What are you thoughts? Is it safe to stop sorting ToRDF output?

[gkellogg]

The spec has relaxed the requirements on blank node labeling to conform with RDF norms. You can no longer depend on the order of quads. The test README says to use RDF isomorphism, but you could also use normalization to test the results.

Note, there is a broad change in test strategy to relax strict order and blank node naming.

[kazarena]

@gkellogg thanks, I'll update the testing methodology and it will fix 3-4 older tests.

[kazarena]

@fils @gkellogg @BigBlueHat,

I've created a new PR that implements a good chunk of the current JSON-LD 1.1 specification, see #24.

Test suite conformance is at around 89%, ex. Framing. See the latest conformance report here. Please note that I'm not using RDF isomorphism yet for test checks, and there are around 10 tests for the ToRDF algorithm that pass but are marked as untested.

I'm planning to publish a new version of the library soon and document the outstanding issues with JSON-LD 1.1 compliance so that somebody can pick it up.