Evolve or Rewrite?

[goofballLogic]

My understanding of the history of this library is that it was ported from the initial Java implementation. It was also built for standalone builds targeting multiple platforms when the tooling was a bit less sophisticated than it is these days.

As we consider supporting JSON-LD 1.1, should we evolve the current library to adopt modern .NET practices or attempt a rewrite?

[asbjornu]

I'm generally opposed to rewrites, just like Joel Spolsky. Unless there's some major deficiency in the core architecture of this codebase, or the changes involved to support JSON-LD 1.1 are so massive that a rewrite is easier, I would prefer a slowly evolving codebase.

In order to support that, we need a fully functional CI/CD pipeline that allows us to get continuous feedback on tests and also make it easy to publish new versions to NuGet and perhaps also pre-releases to the GitHub Package Registry. I've started that work in #58.

[goofballLogic]

Is there anything you can use help with on #58 ?

[goofballLogic]

I'm happy to stick with evolving the codebase. I have some changes which I'd like to propose overall (partly in connection with #59) but these could be accomplished by evolving the codebase through major releases. Initial thoughts below:

1. I think we should aim for the lowest possible .NET standard for basic algorithm functionality (more discussion on Proposed .NET version compatibility #59)
2. Given that the default JSON serializer in .NET Core is now System.Text.Json I propose that we attempt switching internal logic and the default API to use the native JSON tooling.
3. If we switch internal logic to use System.TextJson, I propose that we move the JSON.net-based methods into a distinct package which can optionally be installed to accomodate drop-in JSON.Net compatibility with the existing version but without imposing the installation of JSON.Net for those who don't want the overhead
4. I propose that, in addition, we support working on raw JSON strings without placing a requirement on the caller to have parsed the JSON into a JObject (or System.Text.Json equivalent).

[goofballLogic]

Given the above and discussion around changes to structure in #59, does evolving the current code base make sense or would it be better to start afresh? I'm on the fence, so will take your steer @asbjornu

[asbjornu]

I think we should give "let's evolve this thing" a chance. We always have the opportunity to change our minds and start afresh, but going the other way around is somewhere between very hard to impossible. :)

[goofballLogic]

Resolved to try evolving first.

See #65 , #67, #59, #28 for continued discussion

[sblom]

My understanding of the history of this library is that it was ported from the initial Java implementation.

Original author here. I've been lurking for years, and am excited to see the recent flurry of energy around this project!

I don't consider anything about the current code particularly carefully thought out--it was my best answer while I was working on the NuGet to "how can we avoid reinventing the wheel while also conforming to the standard?"

I used a tool called Sharpen to "auto-port" the Java implementation to C#. There was another week or two of work to fix up some of the quirky differences between C# and Java, especially around type system considerations like value types, generics, and subtle differences in base classes like Dictionary.

I celebrated once it could pass the tests in the standard test suite from its day.

It would be interesting to see how the Java implementation has changed. Despite the origin story for this project, it wouldn't be a trivial exercise to port in any changes that they've made, but it would provide hints as to what a (possibly bigger?) community did from a similar starting point to upgrade for usability and the 1.1 spec.

At any rate, it's really great to see y'all working on this. I personally think JSON-LD is one of the coolest well-kept secrets of the modern web! ;)

[asbjornu]

Original author here. I've been lurking for years, and am excited to see the recent flurry of energy around this project!

Thanks for checking in and for giving us the back-story of this repository, @sblom! 👋

At any rate, it's really great to see y'all working on this. I personally think JSON-LD is one of the coolest well-kept secrets of the modern web! ;)

Thank you for doing the groundwork of not only getting this repository up and running, but also by making it possible for Microsoft to use JSON-LD in any capacity. NuGet's API is still the only Microsoft API using JSON-LD, isn't it?

[goofballLogic]

Yeah I'm hoping we can get the library freshened up and then start to do a bit of "evangelism" within the .Net community - especially now that the various "API-centric" toolsets have had a few years to mostly end up somewhere around the "peak of inflated expectations" or the "trough of disillusionment". Now is the right time for this tech to make a big impact.

[sblom]

NuGet's API is still the only Microsoft API using JSON-LD, isn't it?

It's the only one I know of. Even the Microsoft Graph API is OData-based.

[tmarkovski]

Apologies for chiming in after this issue is closed, I wanted to share my thoughts. I'm working with JSONLD quite a bit as part of implementations of the Verifiable Credentials and Linked Data Suites. I've been a .NET geek since its inception.
I believe that a rewrite of the library is a better option, as the code itself is not very idiomatic to C#. As @sblom pointed out, and thank you for the great work, the code was ported from Java. I support the idea of using System.Text.Json, instead Newtonsoft for the implementation, so I'd like to propose a way forward.
There is already a library supporting JSONLD 1.1 (dotNetRDF). The code itself is incredibly well written and organized and the library is licensed under MIT. We could take the JSONLD feature from it (https://github.com/dotnetrdf/dotnetrdf/tree/master/Libraries/dotNetRDF/JsonLd) and use it as a base to rewrite this library, while swapping out Newtonsoft for STJ. This way, the community will benefit from having different JSON processor, and it would save a lot of time rewriting it from scratch.

[asbjornu]

Great idea, @tmarkovski! The plan is to evolve, but the evolving is only to preserve backwards compatibility. The eventual goal is a rewrite. When we reach the time to rip out the engine, we will have a good look at the one from dotNetRDF and see whether it's a good fit or not. That is, unless we reconsider and just keep the current codebase in maintenance mode and start the work on a version 2 immediately. Thoughts, @goofballLogic?

[goofballLogic]

I've been working towards a version of the library which can work with either System.Text.Json or Newtonsoft or stand-alone. The stand-alone option would leverage something like https://github.com/zanders3/json (which isn't a library dependency - just a drop-in snippet of code) to avoid dependencies on anything other than the core system libraries (plus those needed for retrieval of remote documents).

The ability to process JSON-LD without use of System.Text.Json means that we can continue to support older versions of .NET (and lower memory footprint a little), which I think is worth preserving. I'm taking gradual steps toward this goal (currently over here: #71).

We could also support pluggable engines as part of the rewrite I suppose...