jsonScheme to nickel converter

[blaggacao]

Is your feature request related to a problem? Please describe.
A lot of json schemata are readily available out there.

Describe the solution you'd like
A json scheme to nickel type system translator can help bootstrapping a lot of config schemata in nickel

Describe alternatives you've considered
Implement some sort of builtin jsonSchemaValidator

Additional context
none

[thufschmitt]

Would this need to be built into Nickel proper? It looks like it could essentially be defined as a standalone library (or maybe I’m missing something?)

[blaggacao]

Yes, the best approach IMO would be a standalone-library as part of the nickel project.

That is unless a variant of this is chosen: built-in support for jsonschema.

But since nickel has a versatile type system, I think a faithful (=theoretically roundtrippable) translator would be probably the better approach, here.

That way, the full nickel machinery w.r.t error reporting etc can be used as-is, without bloating the nickel code base with foreign schema validation code.

Btw, for nix, not having a type system, the conclusion would probably be: leap frog a type system light by adopting built in jsonschema support.

[mtoohey31]

I gave something like this a try in the form of a standalone program that generates contract code given a JSON schema, but I've come to the conclusion that being able to fully translate JSON schemas into ergonomic Nickel contracts is impossible with the language in its current form because of the lack of support for union and intersection contracts (see this related blog post). Since JSON schemas support the allOf and anyOf keywords, but we don't support union and intersection, the best approach I could come up with was to turn the whole subschema into a predicate as soon as we encounter, for example, an anyOf. That leads to an absolutely unreadable contract and terrible blame messages though, so I didn't think that was worth implementing.

[yannham]

@mtoohey31 interesting experiment!

I think intersection contracts alone are mostly fine to implement, as you long as you don't fiddle with functions. You can just apply them in sequence, being aware that this might not give what you think on schemas with functions inside (but I guess JSON schemas can't have that anyway). The issue comes from function contracts, which "negate" the argument, so you end up needing something like a union when you have an intersection for the domain contract.

Union is harder to handle. One brutal possibility would be to provide an operation such as in #714, which effectively allows you to turn an arbitrary contract into a predicate. That breaks the laziness unfortunately, but could be used as an escape hatch, because in the end it's probably still better to have JSON schemas although they are not lazy (as long as this is made very clear by the corresponding tool) rather than no interop at all.

My only fear is that people will abuse this operator to write all sort of unions, because this is just too handy, making in practice most contracts coming from a library potentially not lazy. Maybe a good compromise could be to at least wait to have a restricted or operation, as proposed in the blog post, which would hopefully fulfill most use-cases, before releasing such a try/catch like mechanism (which would be an ugly, unsafe primop like %unsafe_catch%)

[mtoohey31]

Thanks for the reply @yannham!

I went and read the section of the paper on intersection; I didn't realize the difficulties there were limited to functions, but that makes sense now. Based on that, something like this seems like it should behave correctly for the JSON case.

On the topic of #714, I looked at the JSON schema specification again from the perspective of figuring out what can be reasonably represented with contracts as they are today, and the not keyword also seems problematic. #714 would allow a Not contract to be defined quite easily, but I agree that the "escape hatch" approach could be risky because it might end up being overused. (If you do decide to implement something like that though, it seems like it might be useful to be able to attribute blame to the point where the contract.validate call failed, so maybe it could return something other than just a Bool?)

If we were to go the route of a restricted or operation instead, we would probably also need a similar not operator if we want to be able to fully support JSON schemas, so that we can cover all the basic boolean operations (and then we could get stuff like oneOf, if, etc. from combinations of the sequential, manually defined All, along with the restricted builtins or and not).

Either way, deciding what to do about unions is definitely a tricky issue...

[vkleen]

Since there continues to be interest in converting JSON schema and related specification into Nickel contracts, I've taken the liberty of updating @mtoohey31's first sketch to Nickel 1.0. I've also started experimenting with a library of predicate combinators as a first step towards converting all JSON schema constructs into predicates for Nickel.

I've also taken the liberty of copying @mtoohey31's code into the nickel-lang organisation at https://github.com/nickel-lang/json-schema-to-nickel.

@mtoohey31: I've invited you as a collaborator there and I hope you don't have objections to this.

[mtoohey31]

No objections, thanks @vkleen!

[vkleen]

https://github.com/nickel-lang/json-schema-to-nickel should now be fully functional 🎉 It would certainly benefit from some stress testing, so please try it out!

[yannham]

Closing as json-schema-to-nickel is up and running.