feat(compiler): support json to struct conversion

[hasanaburayyan]

The changes in this PR make it possible to call fromJson on any compatible struct definition.

Implementation notes:

Previously we treated the jsification of structs as a no-op since there is not a JS equivalent. So with this change structs now JSify into a Struct file that contains a class with a static jsonSchema and a fromJson function which will allow for field validation at runtime.

The schema generated adheres to: https://json-schema.org/understanding-json-schema/

take this simple example Wing code:

struct MyStruct {
	myField: str;
	myOtherField: num;
}

this will now generate a JS file named MyStruct.Struct.js which looks like this:

module.exports = function(stdStruct, fromInline) {
  class MyStruct {
    static jsonSchema() {
      return {
        id: "/MyStruct",
        type: "object",
        properties: {
          myField: { type: "string" },
          myOtherField: { type: "number" },
        },
        required: [
          "myField",
          "myOtherField",
        ],
        $defs: {
        }
      }
    }
    static fromJson(obj) {
      return stdStruct._validate(obj, this.jsonSchema())
    }
    static _toInflightType(context) {
      return fromInline(`require("./MyStruct.Struct.js")(${ context._lift(stdStruct) })`);
    }
  }
  return MyStruct;
};

The piece that brings this all together is the addition of the Struct class in our std that only has a fromJson() methods at the moment that is a Wing macro. The macro just calls the fromJson() method in the generated javascript.

Misc

We want to stop the user at compile time from calling fromJson on a struct that cannot be represented by a Json value ie

struct MyStruct {
	b: cloud.Bucket;
}
let j = {};
MyStruct.fromJson(j);

to prevent this I added a check in the typechecker for structs to confirm that if fromJson is called that all the fields in the struct are valid for conversion attempt

See image below for error when attempting:

Closes: #3653
Closes: #3139

TODO:

• separate the work done here and the remaining work into different tickets.
• update language reference

Followup issues that are out of scope for this PR:

• Json: schema validation support fromJson for imported structs #3792
• Json: schema validation expose schema() object #3790
• Json: schema validation disable partial parsing #3789
• Json: support unsafe schema validation #3788

Checklist

• Title matches Winglang's style guide
• Description explains motivation and solution
• Tests added (always)
• Docs updated (only required for features)
• Added pr/e2e-full label if this feature requires end-to-end testing

By submitting this pull request, I confirm that my contribution is made under the terms of the Wing Cloud Contribution License.

[yoav-steinberg]

Very nice work
added some questions.

[eladb]

Great job 👏

[eladb]

I was wondering if it might make more sense to generate a json schema and use a standard schema validation utility.

We also want to support something like:

let s = MyStruct.jsonSchema();

s.validate()
s.toJson()
// etc..

The use case is for example using wing structs as api shapes with OpenAPI specs.

[yoav-steinberg]

I was wondering if it might make more sense to generate a json schema and use a standard schema validation utility.

After reviewing I thought of this too. I think we should have a getSchema method in std.Struct, the validate method should be:

var v = new require('jsonschema').Validator(); // see https://www.npmjs.com/package/jsonschema
v.validate(obj, self.getSchema());

If there's no major issue with this approach I suggest doing another dev push to making it work this way and void our custom validation code.

[hasanaburayyan]

I was wondering if it might make more sense to generate a json schema and use a standard schema validation utility.

@yoav-steinberg @eladb When I originally started I wanted to generate schemas and went down a few learning rabbit holes, and did not think it was something we could easily do (because of nested arrays that can contain user defined structs), thats when I shifted gears to this approach.

However, back then I was younger, naive and since then I have grown and learned more about the way bundling and some of the other JavaScript vodoo works 😂 😂 and I think generating these schemas for use in a standard validator can be "easishly" done.

Im going to convert this PR to a WIP and investigate changing to that pattern (assuming there are not any objections)

[yoav-steinberg]

Very nice work using schemas

[eladb]

Some issue with module resolution.

Also, please update the language reference with the relevant implemented features from the spec (basically copy from here to here.

I would also add a list of follow up items in the pull request with tracking issues.

[hasanaburayyan]

Also, please update the language reference with the relevant implemented features from the spec (basically copy from here to here.

@eladb forgot to update the docs before I re-requested review. Added as a todo but will complete it in the morning.

[hasanaburayyan]

Added do not merge label until I complete the documentation and issue tracking todos

[eladb]

Hey!

This is amazing and I know you are keen to get this over the finish line, but there are a few tweaks I think we should make.

The generated struct code and the interaction between it and the std.Struct seem a bit convoluted.

I imagined something like;

Foo.Struct.js

class Foo {
  static get schema() {
    // return a full self-contained json schema, including all nested structs...
  }

  static fromJson(x) {
    return std.Struct.validate(x, this.schema);
  }

  // lifting boilerplate 

}

Something along these lines. KISS...

[eladb]

I think Im going to go ahead and leave this PR open and just add the remaining features for the issue rather than split them into a followup pr

That's usually something I regret, especially with a big project like this... I would try to get this merged first and follow up with subsequent PRs.

[eladb]

LGTM. @yoav-steinberg / @Chriscbr / @MarkMcCulloh please give the final approval

[MarkMcCulloh]

🎉

[MarkMcCulloh]

🎉

[mergify]

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

[monadabot]

Congrats! 🚀 This was released in Wing 0.25.1.