Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

builtins.jsonSchemaValidate attrset ./path/to/schema.json #5392

Open
blaggacao opened this issue Oct 15, 2021 · 4 comments
Open

builtins.jsonSchemaValidate attrset ./path/to/schema.json #5392

blaggacao opened this issue Oct 15, 2021 · 4 comments

Comments

@blaggacao
Copy link
Contributor

blaggacao commented Oct 15, 2021 •
edited

Is your feature request related to a problem? Please describe.

  • I have a config management problem that might for example involve nomad job configs or kubernetes manifests.
  • I want to use nix as my single config management language.
  • The only type system light that I have currently available is the nixpkgs (not nix!) module system.
  • I have to re-implement config types manually.
  • Error reporting UX of the nixpkgs module system is not acceptable for my "internal clients".

Describe the solution you'd like

  • Since official json schemata are ubiquitously available either as schema or as openapi spec or similar
  • I do builtins.jsonSchemaValidate attrset ./path/to/schema.json to validate my my config
  • without IFD
  • with great error provenance reporting

Describe alternatives you've considered

  • IFD with ajv
  • Does not solve the problem of error provenance reporting.

Additional context

There are a few candidate json schema validate C++ libs that I've found on https://json-schema.org/

/cc @kamadorueda
@kamadorueda
Copy link
Member

I'm willing to work on this one (on my free time)

I'll be familiarizing myself a little bit with the codebase in #3505

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/kubenix-not-maintained-anymore/15324/4

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-data-merge-mini-dsl-with-array-merge-semantics/15540/1

kamadorueda added a commit to kamadorueda/nix that referenced this issue Oct 18, 2021
@kamadorueda
builtins.validateJsonSchema
5ef5145 
- Add https://github.com/pboettch/json-schema-validator
  as a dependency

  `json-schema-validator` is a library that uses `nlohmann` internally,
  which is being used in other components of Nix already

  `json-schema-validator` is MIT licensed,
  complies with the latest draft of JSON schema
  and is the second one with most stars on GitHub
- Add a new builtin for validating data with a JSON schema.

  This builtin receives two arguments: `data` and `schema`,
  both in Nix format
- Demo:
  ```nix
  let
    data1 = { age = 42; };
    data2 = { age = 42; name = "Albert"; };
    schema = builtins.fromJSON ''
      {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "title": "A person",
        "properties": {
          "name": {
            "description": "Name",
            "type": "string"
          },
          "age": {
            "description": "Age of the person",
            "type": "number",
            "minimum": 2,
            "maximum": 200
          }
        },
        "required": [
          "name",
          "age"
        ],
        "type": "object"
      }
    '';
  in
  [
    (builtins.validateJsonSchema data1 schema)
    (builtins.validateJsonSchema data2 schema)
  ]
  ```

  ```bash
  $ nix-instantiate --eval --strict test.nix
  [ false true ]
  ```

- This is a work in progress and any feedback would be very appreciated!

  A few open questions I have:
  - Is it ok to upload the third party dependency into the repository?
    it's currently being done for cpptoml and nlohmann
  - Is the proposed semantics ok?
    or, for example, receiving the schema as a string would be better?
- Pending work:
  - Add docs
  - Add tests
  - Improve error messages by returning the validation messages instead
    of just success=(true/false)

Mentions NixOS#5392
@kamadorueda kamadorueda mentioned this issue Oct 18, 2021
Closed
kamadorueda added a commit to kamadorueda/nix that referenced this issue Oct 18, 2021
@kamadorueda
builtins.validateJsonSchema
66eca95 
- Add https://github.com/pboettch/json-schema-validator
  as a dependency

  `json-schema-validator` is a library that uses `nlohmann` internally,
  which is being used in other components of Nix already

  `json-schema-validator` is MIT licensed,
  complies with the latest draft of JSON schema
  and is the second one with most stars on GitHub
- Add a new builtin for validating data with a JSON schema.

  This builtin receives two arguments: `data` and `schema`,
  both in Nix format
- Demo:
  ```nix
  let
    data1 = { age = 42; };
    data2 = { age = 42; name = "Albert"; };
    schema = builtins.fromJSON ''
      {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "title": "A person",
        "properties": {
          "name": {
            "description": "Name",
            "type": "string"
          },
          "age": {
            "description": "Age of the person",
            "type": "number",
            "minimum": 2,
            "maximum": 200
          }
        },
        "required": [
          "name",
          "age"
        ],
        "type": "object"
      }
    '';
  in
  [
    (builtins.validateJsonSchema data1 schema)
    (builtins.validateJsonSchema data2 schema)
  ]
  ```

  ```bash
  $ nix-instantiate --eval --strict test.nix
  [ false true ]
  ```

- This is a work in progress and any feedback would be very appreciated!

  A few open questions I have:
  - Is it ok to upload the third party dependency into the repository?
    it's currently being done for cpptoml and nlohmann
  - Is the proposed semantics ok?
    or, for example, receiving the schema as a string would be better?
- Pending work:
  - Add docs
  - Add tests
  - Improve error messages by returning the validation messages instead
    of just success=(true/false)

Mentions NixOS#5392
kamadorueda added a commit to kamadorueda/nix that referenced this issue Oct 18, 2021
@kamadorueda
builtins.validateJsonSchema
159099e 
- Add https://github.com/pboettch/json-schema-validator
  as a dependency

  `json-schema-validator` is a library that uses `nlohmann` internally,
  which is being used in other components of Nix already

  `json-schema-validator` is MIT licensed,
  complies with the latest draft of JSON schema
  and is the second one with most stars on GitHub
- Add a new builtin for validating data with a JSON schema.

  This builtin receives two arguments: `data` and `schema`,
  both in Nix format
- Demo:
  ```nix
  let
    data1 = { age = 42; };
    data2 = { age = 42; name = "Albert"; };
    schema = builtins.fromJSON ''
      {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "title": "A person",
        "properties": {
          "name": {
            "description": "Name",
            "type": "string"
          },
          "age": {
            "description": "Age of the person",
            "type": "number",
            "minimum": 2,
            "maximum": 200
          }
        },
        "required": [
          "name",
          "age"
        ],
        "type": "object"
      }
    '';
  in
  [
    (builtins.validateJsonSchema data1 schema)
    (builtins.validateJsonSchema data2 schema)
  ]
  ```

  ```bash
  $ nix-instantiate --eval --strict test.nix
  [ false true ]
  ```

- This is a work in progress and any feedback would be very appreciated!

  A few open questions I have:
  - Is it ok to upload the third party dependency into the repository?
    it's currently being done for cpptoml and nlohmann
  - Is the proposed semantics ok?
    or, for example, receiving the schema as a string would be better?
- Pending work:
  - Add docs
  - Add tests
  - Improve error messages by returning the validation messages instead
    of just success=(true/false)

Mentions NixOS#5392
kamadorueda added a commit to kamadorueda/nix that referenced this issue Nov 3, 2021
@kamadorueda
builtins.validateJsonSchema
1dbbcee 
- Add https://github.com/pboettch/json-schema-validator
  as a dependency

  `json-schema-validator` is a library that uses `nlohmann` internally,
  which is being used in other components of Nix already

  `json-schema-validator` is MIT licensed,
  complies with the latest draft of JSON schema
  and is the second one with most stars on GitHub
- Add a new builtin for validating data with a JSON schema.

  This builtin receives two arguments: `data` and `schema`,
  both in Nix format
- Demo:
  ```nix
  let
    schema = {
      title = "A person";
      properties = {
        age = {
          description = "Age of the person";
          type = "number";
          minimum = 1;
          maximum = 200;
        };
        name = {
          description = "Complete Name for the person";
          first.type = "string";
          last.type = "string";
          required = [ "first" "last" ];
          type = "object";
        };
      };
      required = [ "name" "age" ];
      type = "object";
    };
  in
  builtins.map (builtins.validateJsonSchema schema) [
    { }
    { age = 24; name.first = "Jane"; }
    { age = 24; name.first = "Jane"; name.last = "Doe"; }
  ]
  ```

  ```bash
  $ nix-instantiate --eval --strict test.nix
  [
    { success = false;
      value = "At '/', required property 'name' not found in object"; }
    { success = false;
      value = "At '/name', required property 'last' not found in object"; }
    { success = true;
      value = { age = 24; name = { first = "Jane"; last = "Doe"; }; }; }
  ]
  ```

- Pending work:
  - Add docs
  - Add tests

Mentions NixOS#5392
kamadorueda added a commit to kamadorueda/nix that referenced this issue Nov 3, 2021
@kamadorueda
builtins.validateJsonSchema
bcb370b 
- Add https://github.com/pboettch/json-schema-validator
  as a dependency

  `json-schema-validator` is a library that uses `nlohmann` internally,
  which is being used in other components of Nix already

  `json-schema-validator` is MIT licensed,
  complies with the latest draft of JSON schema
  and is the second one with most stars on GitHub
- Add a new builtin for validating data with a JSON schema.

  This builtin receives two arguments: `data` and `schema`,
  both in Nix format
- Demo:
  ```nix
  let
    schema = {
      title = "A person";
      properties = {
        age = {
          description = "Age of the person";
          type = "number";
          minimum = 1;
          maximum = 200;
        };
        name = {
          description = "Complete Name for the person";
          first.type = "string";
          last.type = "string";
          required = [ "first" "last" ];
          type = "object";
        };
      };
      required = [ "name" "age" ];
      type = "object";
    };
  in
  builtins.map (builtins.validateJsonSchema schema) [
    { }
    { age = 24; name.first = "Jane"; }
    { age = 24; name.first = "Jane"; name.last = "Doe"; }
  ]
  ```

  ```bash
  $ nix-instantiate --eval --strict test.nix
  [
    { success = false;
      value = "At '/', required property 'name' not found in object"; }
    { success = false;
      value = "At '/name', required property 'last' not found in object"; }
    { success = true;
      value = { age = 24; name = { first = "Jane"; last = "Doe"; }; }; }
  ]
  ```

- Pending work:
  - Add docs
  - Add tests

Mentions NixOS#5392
kamadorueda added a commit to kamadorueda/nix that referenced this issue Nov 3, 2021
@kamadorueda
builtins.validateAsJSON
27bb226 
- Add https://github.com/pboettch/json-schema-validator
  as a dependency

  `json-schema-validator` is a library that uses `nlohmann` internally,
  which is being used in other components of Nix already

  `json-schema-validator` is MIT licensed,
  complies with the latest draft of JSON schema
  and is the second one with most stars on GitHub
- Add a new builtin for validating data with a JSON schema.

  This builtin receives two arguments: `schema` and `data`,
  both in Nix format
- Demo:

  ```nix
  let
    schema = {
      title = "A person";
      properties = {
        age = {
          description = "Age of the person";
          type = "number";
          minimum = 1;
          maximum = 200;
        };
        name = {
          description = "Complete Name for the person";
          first.type = "string";
          last.type = "string";
          required = [ "first" "last" ];
          type = "object";
        };
      };
      required = [ "name" "age" ];
      type = "object";
    };
  in
  map (validateAsJSON schema) [
    { }
    { age = 24; name.first = "Jane"; }
    { age = 24; name.first = "Jane"; name.last = "Doe"; }
  ]
  ```

  ```bash
  $ nix-instantiate --eval --strict test.nix
  [
    { success = false;
      value = "At '/', required property 'name' not found in object"; }
    { success = false;
      value = "At '/name', required property 'last' not found in object"; }
    { success = true;
      value = { age = 24; name = { first = "Jane"; last = "Doe"; }; }; }
  ]
  ```
- This demo was also added to the documentation in a simplified form
- Added language tests

Mentions NixOS#5392
kamadorueda added a commit to kamadorueda/nix that referenced this issue Nov 3, 2021
@kamadorueda
builtins.validateAsJSON
373a8a3 
- Add https://github.com/pboettch/json-schema-validator
  as a dependency

  `json-schema-validator` is a library that uses `nlohmann` internally,
  which is being used in other components of Nix already

  `json-schema-validator` is MIT licensed,
  complies with the latest draft of JSON schema
  and is the second one with most stars on GitHub
- Add a new builtin for validating data with a JSON schema.

  This builtin receives two arguments: `schema` and `data`,
  both in Nix format
- Demo:

  ```nix
  let
    schema = {
      title = "A person";
      properties = {
        age = {
          description = "Age of the person";
          type = "number";
          minimum = 1;
          maximum = 200;
        };
        name = {
          description = "Complete Name for the person";
          first.type = "string";
          last.type = "string";
          required = [ "first" "last" ];
          type = "object";
        };
      };
      required = [ "name" "age" ];
      type = "object";
    };
  in
  map (validateAsJSON schema) [
    { }
    { age = 24; name.first = "Jane"; }
    { age = 24; name.first = "Jane"; name.last = "Doe"; }
  ]
  ```

  ```bash
  $ nix-instantiate --eval --strict test.nix
  [
    { success = false;
      value = "At '/', required property 'name' not found in object"; }
    { success = false;
      value = "At '/name', required property 'last' not found in object"; }
    { success = true;
      value = { age = 24; name = { first = "Jane"; last = "Doe"; }; }; }
  ]
  ```
- This demo was also added to the documentation in a simplified form
- Added language tests

Mentions NixOS#5392
@stale
Copy link

stale bot commented Apr 16, 2022

I marked this as stale due to inactivity. → More info

@stale stale bot added the stale label Apr 16, 2022
@stale stale bot removed the stale label Dec 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@fricklerhandwerk @blaggacao @nixos-discourse @kamadorueda