chore: extend Schema Object to allow defining custom formats

[magicmatatjahu]

---

title: "extend Schema Object to allow defining custom formats"

---

Related issue(s):
#622

---

This PR suggests how we could use custom schemas (in custom formats):

• a new keyword customSchema has been added, which belongs to the AsyncAPI Schema Object vocabulary (described by Custom Schema Object)
• is possible to refer custom schemas in the same way as normal JSON Schema
• the schemaFormat in the Message Object is deprecated
• is possible to use JSON Schema keywords, but only for documentation like title, description, externalDocs alongside customSchema keyword

Examples

Model defined by Custom Schema

{
  "customSchema": {
    "format": "application/vnd.apache.avro+json;version=1.9.0",
    "definition": {
      "type": "record",
      "namespace": "com.example.common",
      "name": "Student",
      "fields": [
        {
          "name": "Name",
          "type": "string"
        },
        {
          "name": "Age",
          "type": "int"
        }
      ]
    }
  }
}

customSchema:
  format: application/vnd.apache.avro+yaml;version=1.9.0
  definition:
    type: record
    namespace: com.example.common
    name: Student
    fields:
    - name: Name
      type: string
    - name: Age
      type: int

Model defined by Custom Schema with documentation fields

{
  "title": "Student Model",
  "description": "A model describing a Student object, defined using the AVRO format.",
  "customSchema": {
    "format": "application/vnd.apache.avro+json;version=1.9.0",
    "definition": {
      "type": "record",
      "namespace": "com.example.common",
      "name": "Student",
      "fields": [
        {
          "name": "Name",
          "type": "string"
        },
        {
          "name": "Age",
          "type": "int"
        }
      ]
    }
  },
  "externalDocs": {
    "description": "Find more info here",
    "url": "https://example.com"
  }
}

title: Student Model
description: A model describing a Student object, defined using the AVRO format.
customSchema:
  format: application/vnd.apache.avro+yaml;version=1.9.0
  definition:
    type: record
    namespace: com.example.common
    name: Student
    fields:
    - name: Name
      type: string
    - name: Age
      type: int
externalDocs:
  description: Find more info here
  url: https://example.com

Model using nested Custom Schema Format

{
  "type": "object",
  "properties": {
    "grades": {
      "type": "array",
      "items": {
        "type": "integer"
      }
    },
    "student": {
      "customSchema": {
        "format": "application/vnd.apache.avro+json;version=1.9.0",
        "definition": {
          "type": "record",
          "namespace": "com.example",
          "name": "Student",
          "fields": [
            {
              "name": "Name",
              "type": "string"
            },
            {
              "name": "Age",
              "type": "int"
            }
          ]
        }
      }
    }
  }
}

type: object
properties:
  grades:
    type: array
    items:
      type: integer
  student:
    customSchema:
      format: application/vnd.apache.avro+yaml;version=1.9.0
      definition:
        type: record
        namespace: com.example
        name: Student
        fields:
        - name: Name
          type: string
        - name: Age
          type: int

Message with custom format (show usage of $ref)

{
  "name": "UserSignup",
  "title": "User signup",
  "summary": "Action to sign a user up.",
  "description": "A longer description",
  "tags": [
    { "name": "user" },
    { "name": "signup" },
    { "name": "register" }
  ],
  "payload": {
    "customSchema": {
      "format": "application/vnd.apache.avro+json;version=1.9.0",
      "definition": {
        "$ref": "path/to/user-create.avsc"
      }
    }
  }
}

name: UserSignup
title: User signup
summary: Action to sign a user up.
description: A longer description
tags:
  - name: user
  - name: signup
  - name: register
payload:
  customSchema:
    format: 'application/vnd.apache.avro+yaml;version=1.9.0'
    definition:
      $ref: 'path/to/user-create.avsc'

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this pull request, add a comment with detailed explanation.

There can be many reasons why some specific pull request has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this pull request forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️

[jonaslagoni]

Just left a small comment.

This looks like it's ready to be reviewed, right @magicmatatjahu?

[jonaslagoni]

Would it not make sense to keep schemaFormat 🤔 ?

[fmvilas]

type: object
properties:
  grades:
    type: array
    items:
      type: integer
  student:
    customSchema:
      format: application/vnd.apache.avro+yaml;version=1.9.0
      definition:
        type: record
        namespace: com.example
        name: Student
        fields:
        - name: Name
          type: string
        - name: Age
          type: int

I'd not allow this. This requires changing JSON Schema. IMHO we should go in a direction where we get rid of the differences with JSON Schema instead of adding more "extra" keys. This also adds an incredible complexity for flexibility I doubt anyone would need.

This is Avro inside a JSON Schema object but what about the opposite? Should we also alter Avro to accept JSON Schema inside? Aren't we mixing different things in the same place? I mean, JSON Schema is a constraint system and Avro is a data-serialization system. I know the same applies without this feature but at least we don't mix and match 🤔

[magicmatatjahu]

More info why I'm closing -> #622 (comment) We can always reopen it, when we will have conclusion.