[Help] Any idea how the request made from VSCode IDE could include the needed schema version?

[smoya]

Description of the feature / enhancement.

I know this is not VS Code repo, but I wanted to share this with you just in case you already faced this issue. I don't see you have GH Discussions enabled so I'm forced to create an issue like this one. If you feel it shouldn't go definitely here, feel free to close it without hesitating.

At Asyncapi.com, we have a handler that collects metrics (hits) every time someone downloads any AsyncAPI JSON Schema version, including the Schema served by Schema Store (https://www.asyncapi.com/schema-store/all.schema-store.json).
The problem is that we lack info about the real JSON Schema the user needs, since it always downloads the same (the one linked above) and it's something that would be awesome to have so we can measure our version adoption as well.

Do you believe there is something we could do (I can't figure any solution) to include info of the version requested (via a header) in the request made from VS Code to the url field located in the Schema Store catalog.json?

I imagine the only way would be to modify VS Code to let parse the asyncapi yaml file, get the info from there and send it. Of course I dont' see it viable (VsCode would need to support thousands of formats and understand from where to grab the version).

Thanks for reading!

Are you making a PR for this?

No, someone else must create the PR.

[GerryFerdinandus]

@smoya
If I understand correctly. VS Code download the all.schema-store.json and always download all other schemas $ref at the same time. So it is not possible to distinguish what is really needed.

If this is the problem, the possible solution is to change
"oneOf" to "anyOf"
With "oneOf" the validator must check that only one in true and all other must be false.
With "anyOf" I expect the IDE to only download until it finds the correct version in the list and stops further processing in the list.
But the number of hits must be adjusted.
Example:
version 1 (250 hits)
version 2 (100 hits)

version 1 actually only has 250 - 100 = 150 hits
Because to get to version 2, version 1 must first be downloaded.

It is more efficient that the version order is in descending order (more popular to less popular). Faster to get the right version.

[jimmylewis]

I don't think changing from oneOf to anyOf makes a difference; looking at the VS Code implementation it computes all branches. The same for the Visual Studio JSON editor. In order to provide completions, all matching branches matches must be considered.

I can't tell if the if-then-else defers resolving (i.e. downloading) a schema. If it does, you might be able to defer resolving each $ref to an external schema by using something like this:

 "oneOf": [
    {
      "if":  
        {
          "properties": {
            "asyncapi": {
              "const": "1.0.0"
            }
          }
        },
      "then": 
        {
          "$ref": "http://asyncapi.com/schema-store/1.0.0.json"
        },
      "else": false
    },
    {
      "if": 
        {
          "properties": {
            "asyncapi": {
              "const": "1.1.0"
            }
          }
        },
      "then": 
        {
          "$ref": "http://asyncapi.com/schema-store/1.1.0.json"
        },
      "else": false
    },
    ...

The idea is the same as you have now: the oneOf requires that one branch must be fulfilled. Each branch either matches both the const value and the "then" value, or it fails ("else": false will always fail validation). If resolving the "then" is deferred, then it'll only be downloaded if the const term is matched. You'll have to test whether this actually works or whether all of the $refs are still resolved during validation.

The other compounding factor for getting this measurement is how the editor caches the schemas. Depending on how their cached, you may only see a single download for a user (or one per cache expiration), or many downloads from the same user (if not cached) for the same document.

[smoya]

In order to confirm if vscode will fetch or not all schemas when using if, as @jimmylewis suggested, I used a mitm proxy to check connections and I confirm it downloads all referenced schemas.

This is the schema I forced to use (can be also found in this gist):

{
    "$id": "http://asyncapi.com/schema-store/all.schema-store.json",
    "$schema": "http://json-schema.org/draft-07/schema",
    "title": "JSON Schema documents for all AsyncAPI spec versions",
    "description": "All AsyncAPI JSON Schema documents listed in one file. Needed for serving all documents through schemastore.org",
    "type": "object",
    "oneOf": [
        {
            "if": {
                "properties": {
                    "asyncapi": {
                        "const": "2.0.0"
                    }
                }
            },
            "then": {
                "$ref": "http://asyncapi.com/schema-store/2.0.0-without-$id.json"
            },
            "else": false
        },
        {
            "if": {
                "properties": {
                    "asyncapi": {
                        "const": "2.6.0"
                    }
                }
            },
            "then": {
                "$ref": "http://asyncapi.com/schema-store/2.6.0-without-$id.json"
            },
            "else": false
        }
    ]
}

And the actual AsyncAPI document:

# yaml-language-server: $schema=https://gist.github.com/smoya/db046f0e9a558877e88d04f062f028a5/raw/b690f6172d8f660295cca1446276cba3bbe6fa2a/all.schema-store.json
asyncapi: '2.6.0'
info:
  title: Account Service
  version: 1.0.0
  description: This service is in charge of processing user signups
channels:
  user/signedup:
    subscribe:
      message:
        $ref: '#/components/messages/UserSignedUp'
components:
  messages:
    UserSignedUp: {}

I can see vscode requests both references:

So I ran out of ideas. I think this is not possible to do without changing the actual vscode extension.
Feel free to close this issue.

Thank you all of you who brought some possible solutions! ❤️

[smoya]

After a conversation with @fmvilas revisiting this issue, I came up with a possible solution we must test: replace the allOf with an if condition.
I checked the code on VS Code JSON parser implementation and it gave me some hope. If my assumptions are correct, it might work since it seems it evaluates first the if, and in case of matching, continues with the then, else otherwise.

We will do some further tests and come back with results.

[smoya]

TL;DR:

I tested it nesting if/then/else but the issue still persists. I assume the dereference process happens at the very beginning just before parsing the JSON file itself.

Long version

Sharing the JSON Schema doc and all I created for testing:

{
  "$id": "http://localhost:8000/all.schema-store.json",
  "$schema": "http://json-schema.org/draft-07/schema",
  "title": "JSON Schema documents for all AsyncAPI spec versions",
  "description": "All AsyncAPI JSON Schema documents listed in one file. Needed for serving all documents through schemastore.org",
  "type": "object",
  "if": {
    "properties": {
      "asyncapi": {
        "const": "3.0.0"
      }
    }
  },
  "then": {
    "$ref": "http://localhost:8000/3.0.0-without-$id.json"
  },
  "else": {
    "if": {
      "properties": {
        "asyncapi": {
          "const": "2.6.0"
        }
      }
    },
    "then": {
      "$ref": "http://localhost:8000/2.6.0-without-$id.json"
    },
    "else": {
      "if": {
        "properties": {
          "asyncapi": {
            "const": "2.5.0"
          }
        }
      },
      "then": {
        "$ref": "http://localhost:8000/2.5.0-without-$id.json"
      }
    }
  }
}

I just created a static file server locally with python3 -m http.server in the directory containing such schemas.

Then, I opened VS Code and created a test.asyncapi.yaml file with the very minimum content:

# yaml-language-server: $schema=http://localhost:8000/all.schema-store.json
asyncapi: 3.0.0
info:
  title: Test
  version: 1.0.0

Validation in VS Code works well. However, server logs show that all files are being fetched:

::ffff:127.0.0.1 - - [15/Dec/2023 11:46:39] "GET /all.schema-store.json HTTP/1.1" 200 -
::ffff:127.0.0.1 - - [15/Dec/2023 11:46:39] "GET /2.5.0-without-%24id.json HTTP/1.1" 200 -
::ffff:127.0.0.1 - - [15/Dec/2023 11:46:39] "GET /3.0.0-without-%24id.json HTTP/1.1" 200 -
::ffff:127.0.0.1 - - [15/Dec/2023 11:46:39] "GET /2.6.0-without-%24id.json HTTP/1.1" 200 -

[smoya]

The only alternative I find it could work but will require development on the Schema Store side (and its plugins) is to set a new field in the Schema Store API, that complements the fileMatch, but matching a field in the document to get the version (or called discriminator). The value could be a JSON Path or similar. Then, if present, plugins could use it in order to extract the right version of the document and point to the right schema.

@GerryFerdinandus WDYT about that? Does it make sense? Happy to turn it into a new feature request.

[hyperupcall]

I don't think I've seen @GerryFerdinandus active here in some time, but I also am a co-maintainer.

I think that sounds like an excellent idea. The idea of parsing a schema, and using it only if a property exists or has a certain value would also be useful to determine which schema to use if there are many matches, for a file with a name like manifest.json. We have many such cases, and such a solution might be able to solve both our cases at once.

What if we introduce a property called fieldMatch - an object where its sub-properties correspond to a particular version specified in the versions object?

"versions": {
    "1.0": "http://asyncapi.com/schema-store/1.0.0",
    "2.0": "http://asyncapi.com/schema-store/2.0.0"
},
"fieldMatch": {
    "1.0": ".version === '1.0' || .version === 1", /* syntax for example only */
    "2.0": ".version === '2.0' || .version === 2"
}

fieldMatch could also have a * property, which applies to all versions, satisfying my case as well.

What do you think? I'm thinking we could use JSON Schema to "validate" this instead of JSON Path to reduce a dependency (for consumers) and increase consistency - do you think there would be any downsides to that?

[jimmylewis]

Just gonna throw out there: try to get some buy-in from the owners of the VSCode, JetBrains, VS, etc. JSON language tooling (or whoever else uses schemastore, I'm sure there are others) in case they have any design concerns regarding how to parse and dynamically apply the schema conditions.

I think there's a need to be served here (and has been for a long time), but I worry that just implementing a feature in the catalog won't do as much good if it's not broadly adopted by the catalog consumers - especially if a custom syntax is required to apply filter conditions.

[hyperupcall]

@jimmylewis Thanks for your input - I most definitely agree with you. On the open source side, I have been planning to coordinate with the people that make VSCode, the Taplo's TOML extension, and Red Hat's YAML extension. In the case of VSCode, it is a bit of an oddball, because it doesn't seem to be dynamically selecting a JSON schema from catalog.json (that is why there is a JSON Schema extension), but they might have something to say anyways.

As for closed source Visual Studio and JetBrains, I wasn't sure how effective filing an issue in their issue tracker would be, but I don't think it would hurt to do so and gather feedback.

[smoya]

I'm happy this feels interesting to you folks.

I will open an issue later on this repository and in the repository hosting vscode as well (any others?).

We can then move the discussion to our issue and keep elaborating the idea

[smoya]

Additionally, happy to listen for your advice @jimmylewis regarding communication with implementers such as VSCode team, etc.
If you think there is a better alternative for reaching the owners directly, would definitely help

[hyperupcall]

@smoya That sounds good - once a new issue is filed about this "extension" here, I will be filing issues for the projects I mentioned in my previous comment. If you open an issue in VScode be sure to post its link so I don't create a duplicate 😁

[smoya]

The issue has been created. Very vague implementation example, but I feel we can keep the discussion there and come up with the best implementation covering several edge cases: #3460

cc @hyperupcall

Feel free to close this issue.