When using "all_refs=True" to generate a jsonschema, validation fails because the top-level object has invalid reference url

[gshank]

• mashumaro version: latest
• Python version: 3.10
• Operating System: MacOS

Description

I have the conversion from our old jsonschema generation app (hologram) to mashumaro working okay without the "all_refs" parameter turned on, but that makes the generated jsonschema very large. When I turn on "all_refs", it generates a much smaller jsonschema, but validation fails because the top-level object is a ref.

What I Did

If you look at dbt-labs/dbt-core#8132, the schemas/dbt/manifest/v11.json file was generated from mashumaro (it's big and might not load in the browser). The previous generated file was schemas/dbt/manifest/v10.json, for comparison purposes.

When I modify core/dbt/dataclass_schemas.py, dbtClassMixin.json_schema to use "all_refs=True" to the "build_json_schema" call, the generated schema (added to the schemas directory by scripts/collect-artifact-schema.py --path schemas --artifact manifest) fails the schema tests (tests/functional/artifacts) with the error message: jsonschema.exceptions.RefResolutionError: Unresolvable JSON pointer: 'defs/WritableManifest'

The WritableManifest is the top-level object that I am creating the jsonschema for, so my guess is that the base object cannot be a ref, but this is not my area of expertise. Let me know if I should try to create a test case.

[Fatal1ty]

Hi @gshank

This is a very interesting issue!

so my guess is that the base object cannot be a ref, but this is not my area of expertise

I thought so at first too and went to look for confirmation which led me to this:

• Keywords alongside of $ref json-schema-org/json-schema-spec#523

So, it's legit to have "$ref" and "$defs" keywords in the root schema. But something was totally wrong here. After taking a close look at difference between v11.json and an example from the linked issue I figured out that problem was with "$". By mistake, we use prefix "#/defs" instead of "#/$defs" for 2020-12 dialect. So, instead of this:

"$ref": "#/defs/WritableManifest"

there should be this:

"$ref": "#/$defs/WritableManifest"

I can confirm that after passing correct ref_prefix parameter to build_json_schema and running python scripts/collect-artifact-schema.py --path schemas --artifact manifest all the functional tests are successful:

json_schema_obj = build_json_schema(cls, all_refs=True, ref_prefix="#/$defs")

pytest tests/functional/artifacts
================================================================================ test session starts ================================================================================
platform darwin -- Python 3.10.1, pytest-7.3.2, pluggy-1.2.0
rootdir: /Users/tikhonov_a/projects/dbt-core
configfile: pytest.ini
plugins: csv-3.0.0, logbook-1.2.0, flaky-3.7.0, cov-4.1.0, mock-3.11.1, dotenv-0.5.2, ddtrace-1.17.0, xdist-3.3.1
collected 15 items

tests/functional/artifacts/test_artifact_fields.py .                                                                                                                          [  6%]
tests/functional/artifacts/test_artifacts.py .....                                                                                                                            [ 40%]
tests/functional/artifacts/test_docs_generate_defer.py .                                                                                                                      [ 46%]
tests/functional/artifacts/test_override.py .                                                                                                                                 [ 53%]
tests/functional/artifacts/test_previous_version_state.py ....                                                                                                                [ 80%]
tests/functional/artifacts/test_run_results.py ...                                                                                                                            [100%]

================================================================================ 15 passed in 37.19s ================================================================================

I will change the default ref_prefix value in JSONSchemaDraft202012Dialect class but you're encouraged to double check if it resolves this issue.

[Fatal1ty]

Hey @gshank

I merged #130, so if everything works now, let me know and I'll prepare a release.