Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Description is not allowed for elements containing $ref #1514

Closed
vincnetas opened this issue Mar 19, 2018 · 30 comments
Closed

Description is not allowed for elements containing $ref #1514

vincnetas opened this issue Mar 19, 2018 · 30 comments

Comments

@vincnetas
Copy link

I know that specification says that when using $ref any other elements are ignored.
So here a real life scenario.

    location:
      description: User home address location  <- error
      $ref: "#/components/schemas/Location"
    carLocation:
      description: Where user has left his car  <- error
      $ref: "#/components/schemas/Location"

So if i have same structure of location but different semantics, only way to communicate this would be through field names, but that is very limiting.
Also two locations could be created with identical structure, just to distinguish between semantics, but this is code duplication.

I understand that this is not a trivial issue, but this is obvious shortcoming in spec which i noticed after just couple of days of basic real life usage... And i'm not the only one.

There are people with the same problem here
swagger-api/swagger-editor#1184 (comment)

@darrelmiller
Copy link
Member

darrelmiller commented Mar 19, 2018

There are few points regarding this:

  • This is a design choice that the OpenAPI specification inherited from JSON Schema. The JSON Schema folks are working on trying to find a solution to this issue. That would be the best place to have this conversation.
  • This is what appears at first to be a simple issue and then upon deeper inspection it becomes a non-trivial issue to resolve all the possible conflicts arising from merging two overlapping schemas. This is an issue that has been discussed literally for years.
  • The allOf operator can be used to define one schema to define the local properties, e.g. description and another schema with the $ref. It doesn't solve all scenarios and it is not the most concise syntax but it can work.
  • This can also be resolved by modelling the relationship between the user and the car as an object. So, the user object has a car object that has a location property. This has the added benefit of being able to attach other properties to the relationship between the user and the car and group those properties together.

@hkosova
Copy link
Contributor

hkosova commented Mar 22, 2018

Related (or duplicate): #556

vthib added a commit to Intersec/lib-common that referenced this issue Nov 26, 2019
In OpenAPI, components schemas can be declared independently, and then
referred to to avoid duplication.
However, when referencing an existing schema, no other information can
be provided. This means that a field of a struct whose type has its own
schema must use a '$ref', but then cannot specify anything else,
especially no default value, no description, no constraints...

This is a pretty bad overlook from the OpenAPI spec:
�ihttps://github.com/OAI/OpenAPI-Specification/issues/1514

The proposed workaround is to use an untyped schema with all the
details, and push the $ref inside an allOf clause, on its own.

This is what is done in this commit. As you can see, this is pretty bad
semantically, and I can't really imagine expecting a client to properly
understand what it means, but that's the solution...

Change-Id: Ifdba5fe6a7a2494aa136ec3ea3f0952edd45d6d0

rip-it: 0c86011
@avantipande1
Copy link

avantipande1 commented Dec 24, 2019

I was looking for same use case where I want to add description for the $ref field in swagger. Above answer helped, when we add description with allOf, swagger-ui picks up this description. Now my problem is how to generate allOf programmatically? Is there any annotation that can be used to generate allOf. I am using swagger-maven-plugin with @apimodel and @ApiModelProperty to generate swagger from APIs.
Any help would be greatly appreciated.
Thank you

jcoyne added a commit to sul-dlss/cocina-models that referenced this issue Jul 7, 2020
See OAI/OpenAPI-Specification#1514
The openapi parser raises an error when this happens
@greatvovan
Copy link
Contributor

This looks like a serious design flaw in both OpenAPI and JSON Schema.

@LikeLakers2
Copy link

LikeLakers2 commented Oct 31, 2020

  • This is a design choice that the OpenAPI specification inherited from JSON Schema. The JSON Schema folks are working on trying to find a solution to this issue. That would be the best place to have this conversation.

This part of the comment appears to be out-of-date, it seems? The version of the JSON Schema Core used by 3.1.0-RC1 does not seem to explicitly disallow additional properties being beside $ref, as seen at the example in section 7.7.1.1, although that same section does appear to note that "The application programmer and the schema author need to agree on the usage" in regards to this. (I might be misunderstanding this, though)

Additionally, within the changelog for that version of JSON Schema Core, there is this note:

  • Allow keywords adjacent to "$ref"

which would also support that keywords can be used next to the $ref property. However, OpenAPI still disallows additional properties next to $ref fields, except for summary and description. I will also note section 7.2 here, as it seems to regard this very issue (though maybe I'm misunderstanding section 7.2 about this?).

If I might suggest something -- perhaps specification extensions could be allowed next to $ref properties, so long as they are about the Reference Object itself? I can provide an example of what I mean, if you'd like.

If this is off-topic for this issue, I'd be happy to open a new issue regarding this.

@greatvovan
Copy link
Contributor

Nice research @LikeLakers2, I think it worth opening a new issue as I believe OpenAPI must eventually move towards a graceful resolution of this problem.

Also you write

OpenAPI still disallows additional properties next to $ref fields, except for summary and description.

This page explicitly says that any such properties will be just ignored. In what way does it disallow additional properties?

@LikeLakers2
Copy link

LikeLakers2 commented Oct 31, 2020

In what way does it disallow additional properties?

Bad wording on my part. Woops.

Anyways, I'll open an issue soon. And I'll make sure to use the right wording this time, haha.

@darrelmiller
Copy link
Member

@LikeLakers2 @greatvovan

In OpenAPI 3.1 you can use any JSON Schema property alongside a $ref in a schema object.

However, $ref used in other OAS objects like parameter, requestBody, can only have summary and description as peers.

JSON Schema has done the hard work to clearly define how peer properties interact with $ref. In OpenAPI we are going to have to define the semantics of peer properties on a case by case basis. We have allowed summary and description as they don't impact behavior. We are open to considering other properties but we will need to do that carefully.

Regarding the swagger.io doc, I believe that wording is incorrect. in 3.0 of OAS you are not allowed to include properties alongside $ref, they would cause a validation error.

@LikeLakers2
Copy link

LikeLakers2 commented Oct 31, 2020

@darrelmiller That... feels confusing. Why do we have two different definitions for $ref based on what object it's replacing?

Also,

Regarding the swagger.io doc, I believe that wording is incorrect. in 3.0 of OAS you are not allowed to include properties alongside $ref, they would cause a validation error.

If the swagger.io doc is incorrect, then the specification itself should be updated.

In the Reference Object documentation for both v3.0.3 and v3.1.0-RC1, and even on v3.1.0 on the development branch (as of the time of this comment), the following text is found regarding additional properties:

This object cannot be extended with additional properties and any properties added SHALL be ignored.

The use of the word "ignore" here would imply that additional properties (including specification extensions) can be in the object, but will never be parsed. If that isn't what is meant, then the wording in the specification should be fixed.

@darrelmiller
Copy link
Member

This object cannot be extended with additional properties and any properties added SHALL be ignored.

My mistake. I thought the spec explicitly disallowed those properties.

Why do we have two different definitions for $ref based on what object it's replacing?

We have the OAS Reference object and JSON schema has a $ref object. We use the same keyword because they have approximately the same behavior. This is the impact of enabling full JSON Schema support. JSON Schema is able to identify every property as either an annotation or a constraint. It has defined how annotations and constraints can be combined. Every property in JSON schema can be evaluated independently. In OAS the relationships between properties is more complex. Allowing arbitrary inclusion of peer properties to a $ref could create some very confusing situations.

e.g. Here's just one example

paths:
   '/{foo}':
     parameters:
        - $ref: #/components/parameters/foo
           in: path
     
components:
  parameters:
     foo:
       in: query
       required: false
       schema:
         type: string

foo is not required, but path parameters are. Therefore we need to define if this reuse is allowed, and what that means for the required property. I suspect there are a dozens more examples like this. Not impossible to resolve, but will take time and add complexity.

@LikeLakers2
Copy link

LikeLakers2 commented Oct 31, 2020

My mistake. I thought the spec explicitly disallowed those properties.

The wording is admittedly a bit vague -- I can see why it might trip you up, even if you knew it said that. It says "cannot be extended" only to say that additional properties will "be ignored". "Cannot be extended" would imply that you are disallowed from even including them, whereas "be ignored" would imply that you can include them if you don't mind the parser ignoring them. However, both of them being in reference to how an object can be structured, means that the actual meaning might be vague -- and parsers will probably take the least restrictive route, just-in-case.

So I think it may be worth clarifying this wording anyways... maybe replace the line with something like "Any additional properties (including specification extensions) SHALL be ignored."

[example]

Ah, I think I understand the difference now. Thanks. I can see why that might be hard to deal with.

Is there any active issue regarding trying to figure that out?


As an aside, it may be worth clarifying that $ref in Schema Objects is different than $ref in Reference Objects. I notice that this is already done in the Reference Objects section, but it may be worth including such a thing under the Schema Objects section too -- especially for people like me who would be transitioning from 3.0 to 3.1.

@greatvovan
Copy link
Contributor

I have a question about wording here:

If the referenced object-type does not define a description (summary) field, then this field has no effect.

Does it mean that if the schema object by the reference does not have a description but the Reference object has description, then the "resulting" object will not anyway have description? If so, why? Why can't it just ADD-OR-OVERRIDE the field?

@darrelmiller
Copy link
Member

darrelmiller commented Nov 1, 2020

@greatvovan OAS Reference Objects are no longer used by Schema Objects in OAS. The comment quoted above is only used for other components. e.g If you $ref a Response Object and put a "summary" property in the Reference Object it will be ignored because Response Object does not have a summary property.

We don't want to be in a position where allowing a property in a Reference Object implicitly means it can be added to any other component.

@greatvovan
Copy link
Contributor

Got it.

Maybe rephrase it then as "does not allow" instead of "does not define"?

@MikeRalphson
Copy link
Member

Maybe rephrase it then as "does not allow" instead of "does not define"?

Might be an improvement. PR welcome!

@LikeLakers2
Copy link

@darrelmiller I saw you gave a thumbs up to my comment -- do you want me to open a PR to make those suggested changes?

jerstlouis added a commit to jerstlouis/ogcapi-tiles that referenced this issue Dec 24, 2021
…rsion only

- swagger-cli was validating the schemas, bundling and validating the bundle without any warning
- SwaggerUI was showing the API as invalid with these messages:
   "attribute components.schemas.tileSetMetadata.items is missing"
- A $ref fully replaces sibling properties and should always be used alone:
   "Any members other than "$ref" in a JSON Reference object SHALL be ignored."
   https://datatracker.ietf.org/doc/html/draft-pbryan-zyp-json-ref-03#section-3
- See also:
   https://stackoverflow.com/questions/33565090/json-schema-property-description-and-ref-usage
   OAI/OpenAPI-Specification#556
   OAI/OpenAPI-Specification#1514
   swagger-api/swagger-editor#1184
- NOTE: We may need to update the 2DTMS schemas as well with this in mind
jerstlouis added a commit to jerstlouis/ogcapi-tiles that referenced this issue Dec 24, 2021
- A $ref fully replaces sibling properties and should always be used alone:
   "Any members other than "$ref" in a JSON Reference object SHALL be ignored."
   https://datatracker.ietf.org/doc/html/draft-pbryan-zyp-json-ref-03#section-3
- See also:
   https://stackoverflow.com/questions/33565090/json-schema-property-description-and-ref-usage
   OAI/OpenAPI-Specification#556
   OAI/OpenAPI-Specification#1514
   swagger-api/swagger-editor#1184
- NOTE: We may need to update the 2DTMS schemas as well with these changes
jerstlouis added a commit to jerstlouis/ogcapi-tiles that referenced this issue Dec 24, 2021
- A $ref fully replaces sibling properties and should always be used alone:
   "Any members other than "$ref" in a JSON Reference object SHALL be ignored."
   https://datatracker.ietf.org/doc/html/draft-pbryan-zyp-json-ref-03#section-3
- See also:
   https://stackoverflow.com/questions/33565090/json-schema-property-description-and-ref-usage
   OAI/OpenAPI-Specification#556
   OAI/OpenAPI-Specification#1514
   swagger-api/swagger-editor#1184
- NOTE: We may need to update the 2DTMS schemas as well with these changes
@mohamnag
Copy link

this is a flaw in OpenAPI spec and use of allOf is a dirty workaround. meta-data like description beside $ref is a meaningful use-case and not providing a solution by just saying because upstream does this, is not a proper way of handling a problem.

@webron
Copy link
Member

webron commented Jan 17, 2022

Thank you for your comment. We've taken it under consideration and implemented it differently in OpenAPI 3.1. We hope you find the solution satisfactory.

@mohamnag
Copy link

mohamnag commented Jan 18, 2022

@webron thanks for mentioning this. It is not clear from this issue what the solution and outcome was. it is not also clear for me by just looking at current spec of 3.1 (https://spec.openapis.org/oas/v3.1.0) how this is solved. I can only see there is a "Reference Object" that can contain description and summary beside $ref however as this object does not replace a "Schema Object" with $ref, I can not see how this is solving mentioned problem in general.

for example given following sample:

components:
  schemas:
    SomeSchemaDef:
      type: object
      properties:
        someProp:
          $ref: '#/components/schemas/SomeOtherSchemaDef'
          # here I would like to add a `description` for `someProp`

what is the solution that OpenAPI 3.1 provides?

@blakew
Copy link

blakew commented Apr 8, 2022

Thank you for your comment. We've taken it under consideration and implemented it differently in OpenAPI 3.1. We hope you find the solution satisfactory.

@webron Hi - what is the solution in 3.1?

@webron
Copy link
Member

webron commented Apr 8, 2022

@blakew you can use description alongside $ref.

@blakew
Copy link

blakew commented Apr 8, 2022

Within schemas you can only use $ref and not reference objects. The docs note this distinction:

Note that this restriction on additional properties is a difference between Reference Objects and Schema Objects that contain a $ref keyword.

I'm particularly interested in a solution for including descriptions within schemas.

@webron
Copy link
Member

webron commented Apr 8, 2022

Correct. Technically you can create and use your own JSON Schema vocabulary that will also allow it in schemas as well.

@jdesrosiers
Copy link
Contributor

@blakew The JSON Schema dialect used in OpenAPI 3.1 is based on JSON Schema 2020-12 which does allow you to put a description (or anything else) alongside $ref. You are correct that $ref in JSON Schema this is not the same as a Reference Object in OpenAPI. There is a significant difference in the way that they work, but they will appear to work the same in most cases. In OpenAPI, the description in the Reference Object will override the description in the referenced object (if there is one). In JSON Schema, there is no overriding. The description in both schemas apply to the instance. However, in the case of keywords like description that don't affect validation results, implementations such as documentation generators are allowed to choose one description if given multiple.

@Xbloud
Copy link

Xbloud commented Apr 21, 2022

I am not sure I understand the difference between object reference and object schema. Can I include custom properties - extensions - next to the $ref? For example:

components:
  schemas:
    SomeSchemaDef:
      type: object
      properties:
        someProp:
          $ref: '#/components/schemas/SomeOtherSchemaDef'
          # here I would like to add 'x-my-extension' for 'someProp'

@jdesrosiers
Copy link
Contributor

A "Schema Object" is what the documentation calls any part of an OpenAPI document that represents a JSON Schema such as /components/schemas/SomeSchemaDef. The references ($ref) in a "Schema Object" are JSON Schema references and not OpenAPI references. An OpenAPI reference (also $ref) is called a "Reference Object" and covers any references that aren't in a "Schema Object".

So, in your example, yes, you can add any additional keywords you like next to $ref because it's in a "Schema Object". You don't even need the x- if you don't want to.

@michaelcilibrasi
Copy link

I didn't follow this tread, it got fragmented quickly. What's the bottom line for:

    location:
      description: User home address location  <- error
      $ref: "#/components/schemas/Location"

Is there a workaround?

@hkosova
Copy link
Contributor

hkosova commented Jan 18, 2023

@michaelcilibrasi your example works in OpenAPI 3.1.

In earlier versions, you need to wrap the $ref into allOf:

    location:
      description: User home address location  <- error
      allOf:
        - $ref: "#/components/schemas/Location"

@michaelcilibrasi
Copy link

hkosova, I forgot to thank you for your response.

@Kostadin-Ivanov
Copy link

I know that specification says that when using $ref any other elements are ignored. So here a real life scenario.

    location:
      description: User home address location  <- error
      $ref: "#/components/schemas/Location"
    carLocation:
      description: Where user has left his car  <- error
      $ref: "#/components/schemas/Location"

So if i have same structure of location but different semantics, only way to communicate this would be through field names, but that is very limiting. Also two locations could be created with identical structure, just to distinguish between semantics, but this is code duplication.

I understand that this is not a trivial issue, but this is obvious shortcoming in spec which i noticed after just couple of days of basic real life usage... And i'm not the only one.

There are people with the same problem here swagger-api/swagger-editor#1184 (comment)

It would be very handy so if we can prepare some defined structure, lets say a responseBody, and then in each API request,
we just $ref the responseBody and, somehow :), below add list of fields/entries with some example value, which then can be populated in the defined responseBody.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests