Add `$self` for self-identifying documents #4389

handrews · 2025-02-28T15:07:53Z

See the proposal for background information.

This is a rather minimal approach, as @karenetheridge and I plan to work together on a more thorough revamp of the document parsing / reference resolution sections.

This adds $self as a way for a document to define its own URI for use in reference targets, and as the base URI for relative URI references in the document.

This does not impact the resolution of relative API URLs. [NOTE: I'm not entirely sure about this, but it seems more useful this way to allow multiple deployed locations of an OAD to correspond to multiple deployments of the API.)

Tick one of the following options:

schema changes are included in this pull request
schema changes are needed for this pull request but not done yet
no schema changes are needed for this pull request

lornajane

This looks pretty solid, but could we introduce use of $self in one or two examples so people realise it's there?

ralfhandl · 2025-03-04T21:26:58Z

Looks good. An example would be nice.

handrews · 2025-03-06T05:07:05Z

@lornajane @ralfhandl I thought I did include one but apparently I did that on an earlier attempt on a different branch. I'll port it over in the morning.

karenetheridge

A few comments/questions throughout.

src/oas.md

src/schemas/validation/schema.yaml

handrews · 2025-03-19T19:30:28Z

@lornajane @ralfhandl @karenetheridge I ended up reworking this a bit as I added examples.

ralfhandl

+1, after correcting the resolved URLs in the example.

src/oas.md

karenetheridge

I think that's all my comments for this pass.

They will read a bit confusingly though because the order in which they appear in the document is nearly the reverse of that in which I wrote them -- as I had to get down to your examples before I really understood what you're going for, and then I had to go back and drop in some questions higher up where the concepts are first mentioned! (sorry!)

Actually never mind, it looks like github has changed how it does things since I last had a similar complaint -- and the comments are added here in chronological order, not in the order in which they appear in the file! yay github.

src/oas.md

handrews · 2025-04-10T17:04:20Z

@notEthan and others, after talking with @karenetheridge and the rest of the TDC call today, I am going to split out all of the things about further clarification of Server Object behavior into another discussion or issue, and mark things resolved or (if that is not available) off-topic. This does NOT mean that the comment was off-topic in a bad way, just that there was a request on the TDC call to make this PR easier to read and focus on the remaining issues needed to get this across the finish line.

handrews · 2025-04-10T17:30:37Z

Filed issues:

v3.2: Consider additional guidance around $self usage #4539
v3.2: Consider moving guidance on supplying a retrieval URL (and other parts of the spec that justify various features) to the Learn site #4540
v3.2: Should API URLs in general, and the Server Object url field in particular, still only use the retrieval URL as the base URL #4541
v3.2: Consider additional guidance on Server Objects #4542

This adds `$self` as a way for a document to define its own URI for use in reference targets, and as the base URI for relative URI references in the document. This does not impact the resolution of relative API URLs.

Co-authored-by: Ralf Handl <ralf.handl@sap.com>

Co-authored by: Karen Etheridge <ether@cpan.org>

Co-authored-by: Karen Etheridge <github@froods.org>

This ties the `$self` behavior more directly to the sections on API Description URI usage, including examples, and also expands on the use cases for manually providing a retrieval URI.

Co-authored-by: Ralf Handl <ralf.handl@sap.com>

Also tweak the wording a bit as a result, and ran format-markdown.

Co-authored-by: Ralf Handl <ralf.handl@sap.com>

This will allow us to implement the separation of retrieval URL from "$self" in JSON::Schema::Modern::Document::OpenAPI and OpenAPI::Modern (to be added to the OpenAPI Specification in v3.2). see OAI/OpenAPI-Specification#4389

see OAI/OpenAPI-Specification#4389

mikekistler

Looks good. 👍

I left a few minor comments I hope can be addressed in this PR, but do no object to deferring these to a follow-on PR.

mikekistler · 2025-04-20T22:24:26Z

src/oas.md

+1. The base URI used to resolve `$self` is the retrieval URI, resulting in an effective `$self` value of `https://example.com/openapi` (OADs that use HTTP content negotiation to provide both JSON and YAML representations typically do not use a file extension in their self-assigned URI, to avoid needing to change URIs based on the format).
+2. The base URI for the `$ref` comes from `$self` (`https://example.com/openapi`), producing a resolved URI of `https://example.com/shared#/components/pathItems/Foo`.
+
+Relative `$self` values are often used for APIs deployed in multiple locations, such as a device management API that is hosted on each device.


I'm going to reveal my ignorance here, but this seems like the exact situation where you want to use absolute $self value. I thought the idea of distinguishing URIs -- identifiers -- from URLs -- locators -- is that you could have the same document in multiple locations and would want to make clear that it is the same document -- with a single identifier -- even though it could be retrieved from multiple locations.

@mikekistler I just did $self by analogy with JSON Schema's $id. If folks want to deviate from that (and as you note there are reasons to do so for simplicity) then we can, but let's decide that intentionally. I think @karenetheridge brought up this in some way as well? I admit I've somewhat lost track at this point.

I do not have a use case where you would want the identity to change based on the location, although I'm sure I could construct one where it's somehow beneficial. But maybe not so beneficial as to be worth discarding the simplicity of requiring an absolute URI.

Notably, the newly proposed JSON Schema replacement JSON Structure requires an absolute $id (although they also say it SHOULD be a URL which I need to reply to because that is missing the point of $id entirely in my view and guiding people to the wrong behavior).

I have filed issue #4548 to debate the absolute-URI idea, along with several other spin-off issues previously filed. Although perhaps it is worth taking a step back and looking at all of those "spin-off" issues comprehensively. Honestly I'm not sure where to go with this as people keep having fundamental objections that could substantially change the whole thing, and those aren't good questions to resolve in a PR. Those are discussion or issue questions, really.

mikekistler · 2025-04-20T22:30:07Z

src/oas.md

+  description: The test API on this device
+```
+
+For API URLs, the `$self` field, which identifies the OpenAPI Document, is ignored, and the retrieval URL is used instead. This produces a normalized production URL of `https://device1.example.com`, and a normalized test URL of `https://device1.example.com/test`.


This may be a nit but here we use "retrieval URL" but above (line 323) we use "retrieval URI". I'm guessing that these are referring to the same thing, and if so I think we should try to use consistent terminology.

Yeah generally a retrieval URI is a URL by definition, but it gets weird when you are passing the retrieval URI explicitly, because in that context it might not be usable as a URL (e.g. you are passing it explicitly so that the tool treats it as if it came from a location that is currently inaccessible due to network conditions or whatever).

mikekistler · 2025-04-20T22:34:30Z

src/oas.md

@@ -342,6 +425,7 @@ This is the root object of the [OpenAPI Description](#openapi-description).
 | Field Name | Type | Description |
 | ---- | :----: | ---- |
 | <a name="oas-version"></a>openapi | `string` | **REQUIRED**. This string MUST be the [version number](#versions) of the OpenAPI Specification that the OpenAPI Document uses. The `openapi` field SHOULD be used by tooling to interpret the OpenAPI Document. This is _not_ related to the API [`info.version`](#info-version) string. |
+| <a name="oas-self"></a>$self | `string` | This string MUST be in the form of a URI-reference, MUST NOT be an empty string, and MUST NOT contain a fragment (empty or otherwise). The `$self` field provides the self-assigned URI of this document, which also serves as its base URI in accordance with [[RFC3986]] [Section 5.1.1](https://www.rfc-editor.org/rfc/rfc3986#section-5.1.1). Implementations MUST support identifying the targets of [API description URIs](#relative-references-in-api-description-uris) using the resolved URI defined by this field, as shown under [Examples of Base URI Determination and Reference Resolution](#examples-of-base-uri-determination-and-reference-resolution). |


I think it would be good to explain -- or point to the explanation of -- the expected behavior when this field is not present.

@mikekistler if that's covered under "Examples of Base URI Detrermination.." is that sufficient?

handrews added enhancement re-use: ref/id resolution labels Feb 28, 2025

handrews added this to the v3.2.0 milestone Feb 28, 2025

handrews requested review from a team as code owners February 28, 2025 15:07

lornajane reviewed Mar 4, 2025

View reviewed changes

karenetheridge reviewed Mar 6, 2025

View reviewed changes

src/oas.md Outdated Show resolved Hide resolved

src/oas.md Show resolved Hide resolved

src/oas.md Outdated Show resolved Hide resolved

src/oas.md Outdated Show resolved Hide resolved

src/schemas/validation/schema.yaml Show resolved Hide resolved

karenetheridge mentioned this pull request Mar 8, 2025

Disallow fragments in uri-references that point elsewhere in the OpenAPI description #4423

Closed

3 tasks

ralfhandl linked an issue Mar 19, 2025 that may be closed by this pull request

OAS3 Tag/template to define external-components URL #1979

Open

handrews force-pushed the self2 branch from 2da9317 to 3364fc6 Compare March 19, 2025 19:29

handrews mentioned this pull request Mar 20, 2025

Open Community (TDC) Meeting, Thursday 20 March 2025 #4429

Closed

ralfhandl requested changes Mar 21, 2025

View reviewed changes

src/oas.md Outdated Show resolved Hide resolved

ralfhandl previously approved these changes Mar 21, 2025

View reviewed changes

ralfhandl dismissed their stale review via 5013d41 March 21, 2025 17:34

ralfhandl previously approved these changes Mar 21, 2025

View reviewed changes

This comment was marked as off-topic.

Sign in to view

handrews force-pushed the self2 branch from 5013d41 to 97c8a45 Compare March 21, 2025 17:38

This comment was marked as off-topic.

Sign in to view

handrews dismissed ralfhandl’s stale review via ff315fa March 23, 2025 15:15

handrews force-pushed the self2 branch from 97c8a45 to ff315fa Compare March 23, 2025 15:15

This comment was marked as off-topic.

Sign in to view

karenetheridge reviewed Mar 25, 2025

View reviewed changes

handrews mentioned this pull request Mar 26, 2025

Style Guide: Periods at the end of bullet lists? #4502

Closed

handrews and others added 11 commits April 10, 2025 10:32

Add $self for self-identifying documents

6e129a2

This adds `$self` as a way for a document to define its own URI for use in reference targets, and as the base URI for relative URI references in the document. This does not impact the resolution of relative API URLs.

Rework and include examples.

604789f

Fix example

5e3c600

Co-authored-by: Ralf Handl <ralf.handl@sap.com>

Make schema check more precise for $self

f469211

Co-authored by: Karen Etheridge <ether@cpan.org>

Apply suggestions from code review

43cb464

Co-authored-by: Karen Etheridge <github@froods.org>

Further clarifications from review.

1294a86

This ties the `$self` behavior more directly to the sections on API Description URI usage, including examples, and also expands on the use cases for manually providing a retrieval URI.

Apply suggestions from code review

2e5bfbc

Co-authored-by: Ralf Handl <ralf.handl@sap.com>

Align type field for $self with style guide.

e0a2cb6

Also tweak the wording a bit as a result, and ran format-markdown.

Review feedback

d68683c

Co-authored-by: Ralf Handl <ralf.handl@sap.com>

Review feedback from @notEthan

7568b58

Better URI terminology, I hope

34fb6d6

handrews force-pushed the self2 branch from 0baaa64 to 34fb6d6 Compare April 10, 2025 17:32

ralfhandl previously approved these changes Apr 11, 2025

View reviewed changes

ralfhandl requested a review from a team April 11, 2025 09:11

karenetheridge added a commit to karenetheridge/OpenAPI-Modern that referenced this pull request Apr 11, 2025

support $self keyword coming in v3.2

d3027c0

see OAI/OpenAPI-Specification#4389

mikekistler previously approved these changes Apr 20, 2025

View reviewed changes

ralfhandl requested a review from notEthan April 22, 2025 07:37

handrews mentioned this pull request Apr 23, 2025

v3.2: Consider requiring $self to be an absolute-URI #4548

Open

handrews dismissed stale reviews from mikekistler and ralfhandl via 9aec900 April 23, 2025 19:27

handrews mentioned this pull request Apr 23, 2025

Open Community (TDC) Meeting, Thursday 24 April 2025 #4545

Open

More review feedback

8d1c6be

handrews force-pushed the self2 branch from 9aec900 to 8d1c6be Compare April 23, 2025 22:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `$self` for self-identifying documents #4389

Add `$self` for self-identifying documents #4389

handrews commented Feb 28, 2025 •

edited by karenetheridge

Loading

lornajane left a comment

ralfhandl commented Mar 4, 2025

handrews commented Mar 6, 2025 •

edited

Loading

karenetheridge left a comment

handrews commented Mar 19, 2025

ralfhandl left a comment

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

karenetheridge left a comment •

edited

Loading

handrews commented Apr 10, 2025

handrews commented Apr 10, 2025 •

edited

Loading

mikekistler left a comment

mikekistler Apr 20, 2025

handrews Apr 23, 2025

handrews Apr 23, 2025

mikekistler Apr 20, 2025

handrews Apr 23, 2025

mikekistler Apr 20, 2025

handrews Apr 23, 2025

Add $self for self-identifying documents #4389

Are you sure you want to change the base?

Add $self for self-identifying documents #4389

Conversation

handrews commented Feb 28, 2025 • edited by karenetheridge Loading

lornajane left a comment

Choose a reason for hiding this comment

ralfhandl commented Mar 4, 2025

handrews commented Mar 6, 2025 • edited Loading

karenetheridge left a comment

Choose a reason for hiding this comment

handrews commented Mar 19, 2025

ralfhandl left a comment

Choose a reason for hiding this comment

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

karenetheridge left a comment • edited Loading

Choose a reason for hiding this comment

handrews commented Apr 10, 2025

handrews commented Apr 10, 2025 • edited Loading

mikekistler left a comment

Choose a reason for hiding this comment

mikekistler Apr 20, 2025

Choose a reason for hiding this comment

handrews Apr 23, 2025

Choose a reason for hiding this comment

handrews Apr 23, 2025

Choose a reason for hiding this comment

mikekistler Apr 20, 2025

Choose a reason for hiding this comment

handrews Apr 23, 2025

Choose a reason for hiding this comment

mikekistler Apr 20, 2025

Choose a reason for hiding this comment

handrews Apr 23, 2025

Choose a reason for hiding this comment

Add `$self` for self-identifying documents #4389

Add `$self` for self-identifying documents #4389

handrews commented Feb 28, 2025 •

edited by karenetheridge

Loading

handrews commented Mar 6, 2025 •

edited

Loading

karenetheridge left a comment •

edited

Loading

handrews commented Apr 10, 2025 •

edited

Loading