Collection Schemas

[alexrobin]

I'd like to find a way to provide schemas for collections in a way that works for any encoding/representation of items in these collections.

Using a link with describedby relation type inside the collection response itself works only if the collection format can itself encode links. I works fine with JSON but it's not necessarily the case for binary formats such as protobuf, avro or flatbuffers.

So I started experimenting with another approach using link anchors (see IETF Web Linking standard), which allows the describedby link to be provided outside of the collection data itself (this way it can be discovered before the client fetches data from the collection).

This is an example description of a feature collection making data available both as geojson and protobuf that follows this pattern:

{
      "id": "all_deployments",
      "title": "All System Deployments",
      "description": "All deployments involving systems registered on this server",
      "itemType": "feature",
      "featureType": "deployment",
      "links": [
        {
          "rel": "items",
          "title": "Access the features in this collection",
          "href": "https://data.example.org/api/deployments?f=json",
          "type": "application/json"
        },
        {
          "rel": "describedby",
          "title": "Schema for features in the collection",
          "anchor": "https://data.example.org/api/deployments?f=json",
          "href": "https://data.example.org/api/deployments/schema?f=json",
          "type": "application/schema+json"
        },
        {
          "rel": "items",
          "title": "Access the features in this collection",
          "href": "https://data.example.org/api/deployments?f=protobuf",
          "type": "application/x-protobuf"
        },
        {
          "rel": "describedby",
          "title": "Schema for features in the collection",
          "anchor": "https://data.example.org/api/deployments?f=protobuf",
          "href": "https://data.example.org/api/deployments/schema?f=protobuf",
          "type": "application/x-protobuf-schema"
        }
      ]
    }

@cportele what do you think about this?

[cportele]

@alexrobin

Yes, absolutely, you can use use anchor for this.

Some nits:

• https://data.example.org/api/deployments is not the URI of a Features resource.
• There is no registered media type for Protocol Buffers. There is an expired IETF draft for application/protobuf which seems to be used by some, others use application/x-protobuf. In any case, I have never seen a media type for the schemas so far.

In general, including the links in JSON payloads is an OGC API convention for convenience. The standard mechanism is to sent the links in response headers, which also works for encodings that cannot represent links.

[alexrobin]

@cportele

Yes, I know there is no media type for protocol buffers, that's a shame... (that's why I used x-protobuf for now).

You're right that we could use Link headers in the case of HTTP binary payload, but it's still useful for us to know the schema ahead of time. It's the pattern used by most of the flexible binary formats out there such as protobuf, flatbuf, avro, thrift, etc. and this way it's also usable when connecting using streaming and pub/sub protocols.

Why do you say https://data.example.org/api/deployments is not the URI of a feature? The intent here is that it's the canonical URI for the items of the feature collection. I mentioned this before in another issue, but my intent is to have multiple "soft collections" that can include the same feature that would be located at a canonical URL.

Are you saying it's not correct because it doesn't follow the pattern collections/{id}/items like so:
https://data.example.org/api/collections/all_deployments/items?

Indeed, I noticed that the OGC API Features spec says that the server must respond to such URL but I didn't think it had to be the only way to access the collection items.

If clients are meant to follow the links anyway, I guess the link pattern doesn't really matter, does it?

[cportele]

Why do you say https://data.example.org/api/deployments is not the URI of a feature?

I said "not the URI of a Features resource", which refers to the Features resource type as specified in OGC API Features, not features in general. Of course, an API can serve features at any URI as needed by the publisher / API designer.

[alexrobin]

Yes, sorry, I was not specific enough but I really meant Features resource as well.

But I'm still not clear why this is not the URI of a Features resource as per OGC API Features. The only thing I read in the standard is this:

Requirement 33.A: "For every feature collection identified in the feature collections response (path /collections), the server SHALL support the HTTP GET operation at the path /collections/{collectionId}/items."

But it doesn't say this has to be the canonical URL of the Features resource.

I thought this requirement was just intended to define a default path that all servers have to implement so that clients that don't follow the provided web links can still find features at the same place. But in my case, I'd rather use a different canonical URLs for resources, even though they will also be accessible through this URL.

Do you think this can cause compatibility issues with Features API clients?

[cportele]

But it doesn't say this has to be the canonical URL of the Features resource.

The requirement defines the URI of the Features resource (relative to the landing page). A Connected Systems API can serve the same feature content at another URI, too, which of course makes it a different resource. You can create relationships between the resources to state their relationship and you could declare one of the URIs as the canonical one, that is, the preferred URI for the two resources with duplicative content.

It is up to Connected Systems to specify the requirements for those other resources providing features. It may be possible to "reuse" the requirements in OGC API Features regarding Features resources by some kind of reference to them, I never thought about that.

I thought this requirement was just intended to define a default path that all servers ...

It is not a "default path", it is the only path for a resource that provides features according to OGC API Features Part 1. Requirements for other URIs providing features are out-of-scope of Part 1. It is perfectly valid for clients accessing feature to rely on the paths specified in OGC API Features Part 1 and not support features at any other URI pattern.

[alexrobin]

Ok, got it. Thanks.