Ability to set no null/empty geometry in returned response

[ThomasG77]

Most of the data consumers from WFS need the geometry but sometimes, the service can provide data for charts or HTML tables where geometry would slow down the data retrieval due to geometry size in response.

I'm not sure if the specification implicitly manage this use case as I didn't found a way to do it from my understanding of the current Core spec.

[cportele]

The idea so far is to add a capability to exclude feature properties (including the geometry) from the response in a future part and not require that every implementation supports this.

[jvanulde]

Of course, intermediaries can cache as needed. So can the client. @ThomasG77 in the case of just returning a table, for instance, aren't we just talking about a different representation?

[jdesboeufs]

For one of our APIs, we do something like that:

Default: JSON
https://geo.api.gouv.fr/communes/36244

Optional: GeoJSON:
https://geo.api.gouv.fr/communes/36244?format=geojson

Optional: GeoJSON + geometry choice
https://geo.api.gouv.fr/communes/36244?format=geojson&geometry=contour

You can also choose fields you want to be returned:
https://geo.api.gouv.fr/communes/36244?format=geojson&fields=nom,code,population

Work also for collections…

Fast and efficient ;)

[jvanulde]

👍 @jdesboeufs but it would be awesome if those "Optional" resource representations could be included in the "Default" response.

[ThomasG77]

@jvanulde About "aren't we just talking about a different representation?"
We could change output to a "flatter" output (illustrated as you suggested below using GeoJSON/JSON serialization) but if for legacy reason, people need GML output with no geometry, simply changing serialization would not be the best solution IMO.

@jvanulde, your proposition remains better for response size.

GeoJSON

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "id": 1,
      "properties": {
        "prop1": 1414,
        "prop2": "qwerty"
      },
      "geometry": null
    },
    {
      "type": "Feature",
      "id": 2,
      "properties": {
        "prop1": 129,
        "prop2": "wxert"
      },
      "geometry": null
    },
    {
      "type": "Feature",
      "id": 3,
      "properties": {
        "prop1": 237,
        "prop2": "abcde"
      },
      "geometry": null
    },
    {
      "type": "Feature",
      "id": 4,
      "properties": {
        "prop1": 143,
        "prop2": "wxyz"
      },
      "geometry": null
    },
    {
      "type": "Feature",
      "id": 5,
      "properties": {
        "prop1": 2832,
        "prop2": "qwerty"
      },
      "geometry": null
    }
  ]
}

Flat JSON table

[{
  "id": 1,
  "prop1": 1414,
  "prop2": "qwerty"
}, {
  "id": 2,
  "prop1": 129,
  "prop2": "wxert"
}, {
  "id": 3,
  "prop1": 237,
  "prop2": "abcde"
}, {
  "id": 4,
  "prop1": 143,
  "prop2": "wxyz"
}, {
  "id": 5,
  "prop1": 2832,
  "prop2": "qwerty"
}]

[ThomasG77]

@cportele I suppose you are speaking about something similar as the previous WFS versions implementation where propertyname=yourwantedfield1,yourwantedfield2 returns only the 2 field attributes and adding propertyname=yourwantedfield1,yourwantedfield2,geometry returns the 2 fields with the geometry

The main difference seems that you are thinking about using exclusion in the logic instead of inclusion in previous WFS versions.

Sorry for some noise, as it's always difficult to figure if a suggestion should be in the core or the future additional parts and mandatory or not.

[cportele]

@ThomasG77 I was thinking about the same style as in the "projection clause" in the current WFS, i.e. listing explicitly the feature properties to include in the response.

Don't worry about "noise", we are looking for developer feedback and welcome the input and thoughts!

The general idea is to keep the mandatory Core small so that it is not hard to implement a useful server. A capability to subset the returned properties (the "projection clause" in WFS 2.0) is clearly useful for clients in some use cases (see also my issue #13), but so far we felt that it is better to make it an optional capability.

[cholmes]

Another example of requesting properties - http://jsonapi.org/format/#fetching-sparse-fieldsets

[cportele]

Agreement on 2018-02-01: Defer to an extension. (But keep the discussion going.)

[jerstlouis]

@cportele I saw your comment about skipGeometry in ldproxy on Gitter. Could omitGeometry be specified instead?

I think it makes sense to include this in the extension specifying properties as well. In our server we would omit geometry when properties= is used and geometry was not explicitly specified. I like this idea of a separate parameter, but I think omit would be a better term, if the naming is still up for debate.

[cportele]

@jerstlouis - I fully agree, there should be a single extension for what WFS called "projection". With respect to naming, in our implementation we just came up with skipGeometry, but I see your point why omitGeometry would be clearer. The naming is totally up for discussion.

[jerstlouis]

@cportele Cool. speaking of naming, I think I've mentioned before that I really dislike the choice of the projection word for this functionality :)

In addition to an obvious confusion in the geospatial domain about what this extension is about, I still fail to see which definition of the term projection this refers to?

https://www.merriam-webster.com/dictionary/projection

The data is not transformed or projected onto a different plane or anything, you are simply asking for a subset of the full available information.

To me this is a subsetting of the properties... Very similar in fact to the rangeSubset capability in Coverages / WCS, and you could easily see how this is the equivalent functionality for a collection that is available as both Features and as a Coverage.

[cportele]

I don't think we need to spend our time debating the term "projection", there likely is consensus not to use that term in the OGC API context :)

[jerstlouis]

@cportele Ah okay.

I did find this on Wikipedia ( https://en.wikipedia.org/wiki/Projection_(mathematics), as an extension of the general mathematical concept to relational databases.

For relational databases and query languages, the projection is a unary operation written as ... is a set of attribute names. The result of such projection is defined as the set that is obtained when all tuples in R are restricted to the set ... is a database-relation.

I thought projection was the proposed name for the Features extension. Thanks for the clarification.

[cportele]

Yes, this is where the term came from.

[pvretano]

@jerstlouis @cportele I used the term projection because my early background was in relational databases and the part of the query that indicates which properties to include in a response is called the projection clause just like the part that selects which records are in a response is called the selection clause. The name I was proposing to use for the features extension was OGC API - Features - Part X: Property Selection. That is from TB14: https://docs.opengeospatial.org/per/18-045.html#property_selection_extension. If we have a projection capability why exactly do we need a skip/omitGeometry thing?

[jerstlouis]

@pvretano I like Property selection. Does this really have to be a whole new extension? Couldn't it be an extra conformance class of a future revision of Core?

We need something like omitGeometry for example to be able to easily get all of the properties except the geometry, without having to explicitly name the 100 properties there are. Unless there is a negative form, like omitProperties=geometry.

[cportele]

@pvretano:

If we have a projection capability why exactly do we need a skip/omitGeometry thing?

In addition to what @jerstlouis said, what is the property name of the geometry? In a GML encoding (where it would just one of the properties) it would likely be something else that in a GeoJSON (where geometry has special treatment and is not one of the "properties").

I think we can find a convention one way or the other, but geometry will also always be something special, also because the geometry is often the most significant part of the feature size, and it may be easier for API users to have special treatment for the geometry.

@jerstlouis:

Does this really have to be a whole new extension? Couldn't it be an extra conformance class of a future revision of Core?

Yes. No.

Core is and should always be restricted to the capabilities that every implementation has to support. That is why it is called "Core". Everything else is an extension. And keeping extensions small with a focussed scope is a good thing, too, although the OGC processes do not really support that well.

[jerstlouis]

Core is and should always be restricted to the capabilities that every implementation has to support. That is why it is called "Core"

Is that really the case though?

Core defines GeoJSON and HTML conformance classes, but an implementation doesn't necessarily have to implement both or even either of these, or is that not the case? Common Core defines the OAS3 conformance class, but it was agreed that an API description could be provided in an alternate API description language instead and still conform to Common Core.

And keeping extensions small with a focussed scope is a good thing, too, although the OGC processes do not really support that well.

That's why I thought conformance classes might be the more granular units that can be conveniently grouped in an extension (making the process easier for the SWGs), where supporting an extension does not necessarily imply supporting all conformance classes defined in it. I think that does apply to Simple Transactions where PATCH for example is optional.

I think we will need to review the implications of this for Coverages as well...

[pvretano]

@cportele ok about the geometry although I would have assumed that the projection clause for JSON-encoded feature would reference the keys so if the projection clause was specified and included the "geometry" key then the geometry would be presented; otherwise it would not.

@jerstlouis @cportele I supposed we could "gather" a bunch of small extensions into a single document although I agree with @cportele I don't really see a problem with having lots of small and concise extensions; after all these are supposed to be LEGOs! ;)

[jerstlouis]

@pvretano The document boilerplate and process overhead associated with extensions is far from small and concise though.

I would see the conformance classes as LEGOs, and the extensions as LEGO boxsets containing the blocks to build something cool. You don't have to use all the blocks in the set, you just pick the ones you want. to use. :)

[pvretano]

@jerstlouis the entire effort for writing an extension is mostly 1 clause; the rest -- as you say -- is boilerplate. It would be good if the boiler plate were a lot smaller but it is not that onerous. I have always wanted OGC to put most of the boilerplate into some persistent location and then extensions could simple include a link. Also, @cportele wrote a leaner template for 1-conformance class extensions that makes the process easier; its located somewhere in the features repo.

As for the LEGO analogy, sure! ;)

[cportele]

Conformance classes should not be bundled in a single document, unless they are closely related and implemented together in many/most cases. It is easier for those not knowing the standards by heart to read and implement everything from a short document than to find and read a chapter in a large document, understand the context and relationship with the other chapters and then implement just that chapter.

[jerstlouis]

Waking this thread up ahead of the November 2021 sprint as we are implementing support for this.

I am guessing the proposal should now be kebab-cased omit-geometry, and perhaps omit-properties to request all except some specific properties (some of which could be geometries)?

[jvanulde]

@jerstlouis perhaps a properties parameter could be used to specify which properties one wants returned? If the parameter is not provided all properties are returned. Just an idea. This is how it's implemented in pygeoapi.

[jerstlouis]

@jvanulde Yes that is how it's already done, but the use case for omit-properties is when you have either a very long list of properties, or you don't even have a fixed schema listing them all, and you want to specifically omit some properties (omitting fewer than you want to include, either because they take up too much space, or differentiate features in an unwanted ways prior to merging by attributes, or are otherwise problematic).

[pvretano]

The omit-properties/omit-geometry/skipGeometry approach seems very GeoJSON specific to me. Other encodings don't organize their features that way. Does, for example, omit-geometry make sense for GML where a feature can have any number of geometry-valued properties? Would omit-geometry in that case mean omit all of them? What if I wanted some geometries and not others? Say for example I want to omit the "big" ones but the smaller point geometries I would like to keep? Also, what does omit-properties mean in the context of GML? The geometry-valued properties are also properties so do I omit those too?

I view the properties parameter as a negotiation between the client and the server. The clients lists the properties that they want to get and the server does its best to return those properties for each feature in the response. There is also likely a need for a returnables schema but that is already in the works in the schema proposal.

[jerstlouis]

@pvretano Yes I think omit-geometry would omit all of them. The idea with omit-properties is that you could omit specific geometry per their names.

(To clarify, we already implement properties and I am not suggesting omit-properties as a replacement, but as an additional way to restrict the set of properties to be returned).

[cportele]

Meeting 2024-06-03: The general capability is supported by the current draft of Part 6. @cportele will review the disucssion to see, if there are additional aspects not yet covered/discussed and otherwise close the issue as completed.