Should FlatGeobuf support features with null geometries when spatially indexed?

[bjornharrtell]

FlatGeobuf supports null geometries in the spec in some capacity but it has not been well explored.

Recently it was discovered that it doesn't currently seem possible to support the specific case when you want spatial indexed FlatGeobuf and the same time want it to contain features without geometry. See OSGeo/gdal#7401 for more details.

In short, it is believed that since the header has a feature count that is also used to establish the index size and structure it must represent indexable features. Since features without geometry cannot be indexed there is a problem.

The FlatGeobuf spec can be extended with backwards compatibility if done carefully and I believe there could be a way to add the needed metainformation to support this case but I'm hesistant to do it / invest time into it since I don't personally like the mixed concern of non spatial and spatial features in the same file or feel I have use for it myself.

[alexbodn]

definitely, it's nice to keep all related data in one place,
even not strictly geometric data.
without monstruous data duplication, a point, or a tl, dr
multipoint could be autoindexed at the same location.
the user should be responsible of selecting the right records
for the right task.

[palmerj]

I also think FlatGeobuf files should support rows without a geometry, even if there is a spatial index. We have a few layers like this and still would like the option to publish the data as a single logical dataset, even if the spatial index request doesn't return that data. Note this also matches what is possible with most other spatial file or database stores.

[bjornharrtell]

I'm still hesistant, but perhaps more importantly still not interested so would probably need someone else to champion it.

As for other formats/databases supporting it I'm not sure is the best argument. For example, I find it rather annoying when null geometry has been left allowed in PostgreSQL/PostGIS more often than not by mistake and some crap data has sneaked its way in.

Disclaimer. I really dislike null in general. :D

[palmerj]

Up to you. But it's a well known design pattern that is accepted in the relational data world. Also the OGC simple features model allows it too, and OGR also implements that. So you will be going all of this and interop will decrease for your format.

[bjornharrtell]

That's a good argument that I agree with.

[alexbodn]

After reading the other comments, I feel I might have misunderstood your original post.
My vision was of one file containing geometries and let's say some comments, also localized.
Now I understand you meant one file containing spatially indexed geometries and non spatially indexed data.
It may lead to some fictional geometry designed not to interfere with spatial queries.
The problem is, this fictional data may be differ by application, such that the null value may still get a better, unified exception from index operations.

[bjornharrtell]

Revisited thought about this.. and perhaps all that is needed is an aux feature count for the features with null geometries.

[mtravis]

This would be a welcome addition. I've recently had issues when trying to convert parquet to FGB using DuckDB and the source data had tiny polygons that DuckDB reads as having no geometry.

[bjornharrtell]

Ew! Data destruction! 😱😂

[olivierdalang]

Just a little motivator for this one as it's the only drawback I know of flatgeobufs in read-only scenarios. If this was fixed we'd finally know what to tell to GIS users asking what format they should use :-D

[bjornharrtell]

It's too bad christmas is already over though. :)