-
-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
geoJSON feature IDs: Saved as a property instead of ID #40805
Comments
@arcataroger how? using the "export as..." dialog? using a tool (i.e "concert format") in Processing toolbox? have you tried the conversoin with ogr2ogr? if yes, how about the result? if no, can you test it? |
@gioman Thanks for checking in, and happy new year :) Yes, using the Export As dialog. It happens with ogr2ogr as well, both by default and using the I am not sure if QGIS uses GDAL/ogr2ogr behind the scenes, but if so, perhaps the I've updated the original issue summary with more details. |
@arcataroger the "help" button in the "save vector layer as..." dialog point to were you can find this paragraph: So... if you add the necessary GDAL layer creation option: you get what you expect. |
Great, thanks for that tip! Super helpful as a workaround. Probably should still keep this open as a bug, though? The geoJSON spec is pretty clear that the ID should be a member, not part of |
@arcataroger I don't agree is a workaround nor a bug, at best it is a feature request asking to improve the GUI of the dialog in case the geojson format is selected (in a way that "forces" the user to select the column for the "id_field" LCO. |
@gioman Are there situations where a user might want to export geoJSON that's not to spec? If not, why wouldn't the exporter want to conform to the spec by default? IMO, the bug isn't that the
Edit to add: The ID of a feature isn't just a generic, user-defined property, but a core part of geoJSON used by other software and expected in a certain format/place. By default, QGIS right now creates broken exports that other software cannot use without this modification/workaround. |
@arcataroger if I'm not wrong even ogr2ogr do not use this LCO by default, correct? if yes QGIS behaves coherently I guess. Also: how QGIS would know what is the column to choose for "id_field", there is no garauntee that a "id", "fid", etc. column exist, and in general a user could want to use a column with a generic name, correct? Anyway I have no problem leave this open as bug, I personally just don't think it is given than there is a clear way (not a workaround) to get the expected result. |
Sorry, I've never used ogr2ogr before this bug (downloaded it because you When I create a new shapefile, the But if that is a default, it seems like using that But also happy to bring this up to the ogr2ogr folks (is that part of the |
Hmm, it seems like the In that case, what you said makes sense... not all layers can be assumed to have an I think the dropdown would still be an easy fix, but whether that's more properly a bug or a feature request, I'll leave to you! |
@arcataroger yes, but I'm pretty sure you'll get a similar answer... it is up to the user to apply this LCO because there is no way to automatically the proper column. See below
even for shapefiles... QGIS GUI adds a "id" column just for user convenience, but is not mandatory (or it could have a completely different name). If in your workflow you always have shapefiles with an "id" column with a unique identifier, why you don't just create a mini model with the "convert format" from the QGIS processing toolbox where you hard code the LCO you need? |
That works only for the first conversion from the shapefile to geoJSON, not for subsequently working with the geoJSON as a layer directly. It would be nice to be able to add/edit geometries in geoJSON layers without losing the IDs with subsequent saves. For example, if you have a geoJSON like this: {
"type": "FeatureCollection",
"name": "my_geojson",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:EPSG::3857" } },
"features": [
{ "type": "Feature", "id": "only_top_level", "properties": {}, "geometry": { "type": "MultiPolygon", "coordinates": [ [ [ [ -9753600.426300490275025, 5141026.099048308096826 ], [ -9753526.289699155837297, 5141027.468623659573495 ], [ -9753525.897439897060394, 5141006.235203934833407 ], [ -9753562.965740563347936, 5141005.550416259095073 ], [ -9753600.034041231498122, 5141004.865628583356738 ], [ -9753600.426300490275025, 5141026.099048308096826 ] ] ] ] } },
{ "type": "Feature", "properties": { "id": "only_property" }, "geometry": { "type": "MultiPolygon", "coordinates": [ [ [ [ -9753599.832634938880801, 5140988.417339080944657 ], [ -9753525.862954199314117, 5140989.783830795437098 ], [ -9753525.52837342210114, 5140971.672609879635274 ], [ -9753570.321762247011065, 5140970.845030345022678 ], [ -9753599.498054161667824, 5140970.306118167936802 ], [ -9753599.832634938880801, 5140988.417339080944657 ] ] ] ] } },
{ "type": "Feature", "id": "both_1", "properties": {"id": "both_2" }, "geometry": { "type": "MultiPolygon", "coordinates": [ [ [ [ -9753622.106315340846777, 5140958.046617023646832 ], [ -9753612.610966777428985, 5140956.94570704549551 ], [ -9753615.167930025607347, 5140934.891899031586945 ], [ -9753624.66327858902514, 5140935.992809009738266 ], [ -9753622.106315340846777, 5140958.046617023646832 ] ] ] ] } }
]
}
Then, if you edit that ID field in QGIS, it gets saved back to the { "type": "Feature", "id": "only_top_level", "properties": { "id": "overwritten" }, "geometry": { "type": "MultiPolygon", "coordinates": [ [ [ [ -9753600.426300490275025, 5141026.099048308096826 ], [ -9753526.289699155837297, 5141027.468623659573495 ], [ -9753525.897439897060394, 5141006.235203934833407 ], [ -9753562.965740563347936, 5141005.550416259095073 ], [ -9753600.034041231498122, 5141004.865628583356738 ], [ -9753600.426300490275025, 5141026.099048308096826 ] ] ] ] } },
{ "type": "Feature", "properties": { "id": "overwritten" }, "geometry": { "type": "MultiPolygon", "coordinates": [ [ [ [ -9753599.832634938880801, 5140988.417339080944657 ], [ -9753525.862954199314117, 5140989.783830795437098 ], [ -9753525.52837342210114, 5140971.672609879635274 ], [ -9753570.321762247011065, 5140970.845030345022678 ], [ -9753599.498054161667824, 5140970.306118167936802 ], [ -9753599.832634938880801, 5140988.417339080944657 ] ] ] ] } },
{ "type": "Feature", "id": "both_1", "properties": { "id": "overwritten" }, "geometry": { "type": "MultiPolygon", "coordinates": [ [ [ [ -9753622.106315340846777, 5140958.046617023646832 ], [ -9753612.610966777428985, 5140956.94570704549551 ], [ -9753615.167930025607347, 5140934.891899031586945 ], [ -9753624.66327858902514, 5140935.992809009738266 ], [ -9753622.106315340846777, 5140958.046617023646832 ] ] ] ] } } (Now the first feature has a new ID while the old one still exists, while the third feature had its both_1 ID unintentionally kept there). That's if I try to save the edits in place (using the "Save Layer Edits" button). If I instead try to use the format convert in the toolbox, it won't let me convert the same file in-place, overwriting the one currently open (the dialog just crashes when I try to replace the file). If I manually specify the same filename instead of using the browser, the log output asks me to use If I try to use the Export dialog instead, it won't let me overwrite an existing file: So, still no way to edit and re-save a geoJSON in place, with IDs intact as per the spec :( (This really isn't the end of the world, since it's possible to take care of it as a batch find/replace after edits, but it is a hassle. And perhaps more importantly, this behavior is pretty unexpected when you're working with a format as common as geoJSON. While the workaround of using the I guess the difference is that to geoJSONs, the ID field has special importance. To QGIS and ogr2ogr, it's just another user-defined field with no special significance.) |
@arcataroger this seems anyway a different issue from the one initially reported in this ticket. I suggest filing a new one (if you find multiple issue is 1 issue per ticket). Please note that the messages you are getting are from OGR, you should check first if you get the same problems using ogr2ogr to translate/edit this datasources, if yes the is likely to be an OGR issue, not qgis's. |
@arcataroger as you can see GQIS seems coherent with OGR:
|
Right, but is "coherence with OGR" really the underlying objective here, as opposed to "being able to edit geoJSON in-place without destroying IDs"? The command-line tools have a different use case than QGIS, don't they? For the command-line tools, it totally makes sense to specify flags and parameters. They were built for transformations (is there even an "open, modify, and save in place" workflow for the command-line tools?). But when I'm just trying to save the geometries I'm editing in QGIS, shouldn't I be able to do so without jumping through hoops with every save? It is confusing to users when IDs get ambiguously read and saved. Do you agree that being able to work with geoJSONs in-place, retaining IDs, would be a useful addition to QGIS (maybe more properly as a feature request, as you suggested), or do you think that's just entirely out of scope? If you agree, I believe this would make more sense as a GUI tweak in GQIS, or a modification to the geoJSON driver, regardless of how the command-line tools work. I only brought up those errors to illustrate that the command line tools aren't really made for this use case. To be clear, what I think would be most helpful is an additional geoJSON-specific parameter, "Use field X as ID", built into the exporter/driver (and retained with subsequent saves), alongside the other current geoJSON-specific options. I understand this would be inconsistent with the command-line tools, but maybe that's OK given the different use case? This is a quirk of the geoJSON format, granted, but IMHO it's not an unreasonable one for QGIS to understand and deal with in a more user-friendly way. It's a fantastic tool for editing geoJSONs otherwise (and there aren't many out there!) I would be happy to try and make a PR for this myself if I could... if you have any suggestions as to where to start, I'll look into it and try to learn. Or if you think this is entirely out of scope for QGIS, that being able to read/edit/save geoJSONs as layers directly isn't an intended use case, then I understand and thank you for all your time and patience here. I especially appreciate you taking the time to walk me through the thought processes -- I had no idea how QGIS worked behind the scenes, so this discussion illuminated a lot for me. Thank you either way :) |
@arcataroger this seems reasonable to me, it is anyway a feature request, not a bug. Feel free to open one. Also you may know (or not)... this is open source, so if for your organization workflow this is really important you have the power to make it happen sooner than later. |
@gioman I will open a new feature request and link it back to this one, closing this one in the meantime. Thank you! I'm a relatively new dev in general, and haven't touch C++ in a decade, but I would love to help contribute if I can. I'll look into what it would take to add that option. Thanks again for all the tips. |
@arcataroger on qgis.org there is a page about commercial support, there are plenty of individuals and companies that can get you where you need. |
@gioman Sadly, the nonprofit I work for has no money to sponsor something like this. However, I do appreciate all the work the community has put into QGIS, so I donated some personal funds just now as a small token of gratitude. Thanks again for taking the time to walk me through this. |
Describe the bug
When exporting vector layers to geoJSON, feature IDs get saved in the
properties
of the feature instead of a top-levelid
, as described in the geoJSON spec.Expected output:
Actual output:
How to Reproduce
id
integer field intactExport
-->Save features as...
--> Format:GeoJSON
Using ogr2ogr
Same thing happens with
ogr2ogr output.geojson input.shp
:With the
-preserve_fid
flag enabled, QGIS's id remains insideproperties
, but ogr2ogr adds its own top-levelid
incrementing from 0:{ "type": "Feature", "id": 0, "properties": { "id": 22 }, "geometry": { "type": "Point", "coordinates": [ -87.617569033727861, 41.866959795008569 ] } }
The output is correct using the
-lco id_field=id
flag, likeogr2ogr output.geojson input.shp -lco id_field=id
QGIS and OS versions
3.16.0 Hanover
macOS 11.1
Additional context
The same issue is described in other places:
The text was updated successfully, but these errors were encountered: