Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Delete field in table delete all features #28585

Closed
qgib opened this issue Dec 10, 2018 · 19 comments
Closed

Delete field in table delete all features #28585

qgib opened this issue Dec 10, 2018 · 19 comments
Labels
Bug Either a bug report, or a bug fix. Let's hope for the latter! Crash/Data Corruption Data Provider Related to specific vector, raster or mesh data providers High Priority Regression Something which used to work, but doesn't anymore

Comments

@qgib
Copy link
Contributor

qgib commented Dec 10, 2018 •
edited
Loading

Author Name: Lene Fischer (@LeneFischer)
Original Redmine Issue: 20765
Affected QGIS version: 3.5(master)
Redmine category:data_provider/ogr

To in a lecture today we should delete a field in a table. Geopackage.

When trying to save all data vanished. We were not able to undo.
When closing and opening again the open dialog showed 160 features - but none in the attribute tabel.

Error message was something about OGR..

This happend for all the students with version 3.4.2 Windows
Students with 3.4.1 had no problem - and the same for students with 3.4.2 Mac - (great fun because it´s alwas the Mac users who are having difficulties..)

Not sure if it´s the same issue as #28327
@qgib
Copy link
Contributor Author

qgib commented Dec 11, 2018

Author Name: Nyall Dawson (@nyalldawson)

Error message was something about OGR..

Have you got the actual error?

Can you share the original file and steps to reproduce?

  • status_id was changed from Open to Feedback

@qgib
Copy link
Contributor Author

qgib commented Dec 11, 2018

Author Name: Lene Fischer (@LeneFischer)

I have tried to replicate the error message but it does not appear any longer.

  • 13929 was configured as skovkort.gpkg

@qgib
Copy link
Contributor Author

qgib commented Dec 11, 2018

Author Name: Lene Fischer (@LeneFischer)

When I close the file and re-open I get a message : WARNING CRS was undefined : defaulting to CRS EPSG:25832 - ETRS89 / UTM zone 32N

@qgib
Copy link
Contributor Author

qgib commented Dec 11, 2018

Author Name: Jérôme Guélat (Jérôme Guélat)

I'm experiencing the same bug... Suprisingly this doesn't happen if you save the layer to a new GeoPackage and then delete one column in the new GeoPackage.

The problem is also occurring with QGIS 2.18.26 (and QGIS is even crashing).

I've also checked on my Mac and it works perfectly, using the same GDAL version as the Windows version (2.3.2).

@qgib
Copy link
Contributor Author

qgib commented Dec 11, 2018

Author Name: Jérôme Guélat (Jérôme Guélat)

Exporting the whole GeoPackage with ogr2ogr using the following command also helps: ogr2ogr newgeopackage.gpkg skovkort.gpkg

I've first thought the problem was related to the GeoPackage version since I've only had problems with GeoPackages using version 1.0 (and not with version 1.2 which is now the default when using GDAL 2.3.2) but this doesn't seem to be the cause since the problem is also solved if I force ogr2ogr to export in version 1.0 (ogr2ogr newgeopackage.gpkg skovkort.gpkg -dsco VERSION=1.0).

@qgib
Copy link
Contributor Author

qgib commented Dec 11, 2018

Author Name: Jérôme Guélat (Jérôme Guélat)

It is fortunately possible to recover the data... Open the GeoPackage with an SQLite editor (e.g., https://sqlitebrowser.org) and look for the table with the removed column.

If your table/layer was called foo, then look for a table named foo_ogr_tmp. Rename this table using the original name (foo). QGIS will again be able to open the GeoPackage layer.

@qgib
Copy link
Contributor Author

qgib commented Dec 11, 2018

Author Name: Giovanni Manghi (@gioman)

Lene Fischer wrote:

I have tried to replicate the error message but it does not appear any longer.

so you can't replicate the problem in a consistent manner?

@qgib
Copy link
Contributor Author

qgib commented Dec 11, 2018

Author Name: Lene Fischer (@LeneFischer)

I can easily replicate the problem - but not the error message. I think that it initially was a multiple session of deleting and creating fields that caused the OGR error message.

@qgib
Copy link
Contributor Author

qgib commented Dec 11, 2018

Author Name: Giovanni Manghi (@gioman)

Confirmed on master on clean Windows 10 and QGIS installations. This is VERY bad.

  • version was changed from 3.4.2 to 3.5(master)
  • status_id was changed from Feedback to Open

@qgib
Copy link
Contributor Author

qgib commented Dec 11, 2018 •
edited
Loading

Author Name: Jürgen Fischer (@jef-n)

  • description was changed from To in a lecture today we should delete a field in a table. Geopackage.

When trying to save all data vanished. We were not able to undo.
When closing and opening again the open dialog showed 160 features - but none in the attribute tabel.

Error message was something about OGR..

This happend for all the students with version 3.4.2 Windows
Students with 3.4.1 had no problem - and the same for students with 3.4.2 Mac - (great fun because it´s alwas the Mac users who are having difficulties..)

Not sure if it´s the same issue as https://issues.qgis.org/issues/20507 to To in a lecture today we should delete a field in a table. Geopackage.

When trying to save all data vanished. We were not able to undo.
When closing and opening again the open dialog showed 160 features - but none in the attribute tabel.

Error message was something about OGR..

This happend for all the students with version 3.4.2 Windows
Students with 3.4.1 had no problem - and the same for students with 3.4.2 Mac - (great fun because it´s alwas the Mac users who are having difficulties..)

Not sure if it´s the same issue as #28327

@qgib
Copy link
Contributor Author

qgib commented Dec 12, 2018

Author Name: Alessandro Pasotti (@elpaso)

This is probably an upstream bug in OGR/GDAL or the underlying sqlite libraries.

I checked with a debugger on windows and QGIS is not doing anything strange: it calls the right OGR function with the right parameters, the call returns no errors but the table is completely removed from the DB (that's why QGIS asks for the CRS when re-opened).

@qgib
Copy link
Contributor Author

qgib commented Dec 12, 2018

Author Name: Jérôme Guélat (Jérôme Guélat)

You're right... I think I remember that SQLite was updated with QGIS 3.4.2.

Using OSGeo4W I downgraded SQLite from 3.25 to 3.17 and the bug is gone.

@qgib
Copy link
Contributor Author

qgib commented Dec 12, 2018

Author Name: Alessandro Pasotti (@elpaso)

@jérôme, I can confirm: downgrading sqlite to 3.17 solved the issue, I'm not sure what you should do now, perhaps file a bug report for sqlite?

  • resolution was changed from to up/downstream

@qgib
Copy link
Contributor Author

qgib commented Dec 12, 2018

Author Name: Jürgen Fischer (@jef-n)

Alessandro Pasotti wrote:

@jérôme, I can confirm: downgrading sqlite to 3.17 solved the issue, I'm not sure what you should do now, perhaps file a bug report for sqlite?

Or try 3.26.0-1

@qgib
Copy link
Contributor Author

qgib commented Dec 13, 2018

Author Name: Jérôme Guélat (Jérôme Guélat)

Unfortunately the bug is also occurring with this version.

@qgib
Copy link
Contributor Author

qgib commented Jan 21, 2019

Author Name: Even Rouault (@rouault)

OK, I believe this is the issue that was fixed per OSGeo/gdal@556738b in GDAL 2.4.0
The GeoPackage specification had an error in the formulation of a trigger, but before SQLite 3.25, SQLite was not sensitive to this error.
So GeoPackage generated with GDAL 2.4.0 should now work fine, but files generated by older versions will cause the issue.

@qgib
Copy link
Contributor Author

qgib commented Jan 22, 2019

Author Name: Jérôme Guélat (Jérôme Guélat)

Thanks for having a look at it.

Indeed this seeems to be the problem... If I correct the trigger in the "older" GeoPackages, then the issue doesn't occur. Would it be possible to correct this trigger automatically when the incorrect one is detected in a GeoPackage used in QGIS?

@qgib
Copy link
Contributor Author

qgib commented Jan 22, 2019

Author Name: Giovanni Manghi (@gioman)

Jérôme Guélat wrote:

Thanks for having a look at it.

Indeed this seeems to be the problem... If I correct the trigger in the "older" GeoPackages, then the issue doesn't occur. Would it be possible to correct this trigger automatically when the incorrect one is detected in a GeoPackage used in QGIS?

I think this should be filed as feature request.

  • status_id was changed from Open to Closed

@qgib qgib closed this as completed Jan 22, 2019
@qgib qgib added Bug Either a bug report, or a bug fix. Let's hope for the latter! High Priority Data Provider Related to specific vector, raster or mesh data providers Crash/Data Corruption Regression Something which used to work, but doesn't anymore labels May 25, 2019
@agiudiceandrea agiudiceandrea mentioned this issue Apr 12, 2021
Closed
@agiudiceandrea
Copy link
Contributor

Fixed by #43309, #43322, OSGeo/gdal#3863.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Either a bug report, or a bug fix. Let's hope for the latter! Crash/Data Corruption Data Provider Related to specific vector, raster or mesh data providers High Priority Regression Something which used to work, but doesn't anymore
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@agiudiceandrea @qgib