Edits in GeoJson datasources are not saved anymore #28580

qgib · 2018-12-09T22:08:01Z

Author Name: Ehsan Aliverdi (Ehsan Aliverdi)
Original Redmine Issue: 20760
Affected QGIS version: 3.4.4
Redmine category:data_provider

I tested this issue in Qgis 3.4.0, 3.4.1, 3.4.2 in all the same issue
in our company we work with Geojson files in Qgis. When editing the attributes of an existing feature in Geojson layer, QGIS creates a temp layer and applies the changes on that new temp layer and never applies the changes on original layer.
this problem wasn't there in version 3.2.3

equipment.geojson (Ehsan Aliverdi)
2018-12-18_9-15-07.png (Ehsan Aliverdi)
edit-geojson.gif (Ben Hur Pintor)
geoJsonProblem.gif (Ehsan Aliverdi)

qgib · 2018-12-10T09:05:00Z

Author Name: Giovanni Manghi (@gioman)

subject was changed from QGIS does not update GeoJson Files after creation to Edits in GeoJson datasources are not saved anymore
category_id was changed from Attribute table to Data Provider
version was changed from 3.4.0 to 3.4.2
operating_system was changed from Windows 10 to
crashes_corrupts_data was changed from 1 to 0

qgib · 2018-12-17T11:05:35Z

Author Name: Ben Hur Pintor (@benhur07b)

Hi Ehsan,

Can you provide a sample dataset to test this? I just tested editing a GeoJSON layer on 3.4.2 (Ubuntu 18.04) and everything works as expected. I can edit the fields and even add/delete fields. No temp layer is created and the original layer is edited.

Thanks!

Sincerely,
Ben Hur

qgib · 2018-12-17T22:40:06Z

Author Name: Ehsan Aliverdi (Ehsan Aliverdi)

Ben Hur Pintor wrote:

Hi Ehsan,

Can you provide a sample dataset to test this? I just tested editing a GeoJSON layer on 3.4.2 (Ubuntu 18.04) and everything works as expected. I can edit the fields and even add/delete fields. No temp layer is created and the original layer is edited.

Thanks!

Sincerely,
Ben Hur

Hi Ben
Thanks for reply
Please find a sample dataset enclosed. I have tested this in more than 10 different pc with windows. I just did another test with the following qgis(see the image enclosed)
step to reproduce the issue:
-Put equipment layer on editable mode
-use identify tool to id a feature(anyone will work)
-use edit feature form to edit an attribute of a feature
-save the changes
-toggle edit
-use identify tool to id the feature.
You will see nothing is changed.

Kindly Regards
Ehsan

13985 was configured as 2018-12-18_9-15-07.png
13984 was configured as equipment.geojson

2018-12-18_9-15-07.png (Ehsan Aliverdi) - QGIS about
equipment.geojson (Ehsan Aliverdi)

qgib · 2018-12-18T12:36:08Z

Author Name: Ben Hur Pintor (@benhur07b)

Ehsan Aliverdi wrote:

Hi Ben
Thanks for reply
Please find a sample dataset enclosed. I have tested this in more than 10 different pc with windows. I just did another test with the following qgis(see the image enclosed)
step to reproduce the issue:
-Put equipment layer on editable mode
-use identify tool to id a feature(anyone will work)
-use edit feature form to edit an attribute of a feature
-save the changes
-toggle edit
-use identify tool to id the feature.
You will see nothing is changed.

Kindly Regards
Ehsan

Hi Ehsan,

For my setup (QGIS 3.4.2 on Ubuntu 18.04), I could edit the attributes of the GeoJSON. Kindly check the GIF attached if I followed your instructions to reproduce the bug. Maybe this is a Windows-specific problem. If possible, could you also send a video or GIF of the bug in action.

Thank you.

Sincerely,
Ben Hur

13989 was configured as edit-geojson.gif

edit-geojson.gif (Ben Hur Pintor)

qgib · 2018-12-20T05:01:11Z

Author Name: Ehsan Aliverdi (Ehsan Aliverdi)

Ben Hur Pintor wrote:

Ehsan Aliverdi wrote:

Hi Ben
Thanks for reply
Please find a sample dataset enclosed. I have tested this in more than 10 different pc with windows. I just did another test with the following qgis(see the image enclosed)
step to reproduce the issue:
-Put equipment layer on editable mode
-use identify tool to id a feature(anyone will work)
-use edit feature form to edit an attribute of a feature
-save the changes
-toggle edit
-use identify tool to id the feature.
You will see nothing is changed.

Kindly Regards
Ehsan

Hi Ehsan,

For my setup (QGIS 3.4.2 on Ubuntu 18.04), I could edit the attributes of the GeoJSON. Kindly check the GIF attached if I followed your instructions to reproduce the bug. Maybe this is a Windows-specific problem. If possible, could you also send a video or GIF of the bug in action.

Thank you.

Sincerely,
Ben Hur

Thanks alot for your reply. I am using windows. for your information I attached a gif file which shows the problem. we use qgis on 20 PCs in our company and all have the same issue because of this issue we downgraded every qgis to 2.3.2 which works perfect.

Regards
Ehsan

14001 was configured as geoJsonProblem.gif

geoJsonProblem.gif (Ehsan Aliverdi)

qgib · 2019-02-15T13:10:32Z

Author Name: Paul Blottiere (Paul Blottiere)

FWIW, I didn't succeed in reproducing the issue either.

qgib · 2019-02-16T09:37:54Z

Author Name: Giovanni Manghi (@gioman)

Paul Blottiere wrote:

FWIW, I didn't succeed in reproducing the issue either.

I can easily replicate on 3.4.4 (following the steps in the attached cast) on Windows (clean install of both QGIS and the OS).

version was changed from 3.4.2 to 3.4.4

qgib · 2019-02-18T08:13:54Z

Author Name: Paul Blottiere (Paul Blottiere)

Hi @giovanni,

I can easily replicate on 3.4.4 (following the steps in the attached cast) on Windows (clean install of both QGIS and the OS).

OK, so according to @ben Hur comment and my own tests on Debian, it seems that it's pretty specific to Windows.

operating_system was changed from to Windows

qgib · 2019-02-21T12:01:54Z

Author Name: Peter Petrik (@PeterPetrik)

unable to replicate on MacOS too. So definitely windows-only issue.

qgib · 2019-02-26T11:50:01Z

Author Name: Adam Liddell (@aaliddell)

Also replicated this bug on 3.6.0 with Windows 7. The edits are written to a *.geojson.tmp file, but never copied over the top of the *.geojson file.

qgib · 2019-02-26T13:16:33Z

Author Name: Adam Liddell (@aaliddell)

Assuming this is the driver used for GeoJSON, could this be the offending section in GDAL OGR: https://github.com/OSGeo/gdal/blob/ab9e80368ef662712c963191fd86fde2c5b75600/gdal/ogr/ogrsf_frmts/geojson/ogrgeojsondatasource.cpp#L1056-L1098

Here's the steps it's taking:

The new *.geojson.tmp file containing the changes is written
The *.geojson is moved to *.geojson.bak
The *.geojson.tmp file is renamed to *.geojson
The *.geojson.bak file is removed

On Windows, this appears to be failing in step 2, as the tmp file is created but the bak file is not. Perhaps this is due to how Windows does file locking, meaning the rename is prohibited as the file appears to be open? Unfortunately I can't seem to get the log output from this section of code, so I don't see 'Cannot create backup copy' anywhere.

qgib · 2019-04-21T13:24:33Z

Author Name: Amine Aboufirass (Amine Aboufirass)

Has there been any progress on this? I also submitted a similar question on stackexchange... https://gis.stackexchange.com/questions/319450/qgis-values-in-attribute-table-become-null-after-saving

Pehaps there is a workaround?

qgib · 2019-04-22T11:34:58Z

Author Name: Evgeniy Lazarev (Evgeniy Lazarev)

I asked Oct-31-2018 on stackexcange about this problem: https://gis.stackexchange.com/questions/300860/why-does-qgis-3-4-and-higher-create-tmp-file-instead-of-refresh-editing-layer-a Since then I have to use 3.2.3 QGIS version.

qgib · 2019-04-24T19:54:32Z

Author Name: Edmond Lai (Edmond Lai)

I am having the same issue with QGIS version 3.6.2-Noosa on Windows 10.

qgib · 2019-05-24T01:49:15Z

Author Name: louis de clebs (louis de clebs)

I am having the same issue on windows 7 (Unfortunately have to use windows at work).
Might also have something to do with line endings? When I commit a geojson file created by QGIS, git put a warning saying LF line ending in the geojson file are replaced by CRLF. Would it be something there https://github.com/qgis/QGIS/blob/master/src/core/qgsvectorfilewriter.cpp#L1060 ?

When adding a feature to the geojson file it is correctly handled
When deleting or updating a feature the issue is happening

Only workaround at this stage is to save the file, close qgis, delete old geojson, rename new file by removing ".tmp" extension and re-open qgis... Or is there an easier way?

lmdc45 · 2019-06-10T23:18:04Z

Updated workaround :
If the file has been created from QGIS, use notepad++ or unix2dos to update EOL to Windows style.
When EOL is windows style, you can delete a feature. Before saving the file, copy any random feature from the layer, cut and paste it (just looks like you had a feature). Then saving is working. Points is that last action on the layer must be adding a feature.

timlinux · 2019-07-09T09:47:25Z

I tested the equipment.geojson file on macOS and it edits without issues.

aaliddell · 2019-07-11T17:10:13Z

Given this is marked as 'high priority', is a data-loss bug and has been idle for >7 months, is there a plan for addressing this?

Tagging this with 'Crash/Data Corruption' would also probably be suitable

gioman · 2019-07-11T17:13:23Z

@aaliddell Hi agree is high priority as any regression, but it would not be right to tag it as data corruption, as in fact there is not (the original data isn't lost or corrupted, isn't it?). You may want to chime in in this thread too: https://lists.osgeo.org/pipermail/qgis-developer/2019-July/057978.html

rouault · 2019-07-11T17:34:22Z

This sounds pretty much as similar issues with shapefile compaction, and it appears indeed that "something" on Windows keeps a lock on the file that prevents to rename it, but overwriting it should be possible (that's how this was fixed for shapefiles). The strange thing is that that part of the code is tested by GDAL autotests, and they are run on Windows through AppVeyor CI... What I suspect is that it might be related to a process like an antivirus running on normal people machines, but it might be disabled on AppVeyor. Could people try to temporarily disable their antivirus to confirm ?
Also, it would be interested if they could try outside of QGIS, both with antivirus enbled or disabled, the following python script (from the OSGeo4W console):

from osgeo import ogr
ds = ogr.Open('test.geojson', update = 1)
lyr = ds.GetLayer(0)
f = lyr.GetNextFeature()
lyr.SetFeature(f)
ds = None

with any 'test.geojson' file and see if error messages are emitted on the console

aaliddell · 2019-07-11T18:41:54Z

Hi agree is high priority as any regression, but it would not be right to tag it as data corruption, as in fact there is not (the original data isn't lost or corrupted, isn't it?)

Depends what's considered data. The original data is indeed not touched, but there's also the new data you want to add/edit/delete, which vanishes (to the tmp file first but that'll be lost if you save more than once). Before finding this issue I made a set of edits, saved the file and then moved on to other tasks assuming my edits were persisted; the file then not having the expected content appears to the end user as corruption despite the file still being valid GeoJSON, which is little consolation to having to redo work, but this is mostly semantics over a tag and I'm glad to see some movement...

Is there also the possibility that another part of QGIS is hanging onto an open file handle in recent versions? I ask because it's curious that old versions of QGIS still save GeoJSON fine, which I wouldn't expect to see if it's AV related. Perhaps there's a lsof equivalent on Windows.

I'll have a go with that script and also see if there's an AV log.

aaliddell · 2019-07-12T10:22:14Z

Here's a summary of a test that demonstrates this behaviour:

Open QGIS, in new document: Layer -> Create Layer -> New Temporary Scratch Layer
Draw a polygon
Make layer permanent, saving as GeoJSON
The contents are successfully written
Draw another polygon on the same layer
Click 'Save Layer Edits'
The contents are successfully written
Select one of the polygons and delete it
Click 'Save Layer Edits'
The contents are not successfully written and the .tmp file exists on disk. The deleted polygon pops back into existence either immediately or when refreshed
Open field calculator and add a field (e.g. number with all values 1)
Click 'Save Layer Edits'
The contents are successfully written
Pick a polygon and edit the new field you added above
Click 'Save Layer Edits'
The contents are not successfully written and the .tmp file exists on disk. The changed column reverts to the old value when refreshed.

There are no records of blocked actions in the AV log. So deletes and edits don't seem to get persisted, but additions do 🤷‍♂️. Doing the same steps but picking shapefile when making the layer permanent works fine.

rouault · 2019-07-12T10:42:30Z

Here's a summary of a test that demonstrates this behaviour:

Is it systematically reproducible, at the same steps ? There should be no difference really regarding the type of action, so I suspect something random.
What if you disable the AV ?

There are no records of blocked actions in the AV log

I wouldn't expect anything to be recorder in the AV log. But more an annoying interaction between it and QGIS/GDAL. That is that when new files are created / renamed, the AV would notice those file system changes and open the new file to analyze it, thus creating a temporary lock that prevents immediate further renaming.

aaliddell · 2019-07-12T10:57:13Z

Is it systematically reproducible, at the same steps ? There should be no difference really regarding the type of action, so I suspect something random.
What if you disable the AV ?

Yes, I've gone four times through those steps and I'm yet to see a different behaviour. I'm on a work machine currently, so can't disable AV presently 😒 but will try on a different machine later. However, Giovanni Manghi (@gioman) up above had this comment:

I can easily replicate on 3.4.4 (following the steps in the attached cast) on Windows (clean install of both QGIS and the OS).

Using process explorer (lsof equivalent) to examine open files, QGIS has two open file handles for the GeoJSON file whilst idle in edit mode. Once exiting edit mode, one of these file handles is closed but one remains (or possibly both closed and one opened, I cannot see the handle IDs). This may be the handle that is 'locking' the file from being changed. Do we know what part of QGIS is holding these handles and if there is a difference in behaviour with how it closes them when editing vs adding features?

rouault · 2019-07-12T11:32:04Z

OK, I think I know what is happening. Probably not antivirus related after all. The other file handle opened on the file comes from the renderer "threads", and they are cached for one minute even after they have completed. Probably that adding explicit QgsOgrConnPool::instance()->invalidateConnections() calls like done in a number of places in https://github.com/qgis/QGIS/blob/master/src/core/providers/ogr/qgsogrprovider.cpp would help, by forcing them to be closed. Reviewing closer, what you observe can be explained: adding a new feature at the end of a GeoJSON file doesn't require it to be entirely rewritten, so the rename game doesn't happen here. Regarding adding new fields, there's a special processing at https://github.com/qgis/QGIS/blob/master/src/core/providers/ogr/qgsogrprovider.cpp#L4570 that takes care of force closing everything.
So the fix would be two folds:

in QGIS, invalidating cached connections for all actions on GeoJSON files
in OGR, in the event that a reader would still be active despite above precaution (unlikely from the GUI), revert to overwriting the existing file.

aaliddell · 2019-07-12T12:25:26Z

Awesome 👍

rouault · 2019-07-12T12:33:11Z

@aaliddell I guess you could "workaround" the issue by waiting for one minute, and without doing any scroll or pan action or anything on the canvas, after each edit action and before saving the edits (that would be more a way to confirm my above hypothesis than a real workaround of course). That should be sufficient for the cached connections to be automatically closed.

aaliddell · 2019-07-12T13:30:58Z

Yep, can confirm that waiting a minute after making an edit before saving works

radumas · 2019-07-23T21:25:01Z

Yep, can confirm that waiting a minute after making an edit before saving works

I thought this did not work for me on Windows 7 and QGIS 3.6.3, but it was because I had multiple projects open in order to test with a "Clean" user profile. Can confirm waiting works for me as well.

lennoo · 2019-07-25T16:20:21Z

waiting works for me on windows 7 and QGIS 3,6. I wonder what version would fix the bug

Mike-Honey · 2019-08-16T06:23:31Z

Waiting a minute isn't working for me, I'm reduced to trying to snatch copies of the .tmp file, renaming them after closing QGIS. It seems when the QGIS project is saved or even just closed it reverts the .geojson files to their original state? Windows 10 and QGIS 3.8.1, also tried QGIS 3.4.10. I hope this can be fixed soon as it's dramatically slowing my workflow.

I would categorise it as "data loss" - we enter data using the app, it appears as permanent, yet later we find our changes have been lost.

GeoJSON driver: use VSIOverwriteFile() to fix update of file on Windows (fixes qgis/QGIS#28580)

…ws (fixes qgis/QGIS#28580)

FireByTrial · 2019-10-23T15:26:34Z

I'm still getting this on 3.8.3-Zanzibar (on Windows 10). open a .json/.geojson file and added a new column. edited column values and saved edits. these will be lost if a new attribute table windows is opened (after saving edits and stopping edits).
Additionally if the layer is duplicated the new column that was added will not exist in the duplicated layer

gioman · 2019-10-23T15:28:36Z

I'm still getting this on 3.8.3-Zanzibar. open a .json/.geojson file and added a new column. edited column values and saved edits. these will be lost if a new attribute table windows is opened (after saving edits and stopping edits).
Additionally if the layer is duplicated the new column that was added will not exist in the duplicated layer

@FireByTrial
probably the fix was not backported, so you must wait for the next release. You can try qgis master in the meantime.

rouault · 2019-10-23T15:32:18Z

probably the fix was not backported, so you must wait for the next release

The fix is in GDAL. No release have been yet made with it (unless QGIS master on Windows uses GDAL master, but I'm not sure?). Next GDAL maintainance releases (source tarball) are for next week

FireByTrial · 2019-10-23T15:36:21Z

I ended up exporting to geopackage and doing my edits with that format for now. It seems to work fine and I will just export back to geojson when done.

👍 Thanks for the quick replies.

lmdc45 · 2019-11-18T22:35:02Z

seems to work just fine with latest 3.10 release.

Thanks

Abhishek Arya (1): Add OSS-Fuzz fuzzing badge (#1912) Alan Thomas (5): DXF: Generate correct HATCH boundary elliptical arc segments for certain start/end angles DXF: Skip "Embedded Object" sections in 2018 version DXFs DXF: Specify maximum gap between interpolated curve points DXF writer: Re-add some header and trailer elements (fixes #1213) DXF: Correct handling of "off" and "frozen" layers in blocks Alexander Bruy (2): show warning when running gdal2tiles against non-Byte input (fix #1956) add note about non-Byte inputa to the gdal2tiles documentation Andrew Bell (1): libgeotiff: set UOMLength from GeogLinearUnits (for geocentic CRS) (#1596) Andrew Harvey (1): Doc: update ogr2ogr -explodecollections documentation with order relative to -sql (#2159) Anton Kovalev (1): Add PICREP attribute to LNDMRK object class Anton Wagensonner (1): gdal_edit.py: add -a_ulurll switch Anurag Kulshrestha (1): Docs: Fix type sizes in cosar.rst (#2089) Asger Skovbo Petersen (1): flatgeobuf.rst: Add ogr2ogr example (#1843) Bas Couwenberg (1): Fix spelling errors. Björn Harrtell (29): Addd read/write vector FlatGeobuf driver (#1742) Improve flatgeobuf.rst formatting and a short introduction (#1841) FlatGeobuf: Filtering fixes (#1844) FlatGeobuf: Rework/improve error handling FlatGeobuf: Fix VERIFY_BUFFERS open option handling FlatGeobuf: Buffer size bound checks No subtract for header max size Remove redundant const FlatGeobuf: Fixes for index handling FlatGeobuf: WKT roundtrip tests and nullptr validation (#1885) FlatGeobuf: Fix Z/M handling FlatGeobuf: Fixes for weaknesses detected by oss-fuzz (#1930) FlatGeobuf: Avoid early allocation with risk of leak FlatGeobuf: Additional size checks (#1944) FlatGeobuf: Write in two passes if index requested (#1967) (fixes #1964) FlatGeobuf: Read feature indices on demand (#1973) (fixes #1966) FlatGeobuf: Optimize reads for index search (#1993) Allow Travis sanitize build to work on plain Ubuntu 16.04 (#2061) FlatGeobuf: implement spec v2 (#2051) FlatGeobuf: Serialization cleanup (#2073) FlatGeobuf: Simplify memory management with unique_ptr (#2082) FlatGeobuf: Fixes for files with no index WFS: Support FlatGeobuf as WFS outputformat (#2135) FlatGeobuf: spec v3 (#2200) FlatGeobuf: Minor optimization to tree search FlatGeobuf: Advertize strings as UTF8 FlatGeobuf: Support mixed geometries (#2255) FlatGeobuf: Fix FID stability on spatial searches FlatGeobuf: add null check for readMultiPolygon (#2461) (fixes #2459) Blayne Chard (1): Docker: Allow https connections with curl Bojan Bizjak (1): Update WindowsLatin2 definition #1571 Chiara Marmo (3): Update FITS documentation for version 3.0 (#1598) Documentation: add FITS specific installation instructions in fits.rst. (#1646) Details on FITS implementation. Chris Mayo (7): configure: Run Autoconf 2.69 Enable Bash completions and control installation configure: Run autoreconf after bash-completion changes GDALmake.opt.in: silence datarootdir warning Doc: allow user full control over installation directory Python bindings: Fix SWIG warning of missing path for input file ogrsf_frmts/sqlite: simplify to one #include sqlite3.h Chris Tapley (1): ecwjp2 sdk 5.5 changes Cody Benkoski (1): DOCS: Updated broken links to point to correct pages (#2054) Craig de Stigter (4): Automated: Convert various comparisons into pytest.approx() calls Also use pytest.approx() where the tolerance is a variable tests: test cleanup for gcore/misc.py Fix test_tiff_write_157 failures due to faulty NaN comparisons Dan Baston (5): Format Virtual File Systems doc Add page for software using GDAL Update software links, remove dead entries Doc: Add documentation for vsis3 AWS_HTTPS option Doc: declare config options for virtual file systems Daniel Baston (3): Add ogrinfo doc .gitignore update Add ogr2ogr doc Daniel Brookes (1): Contour generator: fix SegmentMerger list iterator skipping and out of bounds error. (fixes #1670) (#1671) Daniel Lee (1): MSG driver doc: Update link (fixes #2205) (#2218) Daniel Morissette (1): Return BSB Dataset's PLY coordinates as a WKT POLYGON in a BSB_CUTLINE metadata DanielFEvans (1): Add missing documentation of -optim option (#2260) David Burken (1): Added search for proj library in lib64 directory. David Manthey (1): nitf/nitfimage.c: skip bad UDID data (fixes #1578) (#1586) David Morais Ferreira (1): Doc: Update gdal_fillnodata.rst (#2401) [skip travis] [skip appveyor] Dmitry Baryshnikov (10): ngw: Change endpoint in autotests CAD: Fix read ellipse and arc (fix #1886). CAD && SQLite: Removed unnecessary header include and changed strnlen to CPLStrnlen (fix #1920) WMS [AGS]: Remove unused parameters from url WMS [AGS]: Add unit test for ArcGIS Server URL with additional parameters NGW: Add support for additional raster types and QGIS styles. Add more server side attribute filters. Fix driver documentation. Add more unit tests. NGW: Fix dataset access mode MITAB: cleanup management of update flag (fixes #2170) MITAB: Add friendly layer name (description) support. NGW: Add support for layers geometry types with Z Dominik Nussbaumer (2): Add RDB raster driver to read *.mpx RDB 2 files (#1538) fixes warning reported by cppcheck Ed Leaver (1): configure: strip -L/usr/lib and similar from netCDF, MySQL, GEOS and SFCGAL lib path (#2395) Eli L Adam (5): Update gdal_viewshed.rst Docs: fix typo in virtual_file_systems.rst (#2025) Doc: use built_in_by_default and build_dependencies directive (#2055) [skip travis] [skip appveyor] Doc: use built_in_by_default and build_dependencies directive (#2058) [skip travis] [skip appveyor] Use built_in_by_default and build_dependencies directive for vector drivers (#2083) [skip travis] [skip appveyor] Emmanuel Arias (1): Doc: Fix typo on gdalwarp.rst (#2432) [skip travis] [skip appveyor] Enrico Weigelt, metux IT consult (1): Fix permissions of non-executable files Even Rouault (1308): gcore/gdal_version.h.in: set to 2.6.0dev Add docker/build-all.sh docker/build-all.sh: shellcheck fixes Drop executable permission from files where it is not appropriate Drop executable permission from files where it is not appropriate, and set it correctly on .py files Move include armadillo in a separate header Docker alpine-normal: add hdf4 and libkea Docker alpine-normal: fix previous commit that disabled the PDF driver docker/util.sh: add a --push option, tag builder image, and cleanup old -latest images Docker: improve layering docker/ubuntu-full/Dockerfile: parametrize versions ogr2ogr: emit better error message when using -f VRT ci/travis/common_install.sh: use http:// for now (see https://trac.osgeo.org/osgeo/ticket/2296) Docker: refresh proj-datumgrid-latest when it changes Docker: add ccache capability Python bindings: build modules in parallel Docker: use ccache for Python bindings Docker: add PROJ_DB_CACHE_DIR Docker: completely sync the remote cache after build [ci skip] Docker: tidy build scripts [ci skip] Docker: avoid hardcoding docker0 interface address [ci skip] Docker: bind rsync daemon to Docker inet only [ci skip] gdal_contour: remove explicit width/precision=12/3 of the elev field (fixes #1487) docker/alpine-ultrasmall/build.sh: remove erroneous python testing [ci skip] GTiff: do not generate a TIFFTAG_GDAL_METADATA with color interpretation information for JPEG YCbCr compression OGR_PDS: fix heap-buffer-overflow in error code path. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14512. Credit to OSS Fuzz GTiff driver: ensure that main dataset and overviews share the same lock, so as to avoid crashing concurrent access (fixes #1488) Docker: use more docker-ish way to implement our rsync daemon [ci skip] osr_esri.py: accept PROJ master result Change mentions of GDAL 2.5 to GDAL 3.0 gcore/gdal_version.h.in: set to 3.1.0dev GeoJSON: report 3D layer geometry types (fixes #1495) GPKG, MBTiles, PostgisRaster ddrivers: share lock of overview dataset with parent dataset (refs #1488) Add test case for refs #1488 NEWS: refresh [ci skip] GTiff doc: clarification for subdataset creation (refs #1458) HDF5: avoid error report of the HDF5 library when _FillValue attribute is missing Make GH5_FetchAttribute(CPLString) more robust and able to cope with variable-length string Test previous commit GDALReprojectImage(): properly take into account source/target alpha bands GTiff: prefix all member variables by m_ PostgreSQL: do not attempt to create VARCHAR(n) columns with n >= 10485760 (fixes #1508) ISIS3: fix parsing of lists in JSon metadata, and quote string items when writing lists when needed (fixes #1510) GTiff: decrease size of GTiffDataset structure GPKG: change default value of OGR_GPKG_FOREIGN_KEY_CHECK to NO, so as to avoid issues in downstream software GTiff: reset m_bGeoTIFFInfoChanged flag in FlushDirectory() GTiff: improve performance of internal overview creation (fixes #1442) Python bindings: honour gdal.UseExceptions() in numpy related methods (gdalnumeric module) (fixes #1515) GTiff: remove support for libtiff 3.X NITF: fix parsing of SENSRB TRE (fixes #1520) Add forgotten file in previous commit (refs #1520) formats_list.html: fix BAG capabilities (fixes #1522) [ci skip] Shape: better deal with empty .shp+.shx and SHAPE_RESTORE_SHX (fixes #1525) SHPRestoreSHX(): fix crash when .shx file cannot be created (and a memleak) doc/index.dox: link to 3.0.0 [ci skip] GXF: avoid closing the file pointer before being sure this is a GXF driver, otherwise this can prevent the opening of some raw format files (fixes #1521) ENVI: fix potential use of invalid pointer on some unusual std::string implementations (fixes #1527) GTiff: make -co NUM_THREADS produce reproducible output GTiff: factor some code of DISCARD_LSB functionality and improve test coverage tiff_write.py: test error cases of DISCARD_LSB GTiff: fix build ENVI: add read support for reading GCPs (#1528), and fix off-by-one offset on line,pixel on reading GCP GPKG: fix memory leak in case of corrupted database. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14683. Credit to OSS Fuzz EEDA: fix startTime / endTime comparisons that were incomplete in refs #1506 Fix previous commit (refs #1506) GTiff: fix variable name Add API documentation Add the overhaul doc structure Tweak theme Add gdal_formats.csv ogr_formats.csv WaSP: on creation, make sure the layer geometry type set on the feature definition is wkbLineString25D Travis: use ubuntugis-stable PPA for Trusty since they are no longer in unstable EPSG:4326 to EPSG:3857: fix invalid output with some input coordinates (3.0 regression) Add virtual_file_systems.rst and remove the overloaded highlight_language in conf.py Add raster_data_model.rst Add source/user/vector_data_model.rst Extent doc structure with placeholder pages Add placeholder for programs Port the raster API tutorial Port te raster driver tutorial Add GDAL Warp API tutorial Add OGR API tutorial Add OGR driver tutorial Add GNM API tutorial Add GNM architecture Add OSR API tutorial Add OGR SQL dialect page Add SQL SQLite dialect Add indexes to driver pages Add GDAL VRT driver page Fix links Move eeda driver to raster, add missing GeoPackage raster driver Remove debug stuff Fix in gdaldem.rst Add gnmmanage Add gnmanalyse Add gdal_translate Add gdaladdo Fix redirect for raster format Tweak gdalinfo Add gdalwarp Add gdaltindex Fix index directive in ogrmerge Add gdalbuildvrt and tweak program index Add rgb2pct Add rgb2pct Add gdal_merge and gdal2tiles Add gdal_rasterize Add gdaltransform Add nearblack Add gdal_retile and gdal_grid Add gdal_proximity Add gdal_polygonize Add gdallocationinfo Add gdalsrsinfo Generate all man pages Fix GMLAS pages Geopackage: fix link to aspatial Fix warnings Programs: use include mechanism for common options Adjustments for Doxygen XML output and Sphinx Fix last remaining warnings and turn warning-as-error on Add missing raster formats Fixes in raster formats Fixes in raster formats GeoJSON: fix recognizing some documents with members sorted alphabetically (fixes #1537) Improve PDF output Avoid TOC to be included twice in HTML output, and add logo to PDF front page GTiff: fix multi-threaded compression bug introduced in 2a4297e669166 (fixes #1535, master only) Fix links between drivers and add missing pages Fix links Add raster driver properties Fix warnings in PDF generation and include auxiliary pages; reenable -W for all targets Add full doxygen HTML output Add vector driver capabilities Move access to raster and vector doc to upper level Make 'software using GDAL' available from 'About' Add link to PDF file Fix wrong indentation of a few raster drivers Fix Doxygen build Adjust pipeline to remove the temporary symbolic link configure: fix detection of libpq in a non-standard place (3.0 regression) (fixes #1542) gdal_polygonize.py: fix outputing to geojson without explicit -f switch (fixes #1533) Add and fill a shortname directive fix warning Add redirects MRF: fix crash on corrupted dataset. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14782. Credit to OSS Fuzz MRF: fix memory leak in error code path. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14787. Credit to OSS Fuzz GTiff/libtiff: move defered tile/strip offset/bytecount loading to libtiff libtiff: BigTIFF creation: write TileByteCounts/StripByteCounts tag with SHORT or LONG when possible GPKG: insert more accurate spatial extent in gpkg_contents Fix formatting of tables in DXF & PDF, fix links in X-Plane Python bindings: refresh doc Add Java and Python docs .azure-pipelines.yml: use wget from Docker image Internal libtiff: resync with upstream Fix old Doxygen build fix old Doxygen build with grid tutorial Port gdal_grid tutorial to Sphinx Commit forgotten file .azure-pipelines.yml: remove doc-sprint branch, and disable CNAME .azure-pipelines.yml: set CNAME to gdal.org /vsitar/: support >100 character file names (fixes #1559) Docker scripts: update to openjpeg 2.3.1 /vsitar/: fix 94ffaf551457a660c7cdc5b04b42334911d0f46c commit (refs #1559) Doc: add missing pages for MG4Lidar driver .azure-pipelines.yml: copy gdalicon.png to root to avoid breaking autotest .azure-pipelines.yml: copy gdalicon.png to root to avoid breaking autotest Documentation: remove old .html and .dox files gdalwarp: fix wrong axis order when using source/target CRS being a geographic3D CRS such as EPSG:4979, and with vertical shift grid application (fixes #1561, GDAL 3.0 regression) Doc: migrate RFCs from Trac libtiff: fix vulnerability of https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14894 (master only) Doc: fix docs of NGW driver (fixes #1573) Add file that should have gone with previous commit (refs #1573) Doc: change links to RFCs from Trac wiki to internal versions Fix typos, mostly in doc/ Doc: fix install-docs and install-man target, make 'man' target depend on Doxygen, and fix a RST issue Doc: fix typo Doc: fix more links from svn to github Add preliminary text for RFC 75: Multidimensional arrays [ci skip] Make 'gdal_translate foo.tif foo.tif.ovr -outsize 50% 50% -of GTiff' work Doc: adjustments for PDF output due to previous commit Wording: in the context of TIFF, use 'striped' instead of 'stripped' (fixes #1572) Docker images: update to python 3 (fixes #1532) [ci skip] libtiff: Fix vulnerability in 'D' (DeferStrileLoad) mode (master only) (fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14908) Doc: add redirect for gdal_vrttut.html [ci skip] GTiff: make WEBP_LEVEL to be honored in Create() mode (fixes #1594) Disable randomly failing test (refs #1594) libgeotiff/geo_normalize.h: adjust comments (refs #1595) Doc: add redirect for gdal_virtual_file_systems.html [ci skip] GTiff: fix yesterday regression in which compression parameters are ignored in COPY_SRC_OVERVIEWS=YES mode (refs #1594) GTiff writer: in COPY_SRC_OVERVIEWS=YES, interleave mask with imagery, and add leader/trailer to strile gdalcompare.py: take into account mask bands GTiff: optimize read of cloud-optimized geotiffs GTiff: when updating a COG, detect and warn when we break the optimizations /vsicurl/: avoid downloading one extra block when the end offset is just at a chunk boundary Tests: check efficient reading of COG files Doc: mention Docker images [ci skip] GTiff COG optimizations: replace STRILE_ by BLOCK_ GTiff: address review comments of #1600 OGRSpatialReference::GetAxis(): make it work with a compound CRS (fixes #1604) gdal_priv.h: remove decaration of never implemented GDALDefaultBuildOverviews() function gdalinfo --format: avoid crash on invalid XML content for creation options doc/source/conf.py: fix case of links to github [ci skip] doc/source/conf.py: fix canonical_url [ci skip] tiff_ovr.py: restore cleanup of temporary files GTiff: fix crash with multi-threaded compression and overviews (master only) GMLAS: fix typo in error message CPLConfigOptionSetter: only reset thread-locale value, not global one mask.py and tiff_ovr.py: use gdaltest.config_option() Add initial createcopy COG (Cloud Optimized GeoTIF) driver GeoJSONSeq: Open() successfully only if at least one feature is detected GeoJSONSeq: fix previous commit to unbreak test_http_6 test KML: set OAMS_TRADITIONAL_GIS_ORDER for SRS returned on returned layers GDALInvGeoTransform(): make it work with scale and rotation/skew coefficients of small absolute value (fixes #1615) rfc75: update typos raised by @drons COG: add reprojection support gdalwarp: accept output drivers with only CreateCopy() capabilities importFromWkt(): emit a CPLError() in case of import failure (refs #1623) netCDF: avoid buffer overflow. master only. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=15143. Credit to OSS Fuzz /vsis3/: for a long living file handle, refresh credentials coming from EC2/AIM (fixes #1593) Doc: fix a few examples in Doxygen comments Doc: tweak presentation of example Python bindings: remove dead code since we switched to 'new generation' Python bindings COG driver: add NONE as potential value for OVERVIEWS creation option Doc: update a version number Doc PDF: improve formatting Rework code to make cppcheck 1.72 happy (false positive) VRTAddBand() doc: add note about the return type (fixes #1637) COG doc: add precision about NUM_THREADS creation option COG doc: additional precision Internal libgeotiff: upgrade to libgeotiff 1.6.0dev to support OGC GeoTIFF 1.1 GTiff: add support for reading and writing OGC GeoTIFF 1.1 (requires libgeotiff 1.6 for writing part) Fix MSVC linking issue related to TransformWithOptionsCache class (fixes #1639) FileGDB: add instrumentation to display related datasets (not compiled by default) COG: always create temporary files as BigTIFF (fixes #1655) COG: fix and simplify logic for temporary file creation (fixes #1655) COG: force INTERLEAVE=PIXEL for tempoary overview generation of multi-band datasets GDALCreateReprojectionTransformerEx(): do not emit error if reverse transformation fails, and fix crash when trying to use null reverse transformation GTiffDataset::PushMetadataToPam(): early exit when PAM is disabled, to avoid error messages Warper: fix GDAL 2.3 regression in a situation wih source nodata value, multiple bands and nearest resampling where the logic to detect which source pixels are nodata was inverted (fixes #1656) Fix pyflakes issue of previous commit SWIG: add osr.SetPROJSearchPath(path) that can be used since setting PROJ_LIB from C# does no work (fixes #1647) Doc MVT: json formatting MVT: make CONF option accept a filename as well MVT doc: update with CONF option accepting a filename as well Python bindings: use fragments for 2 verbose typemaps Update my email address in lots of files Update my email address in lots of files Hopefully last email replacement VRT Python: also probe libpython3.Xm.so.1.0 (fixes #1660) Docker: install libpython3.6 for ubuntu-small and ubuntu-full (fixes #1660) /vsicurl/: allow 'Connection timed out' CURL errors as candidate for HTTP retry GDALCopyWholeRasterGetSwathSize(): fix potential int overflows for big values of GDAL_SWATH_SIZE or GDAL_CACHEMAX BAG: on reading, handle non-standard pixel-is-area convention. On writing, add a CORNER_POINTS_EXTEND_HALF_PIXEL=YES creation option to use that convention (fixes #1643) BAG doc: document CORNER_POINTS_EXTEND_HALF_PIXEL=YES creation option (refs #1643) Docker: update alpine images to 3.10 [ci skip] /vsimem/: make Rename() error if destination file is not in /vsimem/ BAG: update tests related to #1643 Add a CPLCondTimedWait() /vsizip/ writing: in ZIP64 mode, also advertise 45 as the version in the central directory (avoids a warning from 'zip' utility) Shapefile driver: add read/creation/update support for .shz and .shp.zip NTv2: add support for the Canadian NAD83(CRSR)v7 / NAD83v70VG.gvb velocity grid /vsicurl/: GetFileSize(): when HEAD request does not return Content-Length header, retry with GET MVT: reduce memory usage when processing big geometries (fixes #1673) Python bindings: update to SWIG 3.0.12 to have better error message (fixes #1677) Shape: fix crash when creating a layer with a wkbNone geometry type but a SRS (3.0 regression) MapInfo TAB: don't left truncate numeric values in the .dat when the field formatting is incompatible, but error out (fixes #1636) gdallocationinfo: emit verbose error when dataset canot be opened (fixes #1684) gdallocationinfo and gdaltransform: print a hint when values are expected from the command line and stdin is an interactive terminal (refs #1684) GeoJSON: avoid SetFeature() to repeat first feature when looping over features (fixes #1687) Fix pyflakes issue (refs #1687) Shapefile: launder layer name to get Windows compatible filename HOWTO-RELEASE: ./swig/python/README.txt -> ./swig/python/README.rst [ci skip] COG: for mask band overview computation, use same resampling kernel as other bands autotest: remove use of deprecate message argument of pytest.raises() (fixes #1689) autotest/requirements.txt: avoid too recent versions of pytest and more-itertools incompatible of Python2.7 netCDF: correctly parse grid_mapping attribute in expanded form GTiff: improve performance of generation of internal overviews COG: for mask band overview computation, try to use the same logic as for alpha band: that is 0 values in the source mask are ignored PG: add support for PostgreSQL 12 (fixes #1692) netCDF: refactor SetProjectionFromVar a bit COG: use correct type for RES creation option XML validate_cloud_optimized_geotiff.py: emit error when KNOWN_INCOMPATIBLE_EDITION=YES is found GTiff: set a LAYOUT=COG metadata item in the IMAGE_STRUCTURE metadata domain of the dataset when the hidden feature declarations typical of a COG file are found XLSX driver: add support for .xlsm extension Doc: advertize GDAL 3.0.1 release GTIFF / COG: avoid crash when trying to write dataset with band count invalid for WEBP (master only) cog.py: fix previous commit Doc: add Conda badge Doc: add link to 2.4.2 Map OSRGetAxesCount() to C API and SWIG bindings epsg_tr.py: refresh with PROJ 6 capabilities geotiff.cpp: remove useless assert Add explicit keywords (raised by cppcheck) Fix cppcheck warnings about calling a virtual method in a constructor/destructor Doc: fix PDF build GTiff: fix memory leak with -co APPEND_SUBDATASET=YES gt_overview.cpp: remove obsolete libtiff < 4 test Python bindings: add compatibility with SWIG 4.0 (based on patch by @perestoronin, fixes #1702) tile_extent_from_raster.py: sample script to generate the extent of each raster tile in a overview as a vector layer Internal libtiff: resync with upstream tile_extent_from_raster.py: fix coordinate precision GTiff: fix error message for NBITS != 16 and Float32 PAMRasterBand: add presence flag for Offset and SetScale so that GetOffset()/GetScale() properly reports if they are defined in PAM RFC 75 text: update with implementation work s57objectclasses.csv: add missing TXTDSC attribute for DRYDOC class (fixes #1723) gdalwarp: adjust nodata values, passed with -srcnodata/-dstnodata, and close to FLT_MAX to exactly it (fixes #1724) appveyor.yml: force installation of SWIG 3.0.12 since the C# bindings aren't compatible of SWIG 4.0 (fixes #1726) test_osr_set_proj_search_paths.cpp: augment to test threads ogr_proj_p.cpp: fix code comment [ci skip] GTiff: ensure GDAL PROJ context is used for all libgeotiff functions (requires internal libgeotiff / libgeotiff 1.6) ISSUE_TEMPLATE.md: add note about expectations regarding bug fixing, inspired from QGIS [ci skip] HDF5 and netCDF: fix crash when reading attributes of type string of variable length with NULL values Avoid crash on test_netcdf_dimension_labels_with_null with too old netCDF version netCDF: very partial workaround for an issue in libnetcdf 4.6.3 and 4.7.0 with unlimited dimensions (refs https://github.com/Unidata/netcdf-c/pull/1442) Replace mentions of GDAL 2.5 by 3.0 cpl_safemaths.hpp: safe + and * for GUInt64 Add CPLJSonStreamingWriter class RFC 75: initial support for multidimensional arrays Travis gcc52_stdcpp14_sanitize: upgrade numpy version RFC75: VRT multidimensional support RFC75: add gdalmdimtranslate gdalmdiminfo: uses reference to dimensions already displayed RFC75: add documentation HDF5 test: test file with recursive groups HDF5ImageRasterBand::IReadBlock(): fix memleak in error code path gdalmdiminfo: robustness fix USGSDEM: avoid int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=15715. Credit to OSS Fuzz gdalmdiminfo: display subgroups at end of group gdalmdimtranslate: fix target name array and remove warning when copying from nested array netCDF multidim: fix performance issue GDALMDArray::CopyFrom(): fix memory leak NCDFGetCoordAndBoundVarFullNames(): fix memory leak (non RFC75 related) NCDFWriteSRSVariable(): handle situation where the grid mapping variable already exists (needed for RFC75) netcdf_multidim.py: test writing several georeferenced variables HDF5 type detection: only detect complex data type if the component names start with r/R and i/I (refinement of refs #359) HDF5: add multidimensional read support GDALDriver::CreateMultiDimensional(): set description and driver on newly created dataset gdalmdiminfo: output driver name HDF5: use free() on non-Windows platforms when H5free_memory() not available HDF5: compatibility trick for older hdf5 library GDALDefaultOverviews::BuildOverviews(): fix typo in detection of 1x1 overview (fixes #1730) RFC75: update status and voting history PG: be more restrictive when deducing non-nullability of columns in SQL result layers (fixes #1734) DAAS: accept 4-band RGBA PNG response even when a single band is requested LIBKML driver: make it accept /vsigzip/foo.kml.gz files (fixes #1743) jp2metadata.py: make test_jp2metadata_4 work with latest PROJ master Fix previous commit ci: temporarily freeze spatialite to a particular revision since HEAD builds are broken if rttopo is not enabled ogr_ngw.py: disabled for now (refs #1746) ngw.py: disabled for now (refs #1746) Revert "ogr_ngw.py: disabled for now (refs #1746)" Revert "ngw.py: disabled for now (refs #1746)" osr_metacrs.py: force cleanup of objects to please ASAN /vsicrypt/: fix memleak in error code path vsicrypt.py: fix memleak Travis CI: port gcc52_stdcpp14_sanitize job to Xenial since Precise is discontinued FITS: fix memory leak Fix failure on test_tiff_read_online_2 hdf4_read.py: fix URL and add test for HDF EOS SWATH HDF5 multidim: fix data type conversion with compound data types HDF4: fix retrieval of non-string swath and grid attributes HDF4: fix GR support for non-Byte rasters, remove broken attribure reading, and fixes related to color table HDF4: add multidimensional read support (RFC 75) appveyor.yml: use VS2017 for x64 builds (fixes #1747) Internal libtiff: resync with upstream GDALMDArray::AsClassicDataset(): add support for 1D arrays HDF4 multidim: add simple non-GDAL SDS test, and fix for unlimited dimension OGRExpatRealloc(): fix double-free when size to allocate is above the default 10MB threshold. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16178. Credit to OSS Fuzz Move GDALAttributeString to core Degrib: add support for ReadGrib2Record() wit Grib_Data == nullptr core: add GDALAttributeNumeric class nmake.opt: mention that SETARGV should be wsetargv.obj (refs #1749) GRIB: add support for multidim API Core: add GDALDimensionWeakIndexingVar and GDALMDArrayRegularlySpaced classes, and use them in GRIB and HDF4 drivers PDF: fix error message [ci skip] Skip test_ogr_shape_115_shp_zip() on Travis Mac due to super frequent hangs appveyor.yml: re-enable VS2015 builds (fixes #1747) netCDF: fix warning for 32 bit mode netCDF 1.8 SF writing: remove unused variable NGW: fix warning for 32 bit mode PCIDSK SDK: remove use of deprecated function fuzzers/build_google_oss_fuzzers.sh: use PDF: avoid floating-point division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16327. Credit to OSS Fuzz Fix -Wimplicit-int-float-conversion warnings Internal libpng: upgrade to 1.2.59 multidim: add 'final' keywords and a few fixes for issues raised by cppcheck frmts: add 'final' whereever possible, and otherwise add explicit CPL_NON_FINAL ogrsf_frmts: add 'final' whereever possible, and otherwise add explicit CPL_NON_FINAL Replace a few catch of bad_alloc by exception to handle the case of 32 bit trying to allocate more than 4GB of memory and thus throwing a length_error exception. Found by OSS Fuzz third_party/LercLib/Lerc.cpp: resync with upstream (Android related fix) WMS: IReadBlock(): limit number of tiles downloaded at once libopencad: CADBuffer: replace m_guard by m_nSize to avoid pointer wrap around on 32 bit platforms on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16388. Credit to OSS Fuzz DGN: avoid size_t overflow / illegal memory access. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16393. Credit to OSS Fuzz VSIIngestFile(): avoid attempting to allocate too much RAM when erroneously opening a directory. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16394. Credit to OSS Fuzz libopencad: disable CRC and signature checks on -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION libopencad: CreateFileMap(): avoid infinite loop with section of size 0. Credit to OSS Fuzz gcore/gdalmultidim.cpp: fix use of moved variable (spotted by cppcheck) Fix a few (non-bug) cppcheck warnings OJPEGReadBufferFill(): avoid very long processing time on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16400. master only NAS: avoid long processing time on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16248. Credit to OSS Fuzz PDF: fix nullptr dereference on corrupted file. Credit to OSS Fuzz CPLStrtodDelim(): rework to avoid false positive from Coverity. CID 1404088 ComputeSourceWindowStartingFromSource: avoid mostly false postive from Coverity. CID 1404049 gdal_grid_lib: avoid mostly false postive from Coverity. CID 1404022 and 1404158 gdaldem_bin.cpp: avoid Coverity warning about tainted data. CID 1404163 gdalmdimtranslate_lib.cpp: fix a few Coverity warnings. CID 1404020, 1404147, 1404156, 1404198, 1404240 gdaltransform: silence false positive Coverity warning. CID 1214439 GDALWarp(): fix memleak in error code path. CID 1404058 gnmmanage: silence false positive Coverity warning. CID 1404019 GDALNearblack(): fix memleak if psOptionsIn==null. CID 1404128 ogrlineref: silence false positive Coverity warning. CID 1404238 blx.c: silence false positive Coverity warning. CID 1214445 blx.c: const correctness fixes DAAS: fix Coverity false positives. CID 1404079, 1404179, 1404139 FIT: fix Coverity false positive. CID 1404192 GeoRasterDataset::JP2_CopyDirect(): fix false-positive about memleak. CID 1404200 GeoRasterWrapper::LoadNoDataValues(): avoid memory corruption if method was called several times (which should not happen). CID 1404053 GRIB: avoid Coverity false positive. CID 1404168 TIFFClientOpen(): fix memory leak if one of the required callbacks is not provided. Fixed Coverity GDAL CID 1404110 hdf5multidim.cpp: various Coverity fixes. CID 1404104, 1404106, 1404194, 1404248 netcdfmultidim.cpp: various Coverity fixes. CID 1404093, 1404132, 1404178 netcdfsgwriterutil.cpp: fix Coverity Scan CID 1404035, 1404051, 1404052, 1404086, 1404162, 1404214, 1404217, 1404250 PDF composer from XML: fix wrong use of SetupVectorGeoreferencing(). Spotted by Coverity CID 1404081, 1404166. Also fixes 1404097 (false positive) PDF: silence Coverity false positive. CID 1404067 More uses of const reference as suggested by cppcheck CPLPopFileFinderInternal(): fix cppcheck false positive tiledb: initialize member variable in initializer list as suggested by cppcheck VFK: use a faster implementation of VFKDataBlockSQLite::LoadGeometryPolygon() Internal libtiff: resync with upstream. Includes a security fix for 32-bit builds GTiff: fix buffer overflow in DiscardLsb on 32-bit data type (master only) PDF read vector: avoid floating point division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16419 gdal_contour: silence Coverity warnings. Fixes 1404045 PDF: silence Coverity warning. Fixes 1404221 PDS: silence Coverity warning. Fixes 1404218 PostgisRaster: silence Coverity warning. Fixes 1404244 RIK: check return code of uncompress() method. CID 1404212 SAFE: avoid potential use-after-free. CID 1404037 and 1404140 Terragen: avoid potential use-after-free on error code path in write_header(). CID 1404060 vrtmultidim: avoid false positive Coverity warning. CID 1404072 GDALGCPsToGeoTransform(): silence Coverity warning. CID 1404056 GDALGeneralCmdLineProcessor(): fix double free with --optfile (CID 1404146) Python bindings multidimensional API: fixes buffer overflow when wrong arguments are provided to Array.Read() gdaljp2metadatagenerator.cpp: workaround to avoid ASAN issues when linking against filegdbAPI CTable2Dataset::SetGeoTransform(): fix read buffer overflow from stack Fix memory leak in GDALGeneralCmdLineProcessor() introduced in 04892bffec5bd469547d3562715882c080547392 configure: trick so that LD_SHARED test works with sanitizer flags with clang Add cpl_error_internal.h with logic with error accumulator (avoids a false positive UndefinedBehaviour warning about a function pointer, apparently due to the error handler being in the binary and the call site in the library) Travis sanitize: do fail if a test crashes on ASAN PDF: avoid nullptr dereference on corrupted file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16438. Credit to OSS Fuzz PDF: avoid floating point division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16439. Credit to OSS Fuzz Internal libtiff: fix integer overflow potentially causing write heap buffer overflow, especially on 32 bit builds. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16443. Credit to OSS Fuzz gdaldataset.cpp: silence false positive warnings. CID 1404117 and 1404177 gdaldriver.cpp: silence false positive warning. CID 1404114 gdaljp2abstractdataset.cpp: silence false positive warning. CID 1404223 gdalmultidim.cpp: fix multiple coverity warnings. CID 1404023, 1404040, 1404085, 1404094, 1404164, 1404181, 1404235 gnm.h: silence Coverity warning. CID 1404225 gml2ogrgeometry.cpp: silence Coverity warning. CID 1404036 ogr_geocoding.cpp: silence Coverity warning. CID 1404222 ograpispy.cpp: add move assignment operator. CID 1404059 ogrct.cpp: silence Coverity warning. CID 1404112 ogrgeometryfactory.cpp: silence Coverity warning. CID 1404143 ogrpgeogeometry.cpp: silence Coverity warning. CID 1404087 dgnwrite.cpp: silence Coverity warning. CID 1404084 ogredigeodatasource.cpp: silence Coverity warning. CID 1404173 ogrelasticdatasource.cpp: silence Coverity warnings. CID 1404071, 1404074, 1404111, 1404129, 1404239 FGdbLayer: silence Coverity warnings. CID 1404122, 1404191 VSIStatExL(): fix Coverity warning. CID 1404203 geoconcept.c: attempt at fixing Coverity CID 1404157 libjson/json_util.c: add missing cast. CID 1404169 ogrgeojsonseqdriver.cpp: validate config option value. CID 1404165 gmlreader.cpp: fix false positive warning. CID 1404205 ogrgmldatasource.cpp: avoid false positive CID 1404199 ogrgmlasreader.cpp: silence Coverity warning. CID 1404055 ogrgeojsonseqdriver.cpp: fix recent 19d7faa223cc9a5c1eb15b3e29f67719e0df9fd1 breakage ogrgmlasschemaanalyzer.cpp: silence Coverity warnings. CID 1404031 and 1404137 ogrgeopackagedatasource: silence Coverity warning. CID 1404202 LIBKML: fix potential memory leak. CID 1404148 ogrmemlayer.cpp: avoid assignment of overlapping memory. CID 1404061, 1404141, 1404152 MITAB: fix potential double-free. CID 1404224 MITAB: avoid potential nullptr deref. CID 1404174 nasreader.cpp: avoid warning about potential nullptr deref. CID 1404116 filegdbtable.cpp: silence Coverity warnings. CID 1404102, 1404135 ogrpgtablelayer.cpp: avoid Coverity warning. CID 1404127 ogrshapedatasource.cpp: avoid Coverity false positive warning. CID 1404153 SQLite: avoid false positive Coverity warnings. CID 1404092, 1404229, 1404182 VFK: add move assignment operator to VFKProperty for (micro) better performance. CID 1404215 cpl_http.cpp: silence Coverity warnings. CID 1404161, 1404206 cpl_json.cpp: add move assignment operator. CID 1404096 cpl_json.cpp: fix memory leak introduced in previous commit Internal libtiff: resync with upstream GTiff: add explicit error message when trying to create a too big tiled TIFF file (refs #1786) PDF: fix memleak on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16501. Credit to OSS Fuzz GRIB1: avoid excessive memory allocation attempt and potential crash on allocation failure. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16452. Credit to OSS Fuzz AVCE00: fix alternance from sequential to per-FID reading. Also fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16471. Fix or tag as allowed all uses of unsigned integer overflow, add add -fsanitize=unsigned-integer-overflow to the Travis sanitize configuration CPLStringList: add move assignment operator. CID 1404207 cpl_vsil.cpp: silence false positive warning. CID 1404216 cpl_vsil_az.cpp: silence warning. CID 1404064 cpl_vsil_crypt.cpp: silence warning. CID 1404090 cpl_vsil_curl.cpp: silence warning. CID 1404227 cpl_vsil_gzip.cpp: silence false positive warning. CID 1404220 cpl_vsil_s3.cpp: silence warnings. CID 1404041, 1404062, 1404095 cpl_vsil_swift.cpp: silence warning. CID 1404125 Shape: try to better deal with polygons with parts touching on an edge (which is illegal simple features) (fixes #1787) cpl_vsil_crypt.cpp: catch potential exception. CID 1404090 nasreader.cpp: silence Coverity warning. CID 1404116 PDF: avoid opening too highly corrupted files. Should fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16522 FITS: fix memory leaks PCRaster / libcsf: fix many Coverity warnings about assignment of overlapping memory PDF: fix potential heap buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16546. Credit to OSS Fuzz OGRSpatialReference::SetFromUserInput(): add capability to import PROJJSON PDF: fix potential nullptr deref on corrupted file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16558. Credit to OSS Fuzz BSB: avoid to use unsigned int overflow OSR: add ExportToPROJJSON() (PROJ >= 6.2) RMF: avoid unsigned int overflow on corrupted file degrib: avoid (harmless) unsigned integer overflows ERS: avoid (harmless) unsigned integer overflow Internal libtiff: avoid (harmless) unsigned integer overflow BSB: avoid harmless unsigned integer overflow on corrupted files degrib: avoid harmless unsigned integer overflow on corrupted files degrib: avoid harmless unsigned integer overflow on corrupted files EXIF reader: avoid harmless unsigned integer overflow PCIDSK: avoid harmless unsigned integer overflow PCIDSK: avoid harmless unsigned integer overflow GRIB: avoid potential unsigned integer overflow PCIDSK: avoid harmless unsigned integer overflow PCIDSK: avoid harmless unsigned integer overflow PCIDSK: avoid harmless unsigned integer overflow /vsigzip/: avoid harmless unsigned integer overflow BMP: compute file offsets on vsi_l_offset to avoid unsigned integer overflow Doc: mention PROJJSON format for gdalsrsinfo NWT: avoid harmless unsigned integer overflow NITF: avoid unsigned integer overflow GDALGeneralCmdLineProcessor(): fix master regression regarding display of the type of driver NWT: fix compilation issue on machines where char is signed, introduced by recent master commit /vsitar/: accept space as end of field terminator GRIB: fix MSVC warning introduced in yesterday commit /vsigzip/: avoid harmless unsigned integer overflow Internal libtiff: avoid unsigned integer overflow LercLib: avoid big memory allocation attempt on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16595. Credit to OSS Fuzz Support API-level precision/round for geometry WKT (#1797) (fixes #1174) Internal libtiff: avoid unsigned integer overflow third_party/LercLib: resync with latest upstream suggestions Doc: add mostly empty template for SOSI driver gdalexif: avoid unsigned integer overflow Internal libtiff: resync with upstream /vsigzip/: fix harmless unsigned integer overflow test_ogr_sql_sqlite_26: fix issue with GEOS head OpenFileGDB: ignore unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16629 netCDF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16631 Internal libtiff / tif_thunder: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16632 /vsitar/: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16633 Internal libtiff / tif_ojpeg: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16635 BitStuffer2.h: avoid harmless unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16638 ERS: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16640 libopencad: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16626 Internal libtiff / EstimateStripByteCounts(): avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16643 HFA: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16644 RawDataset / VSISubFileHandle: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16645 RIK: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16646 HFA: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16648 SEGY: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16649 /vsiwebdfs/: avoid harmless unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16650 MSGN: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16652 Internal libtiff / TIFFFillStrip: avoid harmless unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16653 VSISubFileFilesystemHandler::Stat(): avoid unsigned int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16655 NTF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16656 VSISubFileFilesystemHandler::Open(): avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16658 L1B: avoid reporting negative dataset height. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16663 Selafin: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16666 libopencad: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16672 RawRasterBand::AccessLine(): avoid unsigned int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16673 libtiff / OJPEGWriteHeaderInfo(): avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16683 Internal libtiff / _TIFFPartialReadStripArray: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16685 RAW: avoid unsigned integer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16686 NTF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16691 RPFFOC: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16695 PCIDSK SDK: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16698 OpenFileGDB: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16713 LercLib: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16714 OpenFileGDB: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16716 NITF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16717 VRT: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16727 GRIB: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16728 SRP: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16730 GRIB: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16741 OpenFileGDB: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16745 MRF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16757 Internal libtiff: resync with upstream PDF: fix nullptr dereference on corrupted file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16759. Credit to OSS Fuzz NWT: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16762 OpenFileGDB: fix for potential unsigned integer overflow when .gdbtablx would be > 4GB. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16781 HFA: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16783 Internal libtiff: fix unsigned integer overflow / bad cast from uint64 to tmsize_t. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16784 RAWDatasetCheckMemoryUsage(): avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16785 BMP: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16789 MSGN: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16790 PCIDSK SDK: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16791 Internal libtiff: TIFFReadDirEntryData(): rewrite to avoid unsigned integer overflow (not a bug). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16792 appveyor.yml: update for PROJ 6.2 Internal libtiff: tif_ojpeg.c: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16793 NITF: fix unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16806 ENVI: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16808 OpenFileGDB: avoid report of unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16809 NITF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16812 Internal libtiff / tif_dirread.c: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16846 EXIFExtractMetadata: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16849 appveyor.yml: explicitly build PROJ 6.2 Internal libtiff: tif_ojpeg.c: avoid use of uninitialized memory on edge/broken file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16844 Revert unintended changes that went in previous commit SOSI: add a (weak) identify method HDF5: add a way to open datasets split over several files using the 'family' driver Add files that should have gone in previous commit multidim support: handle 0-size dimension VRT multidim: on reading, use correct group fullname NCDFIsUserDefinedType(): make it work for types in subgroups multidim support: deal with different arrays having 'internal' dimensions of same name but different size COG driver: add LERC compression (fixes #1812) Fix compiler warning on MSVC Internal libtiff. ByteCountLooksBad and EstimateStripByteCounts: avoid unsigned integer overflows. Fixes https://oss-fuzz.com/testcase-detail/5686156066291712 and https://oss-fuzz.com/testcase-detail/6332499206078464 Fix issues with thread_local and C++ objects that don't work well with DLL on Windows Doc: mention AWS_DEFAULT_PROFILE instead of AWS_PROFILE (fixes #1807) GRIB: add a hint to increase GRIB_CACHEMAX PCIDSK SDK: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16957 ISIS2: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16967 /vsitar/: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16978 PCIDSK SDK: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16996 GDALFindDataType(): simplify a bit, and add tests VICAR: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16999 Regenerate python bindings GPX: fix memory leak when streaming to /vsistdout/ VICAR: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17081 GTiff: make sure that GetMetadataDomainList() doesn't return EXIF when there's no EXIF metadata (fixes https://github.com/mapbox/rasterio/pull/1740#issuecomment-526660946) PDS: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17125 PDF: avoid potential integer division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17129 ogr_proj_p.cpp: OSRPJContextHolder() remove call to init() in constructor (fixes #1838) ENVI: preserve 'byte order' on update (fixes #1796) vrt_read.py: attempt at fixing random failure on test_vrt_shared_no_proxy_pool() on trusty_clang gdal_translate: avoid unsigned integer overflow with -limit_outsize option. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17186 NITF: avoid unsigned integer underflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17249 HOWTO-RELEASE: multiple updates, in particular to reflect the 2-phase procedure with RC first and then promotion to final Add VSIOverwriteFile() Shapefile driver: use VSIOverwriteFile() Internal libtiff: avoid too large memory allocation attempts. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17244 GeoJSON driver: use VSIOverwriteFile() to fix update of file on Windows (fixes https://github.com/qgis/QGIS/issues/28580) Doc: migrate https://trac.osgeo.org/gdal/wiki/GovernanceAndCommunity to RST, add new members Sean Gillies and Mateusz Loskot, move Andrey to retired members Add OSRPromoteTo3D() and map it to SWIG (PROJ >= 7) (fixes #1852) JP2OpenJPEG: fix reading overviews, when tiled API is used, and the dimensions of the full resolution image are not a multiple of 2^numresolutions (fixes #1860) JP2KAK: add a test related to #1860 WFS3: correctly handle user query string parameters in connection URL (fixes #1710) Add autotest/gdrivers/data/513x513.jp2 that should have gone with ed5fefc1b5ec7aeeec73af4cd687ec958cd5ad3c GeoJSON: on writing, format OFTDate and OFTDateTime as ISO 8601 strings rather than OGR traditional formatting NITF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17594 FlatGeoBuf: fix memory leak in error code path. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17457 PDS4: add a CREATE_LABEL_ONLY=YES creation option, and create a <Header> element (fixes #1832) GPKG: on layer creation, check if the SRS is consistent with its advertize AUTHORITY/ID, and if not do not use official EPSG entries (fixes #1857) GDALRasterAttributeTable::TranslateToColorTable(): avoid potential int overfow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17676 OGRSpatialReference::importFromESRI(): acept COMPD_CS (fixes #1881) PDF: update to use newer versions of pdfium ISIS3: extract band name from BandBin group, and wavelength/bandwidth (fixes #1853) OGR SQLite: do not propagate 'IS / IS NOT value' constructs to OGR SQL JP2OpenJPEG: correctly set coordinates to opj_set_decode_area() on overviews (linked to previous fix) jp2openjpeg.py: avoid potential memory issue with old openjpeg versions ISIS3: preserve label in PAM .aux.xml when copying to other formats (fixes #1854) ElasticSearch: add FORWARD_HTTP_HEADERS_FROM_ENV open option to pass HTTP headers down to the ES server Add CPLLaunderForFilename() Shape: use CPLLaunderForFilename() FlatGeoBuf: add support for multiple layers in a directory of .fgb files, and fix single layer support (fixes #1862) configure: remove useless PDFIUM_LIBDIR test_ogrsf: do not error because FlatGeobuf does not return the expected layer geometry type configure: add -lm -lstdc++ for pdfium linking Travis: add support for PDFium in Ubuntu 16.04 build PDF: fix build against Poppler 0.82.0dev PDS4: update CART schema to 1B10_1931 and PDS to 1C00 (fixes #1851) Move oss-fuzz build.sh script to GDAL repository FlatGeobuf: constify and pass more arguments by reference in PackedRTree and Rect classes USGSDEM: fix reading FEMA generated Lidar datasets whose header is 918 bytes large GDALOpenEx(): more explicit error message if no driver are registered OSR: add an internal cache for importFromEPSG() and importFromWkt(). Helps performance for MapServer PROJ6 migration Doc: fix broken links to functions GTIFGetOGISDefn: avoid querying UOM length info when it is KvUserDefined to avoid an error to be emitted dump_jp2.py: fix for Python2 and non-ASCII characters Add support for Vertical Perspective projection (refs #1856) GTiff and PAM: allow serializing WKT2 for SRS using non-WKT1 compatible projections such as Vertical Perspective (refs #1856) gdal_translate / gdalwarp / ogrct: allow dealing with non-WKT1 representable SRS (refs #1856) ISIS3: add support for PointPerspective projection (refs #1856) OpenJPEG: fix to return the proper number of bytes read when we read more than 2 GB at once (fixes https://github.com/uclouvain/openjpeg/issues/1151) GTiff: do not write in GeoTIFF keys non-standard projections ISIS3: add support for Oblique Cylindrical projection (refs #1856) PDS3: fix support of Oblique Cylindrical (refs #1856) Internal libtiff: resync with upstream VRT: avoid erroneous pixel request do be done with KernelFilteredSource MRF: avoid excessive memory allocation attempt. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18011 ossfuzz: fix Poppler build Revert "ossfuzz: fix Poppler build" cpl_json: add const qualifiers ogrinfo: add a -nogeomtype switch WFS3: align with the latest state of the spec (refs #1878) Rename WFS3 driver as OAPIF (fixes #1878) OAPIF: clamp bounds of bbox filter OAPIF: add persistent curl session for better performance OAPIF: avoid TestCapability(OLCFastFeatureCount) to cause a network access, as well as SetAttributeFilter(NULL) OAPIF: remove no longer needed hack DDS: fix compiler warnings related to NULL vs nullptr OAPIF: remove outdated/no longer working support for XML that prevented to interoperate with updated cubewerx server OAPIF: for now do not look at all the 'crs' property, and always advertize CRS84 Doc OAPIF: update link to official standard ParseGMLCoordinates(): fix typo in error message [ci skip] Doc: fix broken link MSGN: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18233 frmts/hdf5/makefile.vc: unbreak Windows build IGNFHeightASCIIGrid: fix to read RASPM2018.mnt grid OGRSimpleCurve::setNumPoints(): avoid potential int overflow on 32 bit on big values of nNewPointCount. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18092 PDF: fix reading polygon with holes and Bezier curves (fixes #1932) PDF: add a GDAL_PDF_LAUNDER_LAYER_NAMES configuration option Selafin: temptative fix for stack read overflow of https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18342 MRF: do not hardcode GDAL internal zlib header (from https://github.com/opengisch/OSGeo4A/blob/master/recipes/gdal/patches/gdal-zlib.patch) PDF vector: fix reading strings with escape sequences ParseGMLCoordinates(): avoid multiple realloc and potential performance issues, especially under fuzzing instrumentation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18349 OGRSpatialReference::importFromESRI(): avoid potential performance issue on hostile input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18033 HFA: avoid potential performance issue on hostile input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18088 Fix false positive MSVC warning about variable potentially not initialized OGR_SDTS: avoid potential performance issue on hostile input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17740 BNA: code cleanup and hopefully fix performance issue of https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17572 BNA: fix MSVC warning due to previous commit DXF: rework generation of features in INSERT blocks to avoid too much processing time/memory usage. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17282 DXF: ResetReading(): properly reset insert state (follow-up previous commit) XLSX: avoid potential performance issue on hostile input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17425 FlatGeoBuf: fix writing SRS without authority, and on reading avoid null pointer dereference if missing authority ogr_flatgeobuf.py: use a code that works with earlier PROJ 6 versions DXF writer: do not error out if trying to create a OGR_STYLE field (refs https://github.com/OSGeo/gdal/issues/835#issuecomment-544297557) FlatGeoBuf: use envelope from header to implement GetExtent() FlatGeoBuf: fix reading M-only geometries OGRSimpleCurve::getPoints() with XYZM: fix wrong stride used for M array OGRSimpleCurve: remove getPoints(XYZ) method to use getPoints(XYZM) with arguments with default value FlatGeoBuf: avoid crash on empty geometries; small cleanups OGRSimpleCurve: fix reversePoints() and addSubLineString() to take into account M dimension Fix Doxygen warnings of recent commit Fix expected results due to commit ad9178b63f501e9840d86a3220c9e5100c6e3fa2 importFromWkt(): limit size of input to avoid later potential performance problems. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18421 PDF: fix build against Poppler 0.82.0dev Internal libtiff: resync with upstream JPEG: fix further calls to RasterIO after reading full image at full resolution (fixes #1947) Internal libtiff: fix warning caused by previous commit FlatGeobuf: fixes for big-endian hosts. Tested on a PowerPC VM FlatGeobuf: avoid memleak when reading DateTime, and potential integer overflows on hostile input Add OGR_G_CreateGeometryFromEsriJson() and map it to SWIG FlatGeoBuf: fix potential nullptr dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18506 VICAR: support reading more data types and pixel organization (refs #1855) VICAR: fix reading of EOL labels with non BSQ organizations, and possible confusion with LBLSIZE of EOL overwriting the main one Move IEEE <--> VAX double conversion from DGN driver to CPL, so they can be reused by the VICAR driver VICAR: do not error out if the EOL keyword is missing. It is not mandatory VICAR (and RawRasterBand and VRTRawRasterBand): add support for VAX order for floating point data types VICAR: improvements in label reading, and report label in a json:VICAR metadata doma…

qgib added Bug Either a bug report, or a bug fix. Let's hope for the latter! High Priority Data Provider Related to specific vector, raster or mesh data providers Regression Something which used to work, but doesn't anymore labels May 25, 2019

This was referenced May 25, 2019

Changing feature values in the Attribute Table of a layer (geojson) doesn't save (after clicking save button), old values appear after reloading #29042

Closed

Feature edits dont save - temp layer #29838

Closed

rouault closed this as completed in OSGeo/gdal@6fb4c77 Sep 18, 2019

rouault added a commit to OSGeo/gdal that referenced this issue Sep 18, 2019

Merge pull request #1846 from rouault/fix_geojson_rewriting_on_windows

27690b4

GeoJSON driver: use VSIOverwriteFile() to fix update of file on Windows (fixes qgis/QGIS#28580)

rouault added a commit to OSGeo/gdal that referenced this issue Sep 18, 2019

GeoJSON driver: use VSIOverwriteFile() to fix update of file on Windo…

dd7abfb

…ws (fixes qgis/QGIS#28580)

rouault added a commit to OSGeo/gdal that referenced this issue Sep 18, 2019

GeoJSON driver: use VSIOverwriteFile() to fix update of file on Windo…

cbd05da

…ws (fixes qgis/QGIS#28580)

lmdc45 mentioned this issue Feb 19, 2020

GeoJson datasources - unlink to file when checkout git branches #34560

Closed

Edits in GeoJson datasources are not saved anymore #28580

Edits in GeoJson datasources are not saved anymore #28580

Comments

qgib commented Dec 9, 2018

qgib commented Dec 10, 2018

qgib commented Dec 17, 2018

qgib commented Dec 17, 2018

qgib commented Dec 18, 2018

qgib commented Dec 20, 2018

qgib commented Feb 15, 2019

qgib commented Feb 16, 2019

qgib commented Feb 18, 2019

qgib commented Feb 21, 2019

qgib commented Feb 26, 2019

qgib commented Feb 26, 2019

qgib commented Apr 21, 2019

qgib commented Apr 22, 2019

qgib commented Apr 24, 2019

qgib commented May 24, 2019

lmdc45 commented Jun 10, 2019

timlinux commented Jul 9, 2019

aaliddell commented Jul 11, 2019

gioman commented Jul 11, 2019

rouault commented Jul 11, 2019 • edited Loading

aaliddell commented Jul 11, 2019

aaliddell commented Jul 12, 2019

rouault commented Jul 12, 2019

aaliddell commented Jul 12, 2019

rouault commented Jul 12, 2019 • edited Loading

aaliddell commented Jul 12, 2019

rouault commented Jul 12, 2019

aaliddell commented Jul 12, 2019

radumas commented Jul 23, 2019

lennoo commented Jul 25, 2019

Mike-Honey commented Aug 16, 2019 • edited Loading

FireByTrial commented Oct 23, 2019 • edited Loading

gioman commented Oct 23, 2019

rouault commented Oct 23, 2019

FireByTrial commented Oct 23, 2019

lmdc45 commented Nov 18, 2019

rouault commented Jul 11, 2019 •

edited

Loading

rouault commented Jul 12, 2019 •

edited

Loading

Mike-Honey commented Aug 16, 2019 •

edited

Loading

FireByTrial commented Oct 23, 2019 •

edited

Loading