-
Notifications
You must be signed in to change notification settings - Fork 262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NC4_get_vars(): fix out-of-bounds write with unlimited dimension #1442
NC4_get_vars(): fix out-of-bounds write with unlimited dimension #1442
Conversation
This fixes an issue hit by GDAL, and that is found in netcdf 4.6.3 and 4.7.0 git bisect pointed the problem to have started with ``` 77ab979 is the first bad commit commit 77ab979 Author: Ed Hartnett <edwardjameshartnett@gmail.com> Date: Sat Jun 16 09:58:48 2018 -0600 using get_vars but not put_vars :040000 040000 8611e77aaea4f130ca769a852e2c3b26373a78fa fc9ffd1d13d8f5db5ab888044e0cec4def26eba1 M libsrc4 ``` where nc_get_vara_double() started using nc4_get_vars() underneath. It turns out that nc4_get_vars() was buggy in the situation exercised by GDAL. This can be reproduced with the following simple test case: ``` int main() { int status; int cdfid = -1; int first_dim; int varid; int other_var; size_t anStart[NC_MAX_DIMS]; size_t anCount[NC_MAX_DIMS]; double* val = (double*)calloc(3, sizeof(double)); status = nc_create("foo.nc", NC_NETCDF4, &cdfid); assert( status == NC_NOERR ); status = nc_def_dim(cdfid, "unlimited_dim", NC_UNLIMITED, &first_dim); assert( status == NC_NOERR ); status = nc_def_var(cdfid, "my_var", NC_DOUBLE, 1, &first_dim, &varid); assert( status == NC_NOERR ); status = nc_def_var(cdfid, "other_var", NC_DOUBLE, 1, &first_dim, &other_var); assert( status == NC_NOERR ); status = nc_enddef(cdfid); assert( status == NC_NOERR ); /* Write 3 elements to set the size of the unlimited dim to 3 */ anStart[0] = 0; anCount[0] = 3; status = nc_put_vara_double(cdfid, other_var, anStart, anCount, val); assert( status == NC_NOERR ); /* Read 2 elements starting with index=1 */ anStart[0] = 1; anCount[0] = 2; status = nc_get_vara_double(cdfid, varid, anStart, anCount, val); assert( status == NC_NOERR ); status = nc_close(cdfid); assert( status == NC_NOERR ); free(val); return 0; } ``` Running it under Valgrind without this patch leads to ``` ==19637== ==19637== Invalid write of size 8 ==19637== at 0x4C326CB: memcpy@@GLIBC_2.14 (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==19637== by 0x4EDBE3D: NC4_get_vars (hdf5var.c:2131) ==19637== by 0x4EDA24C: NC4_get_vara (hdf5var.c:1342) ==19637== by 0x4E68878: NC_get_vara (dvarget.c:104) ==19637== by 0x4E69FDB: nc_get_vara_double (dvarget.c:815) ==19637== by 0x400C08: main (in /home/even/netcdf-c/build/test) ==19637== Address 0xb70e3e8 is 8 bytes before a block of size 24 alloc'd ==19637== at 0x4C2FB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==19637== by 0x4009E8: main (in /home/even/netcdf-c/build/test) ==19637== ```
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you also incorporate the test code to confirm this fix?
Done per 0c7be1d |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome but don't forget the Makefile.am. The test must be added there as well.
….7.0 with unlimited dimensions (refs Unidata/netcdf-c#1442)
@@ -33,7 +33,7 @@ tst_vars2 tst_files5 tst_files6 tst_sync tst_h_scalar tst_rename \ | |||
tst_rename2 tst_rename3 tst_h5_endians tst_atts_string_rewrite \ | |||
tst_hdf5_file_compat tst_fill_attr_vanish tst_rehash tst_filterparser \ | |||
tst_bug324 tst_types tst_atts3 tst_put_vars tst_elatefill tst_udf \ | |||
tst_put_vars_two_unlim_dim | |||
tst_put_vars_two_unlim_dim tst_bug1442 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Thanks! |
Abhishek Arya (1): Add OSS-Fuzz fuzzing badge (#1912) Alan Thomas (5): DXF: Generate correct HATCH boundary elliptical arc segments for certain start/end angles DXF: Skip "Embedded Object" sections in 2018 version DXFs DXF: Specify maximum gap between interpolated curve points DXF writer: Re-add some header and trailer elements (fixes #1213) DXF: Correct handling of "off" and "frozen" layers in blocks Alexander Bruy (2): show warning when running gdal2tiles against non-Byte input (fix #1956) add note about non-Byte inputa to the gdal2tiles documentation Andrew Bell (1): libgeotiff: set UOMLength from GeogLinearUnits (for geocentic CRS) (#1596) Andrew Harvey (1): Doc: update ogr2ogr -explodecollections documentation with order relative to -sql (#2159) Anton Kovalev (1): Add PICREP attribute to LNDMRK object class Anton Wagensonner (1): gdal_edit.py: add -a_ulurll switch Anurag Kulshrestha (1): Docs: Fix type sizes in cosar.rst (#2089) Asger Skovbo Petersen (1): flatgeobuf.rst: Add ogr2ogr example (#1843) Bas Couwenberg (1): Fix spelling errors. Björn Harrtell (29): Addd read/write vector FlatGeobuf driver (#1742) Improve flatgeobuf.rst formatting and a short introduction (#1841) FlatGeobuf: Filtering fixes (#1844) FlatGeobuf: Rework/improve error handling FlatGeobuf: Fix VERIFY_BUFFERS open option handling FlatGeobuf: Buffer size bound checks No subtract for header max size Remove redundant const FlatGeobuf: Fixes for index handling FlatGeobuf: WKT roundtrip tests and nullptr validation (#1885) FlatGeobuf: Fix Z/M handling FlatGeobuf: Fixes for weaknesses detected by oss-fuzz (#1930) FlatGeobuf: Avoid early allocation with risk of leak FlatGeobuf: Additional size checks (#1944) FlatGeobuf: Write in two passes if index requested (#1967) (fixes #1964) FlatGeobuf: Read feature indices on demand (#1973) (fixes #1966) FlatGeobuf: Optimize reads for index search (#1993) Allow Travis sanitize build to work on plain Ubuntu 16.04 (#2061) FlatGeobuf: implement spec v2 (#2051) FlatGeobuf: Serialization cleanup (#2073) FlatGeobuf: Simplify memory management with unique_ptr (#2082) FlatGeobuf: Fixes for files with no index WFS: Support FlatGeobuf as WFS outputformat (#2135) FlatGeobuf: spec v3 (#2200) FlatGeobuf: Minor optimization to tree search FlatGeobuf: Advertize strings as UTF8 FlatGeobuf: Support mixed geometries (#2255) FlatGeobuf: Fix FID stability on spatial searches FlatGeobuf: add null check for readMultiPolygon (#2461) (fixes #2459) Blayne Chard (1): Docker: Allow https connections with curl Bojan Bizjak (1): Update WindowsLatin2 definition #1571 Chiara Marmo (3): Update FITS documentation for version 3.0 (#1598) Documentation: add FITS specific installation instructions in fits.rst. (#1646) Details on FITS implementation. Chris Mayo (7): configure: Run Autoconf 2.69 Enable Bash completions and control installation configure: Run autoreconf after bash-completion changes GDALmake.opt.in: silence datarootdir warning Doc: allow user full control over installation directory Python bindings: Fix SWIG warning of missing path for input file ogrsf_frmts/sqlite: simplify to one #include sqlite3.h Chris Tapley (1): ecwjp2 sdk 5.5 changes Cody Benkoski (1): DOCS: Updated broken links to point to correct pages (#2054) Craig de Stigter (4): Automated: Convert various comparisons into pytest.approx() calls Also use pytest.approx() where the tolerance is a variable tests: test cleanup for gcore/misc.py Fix test_tiff_write_157 failures due to faulty NaN comparisons Dan Baston (5): Format Virtual File Systems doc Add page for software using GDAL Update software links, remove dead entries Doc: Add documentation for vsis3 AWS_HTTPS option Doc: declare config options for virtual file systems Daniel Baston (3): Add ogrinfo doc .gitignore update Add ogr2ogr doc Daniel Brookes (1): Contour generator: fix SegmentMerger list iterator skipping and out of bounds error. (fixes #1670) (#1671) Daniel Lee (1): MSG driver doc: Update link (fixes #2205) (#2218) Daniel Morissette (1): Return BSB Dataset's PLY coordinates as a WKT POLYGON in a BSB_CUTLINE metadata DanielFEvans (1): Add missing documentation of -optim option (#2260) David Burken (1): Added search for proj library in lib64 directory. David Manthey (1): nitf/nitfimage.c: skip bad UDID data (fixes #1578) (#1586) David Morais Ferreira (1): Doc: Update gdal_fillnodata.rst (#2401) [skip travis] [skip appveyor] Dmitry Baryshnikov (10): ngw: Change endpoint in autotests CAD: Fix read ellipse and arc (fix #1886). CAD && SQLite: Removed unnecessary header include and changed strnlen to CPLStrnlen (fix #1920) WMS [AGS]: Remove unused parameters from url WMS [AGS]: Add unit test for ArcGIS Server URL with additional parameters NGW: Add support for additional raster types and QGIS styles. Add more server side attribute filters. Fix driver documentation. Add more unit tests. NGW: Fix dataset access mode MITAB: cleanup management of update flag (fixes #2170) MITAB: Add friendly layer name (description) support. NGW: Add support for layers geometry types with Z Dominik Nussbaumer (2): Add RDB raster driver to read *.mpx RDB 2 files (#1538) fixes warning reported by cppcheck Ed Leaver (1): configure: strip -L/usr/lib and similar from netCDF, MySQL, GEOS and SFCGAL lib path (#2395) Eli L Adam (5): Update gdal_viewshed.rst Docs: fix typo in virtual_file_systems.rst (#2025) Doc: use built_in_by_default and build_dependencies directive (#2055) [skip travis] [skip appveyor] Doc: use built_in_by_default and build_dependencies directive (#2058) [skip travis] [skip appveyor] Use built_in_by_default and build_dependencies directive for vector drivers (#2083) [skip travis] [skip appveyor] Emmanuel Arias (1): Doc: Fix typo on gdalwarp.rst (#2432) [skip travis] [skip appveyor] Enrico Weigelt, metux IT consult (1): Fix permissions of non-executable files Even Rouault (1308): gcore/gdal_version.h.in: set to 2.6.0dev Add docker/build-all.sh docker/build-all.sh: shellcheck fixes Drop executable permission from files where it is not appropriate Drop executable permission from files where it is not appropriate, and set it correctly on .py files Move include armadillo in a separate header Docker alpine-normal: add hdf4 and libkea Docker alpine-normal: fix previous commit that disabled the PDF driver docker/util.sh: add a --push option, tag builder image, and cleanup old -latest images Docker: improve layering docker/ubuntu-full/Dockerfile: parametrize versions ogr2ogr: emit better error message when using -f VRT ci/travis/common_install.sh: use http:// for now (see https://trac.osgeo.org/osgeo/ticket/2296) Docker: refresh proj-datumgrid-latest when it changes Docker: add ccache capability Python bindings: build modules in parallel Docker: use ccache for Python bindings Docker: add PROJ_DB_CACHE_DIR Docker: completely sync the remote cache after build [ci skip] Docker: tidy build scripts [ci skip] Docker: avoid hardcoding docker0 interface address [ci skip] Docker: bind rsync daemon to Docker inet only [ci skip] gdal_contour: remove explicit width/precision=12/3 of the elev field (fixes #1487) docker/alpine-ultrasmall/build.sh: remove erroneous python testing [ci skip] GTiff: do not generate a TIFFTAG_GDAL_METADATA with color interpretation information for JPEG YCbCr compression OGR_PDS: fix heap-buffer-overflow in error code path. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14512. Credit to OSS Fuzz GTiff driver: ensure that main dataset and overviews share the same lock, so as to avoid crashing concurrent access (fixes #1488) Docker: use more docker-ish way to implement our rsync daemon [ci skip] osr_esri.py: accept PROJ master result Change mentions of GDAL 2.5 to GDAL 3.0 gcore/gdal_version.h.in: set to 3.1.0dev GeoJSON: report 3D layer geometry types (fixes #1495) GPKG, MBTiles, PostgisRaster ddrivers: share lock of overview dataset with parent dataset (refs #1488) Add test case for refs #1488 NEWS: refresh [ci skip] GTiff doc: clarification for subdataset creation (refs #1458) HDF5: avoid error report of the HDF5 library when _FillValue attribute is missing Make GH5_FetchAttribute(CPLString) more robust and able to cope with variable-length string Test previous commit GDALReprojectImage(): properly take into account source/target alpha bands GTiff: prefix all member variables by m_ PostgreSQL: do not attempt to create VARCHAR(n) columns with n >= 10485760 (fixes #1508) ISIS3: fix parsing of lists in JSon metadata, and quote string items when writing lists when needed (fixes #1510) GTiff: decrease size of GTiffDataset structure GPKG: change default value of OGR_GPKG_FOREIGN_KEY_CHECK to NO, so as to avoid issues in downstream software GTiff: reset m_bGeoTIFFInfoChanged flag in FlushDirectory() GTiff: improve performance of internal overview creation (fixes #1442) Python bindings: honour gdal.UseExceptions() in numpy related methods (gdalnumeric module) (fixes #1515) GTiff: remove support for libtiff 3.X NITF: fix parsing of SENSRB TRE (fixes #1520) Add forgotten file in previous commit (refs #1520) formats_list.html: fix BAG capabilities (fixes #1522) [ci skip] Shape: better deal with empty .shp+.shx and SHAPE_RESTORE_SHX (fixes #1525) SHPRestoreSHX(): fix crash when .shx file cannot be created (and a memleak) doc/index.dox: link to 3.0.0 [ci skip] GXF: avoid closing the file pointer before being sure this is a GXF driver, otherwise this can prevent the opening of some raw format files (fixes #1521) ENVI: fix potential use of invalid pointer on some unusual std::string implementations (fixes #1527) GTiff: make -co NUM_THREADS produce reproducible output GTiff: factor some code of DISCARD_LSB functionality and improve test coverage tiff_write.py: test error cases of DISCARD_LSB GTiff: fix build ENVI: add read support for reading GCPs (#1528), and fix off-by-one offset on line,pixel on reading GCP GPKG: fix memory leak in case of corrupted database. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14683. Credit to OSS Fuzz EEDA: fix startTime / endTime comparisons that were incomplete in refs #1506 Fix previous commit (refs #1506) GTiff: fix variable name Add API documentation Add the overhaul doc structure Tweak theme Add gdal_formats.csv ogr_formats.csv WaSP: on creation, make sure the layer geometry type set on the feature definition is wkbLineString25D Travis: use ubuntugis-stable PPA for Trusty since they are no longer in unstable EPSG:4326 to EPSG:3857: fix invalid output with some input coordinates (3.0 regression) Add virtual_file_systems.rst and remove the overloaded highlight_language in conf.py Add raster_data_model.rst Add source/user/vector_data_model.rst Extent doc structure with placeholder pages Add placeholder for programs Port the raster API tutorial Port te raster driver tutorial Add GDAL Warp API tutorial Add OGR API tutorial Add OGR driver tutorial Add GNM API tutorial Add GNM architecture Add OSR API tutorial Add OGR SQL dialect page Add SQL SQLite dialect Add indexes to driver pages Add GDAL VRT driver page Fix links Move eeda driver to raster, add missing GeoPackage raster driver Remove debug stuff Fix in gdaldem.rst Add gnmmanage Add gnmanalyse Add gdal_translate Add gdaladdo Fix redirect for raster format Tweak gdalinfo Add gdalwarp Add gdaltindex Fix index directive in ogrmerge Add gdalbuildvrt and tweak program index Add rgb2pct Add rgb2pct Add gdal_merge and gdal2tiles Add gdal_rasterize Add gdaltransform Add nearblack Add gdal_retile and gdal_grid Add gdal_proximity Add gdal_polygonize Add gdallocationinfo Add gdalsrsinfo Generate all man pages Fix GMLAS pages Geopackage: fix link to aspatial Fix warnings Programs: use include mechanism for common options Adjustments for Doxygen XML output and Sphinx Fix last remaining warnings and turn warning-as-error on Add missing raster formats Fixes in raster formats Fixes in raster formats GeoJSON: fix recognizing some documents with members sorted alphabetically (fixes #1537) Improve PDF output Avoid TOC to be included twice in HTML output, and add logo to PDF front page GTiff: fix multi-threaded compression bug introduced in 2a4297e669166 (fixes #1535, master only) Fix links between drivers and add missing pages Fix links Add raster driver properties Fix warnings in PDF generation and include auxiliary pages; reenable -W for all targets Add full doxygen HTML output Add vector driver capabilities Move access to raster and vector doc to upper level Make 'software using GDAL' available from 'About' Add link to PDF file Fix wrong indentation of a few raster drivers Fix Doxygen build Adjust pipeline to remove the temporary symbolic link configure: fix detection of libpq in a non-standard place (3.0 regression) (fixes #1542) gdal_polygonize.py: fix outputing to geojson without explicit -f switch (fixes #1533) Add and fill a shortname directive fix warning Add redirects MRF: fix crash on corrupted dataset. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14782. Credit to OSS Fuzz MRF: fix memory leak in error code path. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14787. Credit to OSS Fuzz GTiff/libtiff: move defered tile/strip offset/bytecount loading to libtiff libtiff: BigTIFF creation: write TileByteCounts/StripByteCounts tag with SHORT or LONG when possible GPKG: insert more accurate spatial extent in gpkg_contents Fix formatting of tables in DXF & PDF, fix links in X-Plane Python bindings: refresh doc Add Java and Python docs .azure-pipelines.yml: use wget from Docker image Internal libtiff: resync with upstream Fix old Doxygen build fix old Doxygen build with grid tutorial Port gdal_grid tutorial to Sphinx Commit forgotten file .azure-pipelines.yml: remove doc-sprint branch, and disable CNAME .azure-pipelines.yml: set CNAME to gdal.org /vsitar/: support >100 character file names (fixes #1559) Docker scripts: update to openjpeg 2.3.1 /vsitar/: fix 94ffaf551457a660c7cdc5b04b42334911d0f46c commit (refs #1559) Doc: add missing pages for MG4Lidar driver .azure-pipelines.yml: copy gdalicon.png to root to avoid breaking autotest .azure-pipelines.yml: copy gdalicon.png to root to avoid breaking autotest Documentation: remove old .html and .dox files gdalwarp: fix wrong axis order when using source/target CRS being a geographic3D CRS such as EPSG:4979, and with vertical shift grid application (fixes #1561, GDAL 3.0 regression) Doc: migrate RFCs from Trac libtiff: fix vulnerability of https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14894 (master only) Doc: fix docs of NGW driver (fixes #1573) Add file that should have gone with previous commit (refs #1573) Doc: change links to RFCs from Trac wiki to internal versions Fix typos, mostly in doc/ Doc: fix install-docs and install-man target, make 'man' target depend on Doxygen, and fix a RST issue Doc: fix typo Doc: fix more links from svn to github Add preliminary text for RFC 75: Multidimensional arrays [ci skip] Make 'gdal_translate foo.tif foo.tif.ovr -outsize 50% 50% -of GTiff' work Doc: adjustments for PDF output due to previous commit Wording: in the context of TIFF, use 'striped' instead of 'stripped' (fixes #1572) Docker images: update to python 3 (fixes #1532) [ci skip] libtiff: Fix vulnerability in 'D' (DeferStrileLoad) mode (master only) (fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14908) Doc: add redirect for gdal_vrttut.html [ci skip] GTiff: make WEBP_LEVEL to be honored in Create() mode (fixes #1594) Disable randomly failing test (refs #1594) libgeotiff/geo_normalize.h: adjust comments (refs #1595) Doc: add redirect for gdal_virtual_file_systems.html [ci skip] GTiff: fix yesterday regression in which compression parameters are ignored in COPY_SRC_OVERVIEWS=YES mode (refs #1594) GTiff writer: in COPY_SRC_OVERVIEWS=YES, interleave mask with imagery, and add leader/trailer to strile gdalcompare.py: take into account mask bands GTiff: optimize read of cloud-optimized geotiffs GTiff: when updating a COG, detect and warn when we break the optimizations /vsicurl/: avoid downloading one extra block when the end offset is just at a chunk boundary Tests: check efficient reading of COG files Doc: mention Docker images [ci skip] GTiff COG optimizations: replace STRILE_ by BLOCK_ GTiff: address review comments of #1600 OGRSpatialReference::GetAxis(): make it work with a compound CRS (fixes #1604) gdal_priv.h: remove decaration of never implemented GDALDefaultBuildOverviews() function gdalinfo --format: avoid crash on invalid XML content for creation options doc/source/conf.py: fix case of links to github [ci skip] doc/source/conf.py: fix canonical_url [ci skip] tiff_ovr.py: restore cleanup of temporary files GTiff: fix crash with multi-threaded compression and overviews (master only) GMLAS: fix typo in error message CPLConfigOptionSetter: only reset thread-locale value, not global one mask.py and tiff_ovr.py: use gdaltest.config_option() Add initial createcopy COG (Cloud Optimized GeoTIF) driver GeoJSONSeq: Open() successfully only if at least one feature is detected GeoJSONSeq: fix previous commit to unbreak test_http_6 test KML: set OAMS_TRADITIONAL_GIS_ORDER for SRS returned on returned layers GDALInvGeoTransform(): make it work with scale and rotation/skew coefficients of small absolute value (fixes #1615) rfc75: update typos raised by @drons COG: add reprojection support gdalwarp: accept output drivers with only CreateCopy() capabilities importFromWkt(): emit a CPLError() in case of import failure (refs #1623) netCDF: avoid buffer overflow. master only. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=15143. Credit to OSS Fuzz /vsis3/: for a long living file handle, refresh credentials coming from EC2/AIM (fixes #1593) Doc: fix a few examples in Doxygen comments Doc: tweak presentation of example Python bindings: remove dead code since we switched to 'new generation' Python bindings COG driver: add NONE as potential value for OVERVIEWS creation option Doc: update a version number Doc PDF: improve formatting Rework code to make cppcheck 1.72 happy (false positive) VRTAddBand() doc: add note about the return type (fixes #1637) COG doc: add precision about NUM_THREADS creation option COG doc: additional precision Internal libgeotiff: upgrade to libgeotiff 1.6.0dev to support OGC GeoTIFF 1.1 GTiff: add support for reading and writing OGC GeoTIFF 1.1 (requires libgeotiff 1.6 for writing part) Fix MSVC linking issue related to TransformWithOptionsCache class (fixes #1639) FileGDB: add instrumentation to display related datasets (not compiled by default) COG: always create temporary files as BigTIFF (fixes #1655) COG: fix and simplify logic for temporary file creation (fixes #1655) COG: force INTERLEAVE=PIXEL for tempoary overview generation of multi-band datasets GDALCreateReprojectionTransformerEx(): do not emit error if reverse transformation fails, and fix crash when trying to use null reverse transformation GTiffDataset::PushMetadataToPam(): early exit when PAM is disabled, to avoid error messages Warper: fix GDAL 2.3 regression in a situation wih source nodata value, multiple bands and nearest resampling where the logic to detect which source pixels are nodata was inverted (fixes #1656) Fix pyflakes issue of previous commit SWIG: add osr.SetPROJSearchPath(path) that can be used since setting PROJ_LIB from C# does no work (fixes #1647) Doc MVT: json formatting MVT: make CONF option accept a filename as well MVT doc: update with CONF option accepting a filename as well Python bindings: use fragments for 2 verbose typemaps Update my email address in lots of files Update my email address in lots of files Hopefully last email replacement VRT Python: also probe libpython3.Xm.so.1.0 (fixes #1660) Docker: install libpython3.6 for ubuntu-small and ubuntu-full (fixes #1660) /vsicurl/: allow 'Connection timed out' CURL errors as candidate for HTTP retry GDALCopyWholeRasterGetSwathSize(): fix potential int overflows for big values of GDAL_SWATH_SIZE or GDAL_CACHEMAX BAG: on reading, handle non-standard pixel-is-area convention. On writing, add a CORNER_POINTS_EXTEND_HALF_PIXEL=YES creation option to use that convention (fixes #1643) BAG doc: document CORNER_POINTS_EXTEND_HALF_PIXEL=YES creation option (refs #1643) Docker: update alpine images to 3.10 [ci skip] /vsimem/: make Rename() error if destination file is not in /vsimem/ BAG: update tests related to #1643 Add a CPLCondTimedWait() /vsizip/ writing: in ZIP64 mode, also advertise 45 as the version in the central directory (avoids a warning from 'zip' utility) Shapefile driver: add read/creation/update support for .shz and .shp.zip NTv2: add support for the Canadian NAD83(CRSR)v7 / NAD83v70VG.gvb velocity grid /vsicurl/: GetFileSize(): when HEAD request does not return Content-Length header, retry with GET MVT: reduce memory usage when processing big geometries (fixes #1673) Python bindings: update to SWIG 3.0.12 to have better error message (fixes #1677) Shape: fix crash when creating a layer with a wkbNone geometry type but a SRS (3.0 regression) MapInfo TAB: don't left truncate numeric values in the .dat when the field formatting is incompatible, but error out (fixes #1636) gdallocationinfo: emit verbose error when dataset canot be opened (fixes #1684) gdallocationinfo and gdaltransform: print a hint when values are expected from the command line and stdin is an interactive terminal (refs #1684) GeoJSON: avoid SetFeature() to repeat first feature when looping over features (fixes #1687) Fix pyflakes issue (refs #1687) Shapefile: launder layer name to get Windows compatible filename HOWTO-RELEASE: ./swig/python/README.txt -> ./swig/python/README.rst [ci skip] COG: for mask band overview computation, use same resampling kernel as other bands autotest: remove use of deprecate message argument of pytest.raises() (fixes #1689) autotest/requirements.txt: avoid too recent versions of pytest and more-itertools incompatible of Python2.7 netCDF: correctly parse grid_mapping attribute in expanded form GTiff: improve performance of generation of internal overviews COG: for mask band overview computation, try to use the same logic as for alpha band: that is 0 values in the source mask are ignored PG: add support for PostgreSQL 12 (fixes #1692) netCDF: refactor SetProjectionFromVar a bit COG: use correct type for RES creation option XML validate_cloud_optimized_geotiff.py: emit error when KNOWN_INCOMPATIBLE_EDITION=YES is found GTiff: set a LAYOUT=COG metadata item in the IMAGE_STRUCTURE metadata domain of the dataset when the hidden feature declarations typical of a COG file are found XLSX driver: add support for .xlsm extension Doc: advertize GDAL 3.0.1 release GTIFF / COG: avoid crash when trying to write dataset with band count invalid for WEBP (master only) cog.py: fix previous commit Doc: add Conda badge Doc: add link to 2.4.2 Map OSRGetAxesCount() to C API and SWIG bindings epsg_tr.py: refresh with PROJ 6 capabilities geotiff.cpp: remove useless assert Add explicit keywords (raised by cppcheck) Fix cppcheck warnings about calling a virtual method in a constructor/destructor Doc: fix PDF build GTiff: fix memory leak with -co APPEND_SUBDATASET=YES gt_overview.cpp: remove obsolete libtiff < 4 test Python bindings: add compatibility with SWIG 4.0 (based on patch by @perestoronin, fixes #1702) tile_extent_from_raster.py: sample script to generate the extent of each raster tile in a overview as a vector layer Internal libtiff: resync with upstream tile_extent_from_raster.py: fix coordinate precision GTiff: fix error message for NBITS != 16 and Float32 PAMRasterBand: add presence flag for Offset and SetScale so that GetOffset()/GetScale() properly reports if they are defined in PAM RFC 75 text: update with implementation work s57objectclasses.csv: add missing TXTDSC attribute for DRYDOC class (fixes #1723) gdalwarp: adjust nodata values, passed with -srcnodata/-dstnodata, and close to FLT_MAX to exactly it (fixes #1724) appveyor.yml: force installation of SWIG 3.0.12 since the C# bindings aren't compatible of SWIG 4.0 (fixes #1726) test_osr_set_proj_search_paths.cpp: augment to test threads ogr_proj_p.cpp: fix code comment [ci skip] GTiff: ensure GDAL PROJ context is used for all libgeotiff functions (requires internal libgeotiff / libgeotiff 1.6) ISSUE_TEMPLATE.md: add note about expectations regarding bug fixing, inspired from QGIS [ci skip] HDF5 and netCDF: fix crash when reading attributes of type string of variable length with NULL values Avoid crash on test_netcdf_dimension_labels_with_null with too old netCDF version netCDF: very partial workaround for an issue in libnetcdf 4.6.3 and 4.7.0 with unlimited dimensions (refs https://github.com/Unidata/netcdf-c/pull/1442) Replace mentions of GDAL 2.5 by 3.0 cpl_safemaths.hpp: safe + and * for GUInt64 Add CPLJSonStreamingWriter class RFC 75: initial support for multidimensional arrays Travis gcc52_stdcpp14_sanitize: upgrade numpy version RFC75: VRT multidimensional support RFC75: add gdalmdimtranslate gdalmdiminfo: uses reference to dimensions already displayed RFC75: add documentation HDF5 test: test file with recursive groups HDF5ImageRasterBand::IReadBlock(): fix memleak in error code path gdalmdiminfo: robustness fix USGSDEM: avoid int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=15715. Credit to OSS Fuzz gdalmdiminfo: display subgroups at end of group gdalmdimtranslate: fix target name array and remove warning when copying from nested array netCDF multidim: fix performance issue GDALMDArray::CopyFrom(): fix memory leak NCDFGetCoordAndBoundVarFullNames(): fix memory leak (non RFC75 related) NCDFWriteSRSVariable(): handle situation where the grid mapping variable already exists (needed for RFC75) netcdf_multidim.py: test writing several georeferenced variables HDF5 type detection: only detect complex data type if the component names start with r/R and i/I (refinement of refs #359) HDF5: add multidimensional read support GDALDriver::CreateMultiDimensional(): set description and driver on newly created dataset gdalmdiminfo: output driver name HDF5: use free() on non-Windows platforms when H5free_memory() not available HDF5: compatibility trick for older hdf5 library GDALDefaultOverviews::BuildOverviews(): fix typo in detection of 1x1 overview (fixes #1730) RFC75: update status and voting history PG: be more restrictive when deducing non-nullability of columns in SQL result layers (fixes #1734) DAAS: accept 4-band RGBA PNG response even when a single band is requested LIBKML driver: make it accept /vsigzip/foo.kml.gz files (fixes #1743) jp2metadata.py: make test_jp2metadata_4 work with latest PROJ master Fix previous commit ci: temporarily freeze spatialite to a particular revision since HEAD builds are broken if rttopo is not enabled ogr_ngw.py: disabled for now (refs #1746) ngw.py: disabled for now (refs #1746) Revert "ogr_ngw.py: disabled for now (refs #1746)" Revert "ngw.py: disabled for now (refs #1746)" osr_metacrs.py: force cleanup of objects to please ASAN /vsicrypt/: fix memleak in error code path vsicrypt.py: fix memleak Travis CI: port gcc52_stdcpp14_sanitize job to Xenial since Precise is discontinued FITS: fix memory leak Fix failure on test_tiff_read_online_2 hdf4_read.py: fix URL and add test for HDF EOS SWATH HDF5 multidim: fix data type conversion with compound data types HDF4: fix retrieval of non-string swath and grid attributes HDF4: fix GR support for non-Byte rasters, remove broken attribure reading, and fixes related to color table HDF4: add multidimensional read support (RFC 75) appveyor.yml: use VS2017 for x64 builds (fixes #1747) Internal libtiff: resync with upstream GDALMDArray::AsClassicDataset(): add support for 1D arrays HDF4 multidim: add simple non-GDAL SDS test, and fix for unlimited dimension OGRExpatRealloc(): fix double-free when size to allocate is above the default 10MB threshold. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16178. Credit to OSS Fuzz Move GDALAttributeString to core Degrib: add support for ReadGrib2Record() wit Grib_Data == nullptr core: add GDALAttributeNumeric class nmake.opt: mention that SETARGV should be wsetargv.obj (refs #1749) GRIB: add support for multidim API Core: add GDALDimensionWeakIndexingVar and GDALMDArrayRegularlySpaced classes, and use them in GRIB and HDF4 drivers PDF: fix error message [ci skip] Skip test_ogr_shape_115_shp_zip() on Travis Mac due to super frequent hangs appveyor.yml: re-enable VS2015 builds (fixes #1747) netCDF: fix warning for 32 bit mode netCDF 1.8 SF writing: remove unused variable NGW: fix warning for 32 bit mode PCIDSK SDK: remove use of deprecated function fuzzers/build_google_oss_fuzzers.sh: use PDF: avoid floating-point division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16327. Credit to OSS Fuzz Fix -Wimplicit-int-float-conversion warnings Internal libpng: upgrade to 1.2.59 multidim: add 'final' keywords and a few fixes for issues raised by cppcheck frmts: add 'final' whereever possible, and otherwise add explicit CPL_NON_FINAL ogrsf_frmts: add 'final' whereever possible, and otherwise add explicit CPL_NON_FINAL Replace a few catch of bad_alloc by exception to handle the case of 32 bit trying to allocate more than 4GB of memory and thus throwing a length_error exception. Found by OSS Fuzz third_party/LercLib/Lerc.cpp: resync with upstream (Android related fix) WMS: IReadBlock(): limit number of tiles downloaded at once libopencad: CADBuffer: replace m_guard by m_nSize to avoid pointer wrap around on 32 bit platforms on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16388. Credit to OSS Fuzz DGN: avoid size_t overflow / illegal memory access. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16393. Credit to OSS Fuzz VSIIngestFile(): avoid attempting to allocate too much RAM when erroneously opening a directory. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16394. Credit to OSS Fuzz libopencad: disable CRC and signature checks on -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION libopencad: CreateFileMap(): avoid infinite loop with section of size 0. Credit to OSS Fuzz gcore/gdalmultidim.cpp: fix use of moved variable (spotted by cppcheck) Fix a few (non-bug) cppcheck warnings OJPEGReadBufferFill(): avoid very long processing time on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16400. master only NAS: avoid long processing time on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16248. Credit to OSS Fuzz PDF: fix nullptr dereference on corrupted file. Credit to OSS Fuzz CPLStrtodDelim(): rework to avoid false positive from Coverity. CID 1404088 ComputeSourceWindowStartingFromSource: avoid mostly false postive from Coverity. CID 1404049 gdal_grid_lib: avoid mostly false postive from Coverity. CID 1404022 and 1404158 gdaldem_bin.cpp: avoid Coverity warning about tainted data. CID 1404163 gdalmdimtranslate_lib.cpp: fix a few Coverity warnings. CID 1404020, 1404147, 1404156, 1404198, 1404240 gdaltransform: silence false positive Coverity warning. CID 1214439 GDALWarp(): fix memleak in error code path. CID 1404058 gnmmanage: silence false positive Coverity warning. CID 1404019 GDALNearblack(): fix memleak if psOptionsIn==null. CID 1404128 ogrlineref: silence false positive Coverity warning. CID 1404238 blx.c: silence false positive Coverity warning. CID 1214445 blx.c: const correctness fixes DAAS: fix Coverity false positives. CID 1404079, 1404179, 1404139 FIT: fix Coverity false positive. CID 1404192 GeoRasterDataset::JP2_CopyDirect(): fix false-positive about memleak. CID 1404200 GeoRasterWrapper::LoadNoDataValues(): avoid memory corruption if method was called several times (which should not happen). CID 1404053 GRIB: avoid Coverity false positive. CID 1404168 TIFFClientOpen(): fix memory leak if one of the required callbacks is not provided. Fixed Coverity GDAL CID 1404110 hdf5multidim.cpp: various Coverity fixes. CID 1404104, 1404106, 1404194, 1404248 netcdfmultidim.cpp: various Coverity fixes. CID 1404093, 1404132, 1404178 netcdfsgwriterutil.cpp: fix Coverity Scan CID 1404035, 1404051, 1404052, 1404086, 1404162, 1404214, 1404217, 1404250 PDF composer from XML: fix wrong use of SetupVectorGeoreferencing(). Spotted by Coverity CID 1404081, 1404166. Also fixes 1404097 (false positive) PDF: silence Coverity false positive. CID 1404067 More uses of const reference as suggested by cppcheck CPLPopFileFinderInternal(): fix cppcheck false positive tiledb: initialize member variable in initializer list as suggested by cppcheck VFK: use a faster implementation of VFKDataBlockSQLite::LoadGeometryPolygon() Internal libtiff: resync with upstream. Includes a security fix for 32-bit builds GTiff: fix buffer overflow in DiscardLsb on 32-bit data type (master only) PDF read vector: avoid floating point division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16419 gdal_contour: silence Coverity warnings. Fixes 1404045 PDF: silence Coverity warning. Fixes 1404221 PDS: silence Coverity warning. Fixes 1404218 PostgisRaster: silence Coverity warning. Fixes 1404244 RIK: check return code of uncompress() method. CID 1404212 SAFE: avoid potential use-after-free. CID 1404037 and 1404140 Terragen: avoid potential use-after-free on error code path in write_header(). CID 1404060 vrtmultidim: avoid false positive Coverity warning. CID 1404072 GDALGCPsToGeoTransform(): silence Coverity warning. CID 1404056 GDALGeneralCmdLineProcessor(): fix double free with --optfile (CID 1404146) Python bindings multidimensional API: fixes buffer overflow when wrong arguments are provided to Array.Read() gdaljp2metadatagenerator.cpp: workaround to avoid ASAN issues when linking against filegdbAPI CTable2Dataset::SetGeoTransform(): fix read buffer overflow from stack Fix memory leak in GDALGeneralCmdLineProcessor() introduced in 04892bffec5bd469547d3562715882c080547392 configure: trick so that LD_SHARED test works with sanitizer flags with clang Add cpl_error_internal.h with logic with error accumulator (avoids a false positive UndefinedBehaviour warning about a function pointer, apparently due to the error handler being in the binary and the call site in the library) Travis sanitize: do fail if a test crashes on ASAN PDF: avoid nullptr dereference on corrupted file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16438. Credit to OSS Fuzz PDF: avoid floating point division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16439. Credit to OSS Fuzz Internal libtiff: fix integer overflow potentially causing write heap buffer overflow, especially on 32 bit builds. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16443. Credit to OSS Fuzz gdaldataset.cpp: silence false positive warnings. CID 1404117 and 1404177 gdaldriver.cpp: silence false positive warning. CID 1404114 gdaljp2abstractdataset.cpp: silence false positive warning. CID 1404223 gdalmultidim.cpp: fix multiple coverity warnings. CID 1404023, 1404040, 1404085, 1404094, 1404164, 1404181, 1404235 gnm.h: silence Coverity warning. CID 1404225 gml2ogrgeometry.cpp: silence Coverity warning. CID 1404036 ogr_geocoding.cpp: silence Coverity warning. CID 1404222 ograpispy.cpp: add move assignment operator. CID 1404059 ogrct.cpp: silence Coverity warning. CID 1404112 ogrgeometryfactory.cpp: silence Coverity warning. CID 1404143 ogrpgeogeometry.cpp: silence Coverity warning. CID 1404087 dgnwrite.cpp: silence Coverity warning. CID 1404084 ogredigeodatasource.cpp: silence Coverity warning. CID 1404173 ogrelasticdatasource.cpp: silence Coverity warnings. CID 1404071, 1404074, 1404111, 1404129, 1404239 FGdbLayer: silence Coverity warnings. CID 1404122, 1404191 VSIStatExL(): fix Coverity warning. CID 1404203 geoconcept.c: attempt at fixing Coverity CID 1404157 libjson/json_util.c: add missing cast. CID 1404169 ogrgeojsonseqdriver.cpp: validate config option value. CID 1404165 gmlreader.cpp: fix false positive warning. CID 1404205 ogrgmldatasource.cpp: avoid false positive CID 1404199 ogrgmlasreader.cpp: silence Coverity warning. CID 1404055 ogrgeojsonseqdriver.cpp: fix recent 19d7faa223cc9a5c1eb15b3e29f67719e0df9fd1 breakage ogrgmlasschemaanalyzer.cpp: silence Coverity warnings. CID 1404031 and 1404137 ogrgeopackagedatasource: silence Coverity warning. CID 1404202 LIBKML: fix potential memory leak. CID 1404148 ogrmemlayer.cpp: avoid assignment of overlapping memory. CID 1404061, 1404141, 1404152 MITAB: fix potential double-free. CID 1404224 MITAB: avoid potential nullptr deref. CID 1404174 nasreader.cpp: avoid warning about potential nullptr deref. CID 1404116 filegdbtable.cpp: silence Coverity warnings. CID 1404102, 1404135 ogrpgtablelayer.cpp: avoid Coverity warning. CID 1404127 ogrshapedatasource.cpp: avoid Coverity false positive warning. CID 1404153 SQLite: avoid false positive Coverity warnings. CID 1404092, 1404229, 1404182 VFK: add move assignment operator to VFKProperty for (micro) better performance. CID 1404215 cpl_http.cpp: silence Coverity warnings. CID 1404161, 1404206 cpl_json.cpp: add move assignment operator. CID 1404096 cpl_json.cpp: fix memory leak introduced in previous commit Internal libtiff: resync with upstream GTiff: add explicit error message when trying to create a too big tiled TIFF file (refs #1786) PDF: fix memleak on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16501. Credit to OSS Fuzz GRIB1: avoid excessive memory allocation attempt and potential crash on allocation failure. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16452. Credit to OSS Fuzz AVCE00: fix alternance from sequential to per-FID reading. Also fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16471. Fix or tag as allowed all uses of unsigned integer overflow, add add -fsanitize=unsigned-integer-overflow to the Travis sanitize configuration CPLStringList: add move assignment operator. CID 1404207 cpl_vsil.cpp: silence false positive warning. CID 1404216 cpl_vsil_az.cpp: silence warning. CID 1404064 cpl_vsil_crypt.cpp: silence warning. CID 1404090 cpl_vsil_curl.cpp: silence warning. CID 1404227 cpl_vsil_gzip.cpp: silence false positive warning. CID 1404220 cpl_vsil_s3.cpp: silence warnings. CID 1404041, 1404062, 1404095 cpl_vsil_swift.cpp: silence warning. CID 1404125 Shape: try to better deal with polygons with parts touching on an edge (which is illegal simple features) (fixes #1787) cpl_vsil_crypt.cpp: catch potential exception. CID 1404090 nasreader.cpp: silence Coverity warning. CID 1404116 PDF: avoid opening too highly corrupted files. Should fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16522 FITS: fix memory leaks PCRaster / libcsf: fix many Coverity warnings about assignment of overlapping memory PDF: fix potential heap buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16546. Credit to OSS Fuzz OGRSpatialReference::SetFromUserInput(): add capability to import PROJJSON PDF: fix potential nullptr deref on corrupted file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16558. Credit to OSS Fuzz BSB: avoid to use unsigned int overflow OSR: add ExportToPROJJSON() (PROJ >= 6.2) RMF: avoid unsigned int overflow on corrupted file degrib: avoid (harmless) unsigned integer overflows ERS: avoid (harmless) unsigned integer overflow Internal libtiff: avoid (harmless) unsigned integer overflow BSB: avoid harmless unsigned integer overflow on corrupted files degrib: avoid harmless unsigned integer overflow on corrupted files degrib: avoid harmless unsigned integer overflow on corrupted files EXIF reader: avoid harmless unsigned integer overflow PCIDSK: avoid harmless unsigned integer overflow PCIDSK: avoid harmless unsigned integer overflow GRIB: avoid potential unsigned integer overflow PCIDSK: avoid harmless unsigned integer overflow PCIDSK: avoid harmless unsigned integer overflow PCIDSK: avoid harmless unsigned integer overflow /vsigzip/: avoid harmless unsigned integer overflow BMP: compute file offsets on vsi_l_offset to avoid unsigned integer overflow Doc: mention PROJJSON format for gdalsrsinfo NWT: avoid harmless unsigned integer overflow NITF: avoid unsigned integer overflow GDALGeneralCmdLineProcessor(): fix master regression regarding display of the type of driver NWT: fix compilation issue on machines where char is signed, introduced by recent master commit /vsitar/: accept space as end of field terminator GRIB: fix MSVC warning introduced in yesterday commit /vsigzip/: avoid harmless unsigned integer overflow Internal libtiff: avoid unsigned integer overflow LercLib: avoid big memory allocation attempt on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16595. Credit to OSS Fuzz Support API-level precision/round for geometry WKT (#1797) (fixes #1174) Internal libtiff: avoid unsigned integer overflow third_party/LercLib: resync with latest upstream suggestions Doc: add mostly empty template for SOSI driver gdalexif: avoid unsigned integer overflow Internal libtiff: resync with upstream /vsigzip/: fix harmless unsigned integer overflow test_ogr_sql_sqlite_26: fix issue with GEOS head OpenFileGDB: ignore unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16629 netCDF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16631 Internal libtiff / tif_thunder: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16632 /vsitar/: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16633 Internal libtiff / tif_ojpeg: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16635 BitStuffer2.h: avoid harmless unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16638 ERS: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16640 libopencad: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16626 Internal libtiff / EstimateStripByteCounts(): avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16643 HFA: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16644 RawDataset / VSISubFileHandle: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16645 RIK: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16646 HFA: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16648 SEGY: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16649 /vsiwebdfs/: avoid harmless unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16650 MSGN: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16652 Internal libtiff / TIFFFillStrip: avoid harmless unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16653 VSISubFileFilesystemHandler::Stat(): avoid unsigned int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16655 NTF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16656 VSISubFileFilesystemHandler::Open(): avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16658 L1B: avoid reporting negative dataset height. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16663 Selafin: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16666 libopencad: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16672 RawRasterBand::AccessLine(): avoid unsigned int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16673 libtiff / OJPEGWriteHeaderInfo(): avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16683 Internal libtiff / _TIFFPartialReadStripArray: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16685 RAW: avoid unsigned integer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16686 NTF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16691 RPFFOC: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16695 PCIDSK SDK: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16698 OpenFileGDB: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16713 LercLib: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16714 OpenFileGDB: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16716 NITF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16717 VRT: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16727 GRIB: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16728 SRP: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16730 GRIB: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16741 OpenFileGDB: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16745 MRF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16757 Internal libtiff: resync with upstream PDF: fix nullptr dereference on corrupted file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16759. Credit to OSS Fuzz NWT: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16762 OpenFileGDB: fix for potential unsigned integer overflow when .gdbtablx would be > 4GB. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16781 HFA: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16783 Internal libtiff: fix unsigned integer overflow / bad cast from uint64 to tmsize_t. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16784 RAWDatasetCheckMemoryUsage(): avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16785 BMP: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16789 MSGN: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16790 PCIDSK SDK: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16791 Internal libtiff: TIFFReadDirEntryData(): rewrite to avoid unsigned integer overflow (not a bug). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16792 appveyor.yml: update for PROJ 6.2 Internal libtiff: tif_ojpeg.c: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16793 NITF: fix unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16806 ENVI: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16808 OpenFileGDB: avoid report of unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16809 NITF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16812 Internal libtiff / tif_dirread.c: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16846 EXIFExtractMetadata: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16849 appveyor.yml: explicitly build PROJ 6.2 Internal libtiff: tif_ojpeg.c: avoid use of uninitialized memory on edge/broken file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16844 Revert unintended changes that went in previous commit SOSI: add a (weak) identify method HDF5: add a way to open datasets split over several files using the 'family' driver Add files that should have gone in previous commit multidim support: handle 0-size dimension VRT multidim: on reading, use correct group fullname NCDFIsUserDefinedType(): make it work for types in subgroups multidim support: deal with different arrays having 'internal' dimensions of same name but different size COG driver: add LERC compression (fixes #1812) Fix compiler warning on MSVC Internal libtiff. ByteCountLooksBad and EstimateStripByteCounts: avoid unsigned integer overflows. Fixes https://oss-fuzz.com/testcase-detail/5686156066291712 and https://oss-fuzz.com/testcase-detail/6332499206078464 Fix issues with thread_local and C++ objects that don't work well with DLL on Windows Doc: mention AWS_DEFAULT_PROFILE instead of AWS_PROFILE (fixes #1807) GRIB: add a hint to increase GRIB_CACHEMAX PCIDSK SDK: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16957 ISIS2: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16967 /vsitar/: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16978 PCIDSK SDK: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16996 GDALFindDataType(): simplify a bit, and add tests VICAR: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16999 Regenerate python bindings GPX: fix memory leak when streaming to /vsistdout/ VICAR: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17081 GTiff: make sure that GetMetadataDomainList() doesn't return EXIF when there's no EXIF metadata (fixes https://github.com/mapbox/rasterio/pull/1740#issuecomment-526660946) PDS: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17125 PDF: avoid potential integer division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17129 ogr_proj_p.cpp: OSRPJContextHolder() remove call to init() in constructor (fixes #1838) ENVI: preserve 'byte order' on update (fixes #1796) vrt_read.py: attempt at fixing random failure on test_vrt_shared_no_proxy_pool() on trusty_clang gdal_translate: avoid unsigned integer overflow with -limit_outsize option. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17186 NITF: avoid unsigned integer underflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17249 HOWTO-RELEASE: multiple updates, in particular to reflect the 2-phase procedure with RC first and then promotion to final Add VSIOverwriteFile() Shapefile driver: use VSIOverwriteFile() Internal libtiff: avoid too large memory allocation attempts. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17244 GeoJSON driver: use VSIOverwriteFile() to fix update of file on Windows (fixes https://github.com/qgis/QGIS/issues/28580) Doc: migrate https://trac.osgeo.org/gdal/wiki/GovernanceAndCommunity to RST, add new members Sean Gillies and Mateusz Loskot, move Andrey to retired members Add OSRPromoteTo3D() and map it to SWIG (PROJ >= 7) (fixes #1852) JP2OpenJPEG: fix reading overviews, when tiled API is used, and the dimensions of the full resolution image are not a multiple of 2^numresolutions (fixes #1860) JP2KAK: add a test related to #1860 WFS3: correctly handle user query string parameters in connection URL (fixes #1710) Add autotest/gdrivers/data/513x513.jp2 that should have gone with ed5fefc1b5ec7aeeec73af4cd687ec958cd5ad3c GeoJSON: on writing, format OFTDate and OFTDateTime as ISO 8601 strings rather than OGR traditional formatting NITF: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17594 FlatGeoBuf: fix memory leak in error code path. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17457 PDS4: add a CREATE_LABEL_ONLY=YES creation option, and create a <Header> element (fixes #1832) GPKG: on layer creation, check if the SRS is consistent with its advertize AUTHORITY/ID, and if not do not use official EPSG entries (fixes #1857) GDALRasterAttributeTable::TranslateToColorTable(): avoid potential int overfow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17676 OGRSpatialReference::importFromESRI(): acept COMPD_CS (fixes #1881) PDF: update to use newer versions of pdfium ISIS3: extract band name from BandBin group, and wavelength/bandwidth (fixes #1853) OGR SQLite: do not propagate 'IS / IS NOT value' constructs to OGR SQL JP2OpenJPEG: correctly set coordinates to opj_set_decode_area() on overviews (linked to previous fix) jp2openjpeg.py: avoid potential memory issue with old openjpeg versions ISIS3: preserve label in PAM .aux.xml when copying to other formats (fixes #1854) ElasticSearch: add FORWARD_HTTP_HEADERS_FROM_ENV open option to pass HTTP headers down to the ES server Add CPLLaunderForFilename() Shape: use CPLLaunderForFilename() FlatGeoBuf: add support for multiple layers in a directory of .fgb files, and fix single layer support (fixes #1862) configure: remove useless PDFIUM_LIBDIR test_ogrsf: do not error because FlatGeobuf does not return the expected layer geometry type configure: add -lm -lstdc++ for pdfium linking Travis: add support for PDFium in Ubuntu 16.04 build PDF: fix build against Poppler 0.82.0dev PDS4: update CART schema to 1B10_1931 and PDS to 1C00 (fixes #1851) Move oss-fuzz build.sh script to GDAL repository FlatGeobuf: constify and pass more arguments by reference in PackedRTree and Rect classes USGSDEM: fix reading FEMA generated Lidar datasets whose header is 918 bytes large GDALOpenEx(): more explicit error message if no driver are registered OSR: add an internal cache for importFromEPSG() and importFromWkt(). Helps performance for MapServer PROJ6 migration Doc: fix broken links to functions GTIFGetOGISDefn: avoid querying UOM length info when it is KvUserDefined to avoid an error to be emitted dump_jp2.py: fix for Python2 and non-ASCII characters Add support for Vertical Perspective projection (refs #1856) GTiff and PAM: allow serializing WKT2 for SRS using non-WKT1 compatible projections such as Vertical Perspective (refs #1856) gdal_translate / gdalwarp / ogrct: allow dealing with non-WKT1 representable SRS (refs #1856) ISIS3: add support for PointPerspective projection (refs #1856) OpenJPEG: fix to return the proper number of bytes read when we read more than 2 GB at once (fixes https://github.com/uclouvain/openjpeg/issues/1151) GTiff: do not write in GeoTIFF keys non-standard projections ISIS3: add support for Oblique Cylindrical projection (refs #1856) PDS3: fix support of Oblique Cylindrical (refs #1856) Internal libtiff: resync with upstream VRT: avoid erroneous pixel request do be done with KernelFilteredSource MRF: avoid excessive memory allocation attempt. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18011 ossfuzz: fix Poppler build Revert "ossfuzz: fix Poppler build" cpl_json: add const qualifiers ogrinfo: add a -nogeomtype switch WFS3: align with the latest state of the spec (refs #1878) Rename WFS3 driver as OAPIF (fixes #1878) OAPIF: clamp bounds of bbox filter OAPIF: add persistent curl session for better performance OAPIF: avoid TestCapability(OLCFastFeatureCount) to cause a network access, as well as SetAttributeFilter(NULL) OAPIF: remove no longer needed hack DDS: fix compiler warnings related to NULL vs nullptr OAPIF: remove outdated/no longer working support for XML that prevented to interoperate with updated cubewerx server OAPIF: for now do not look at all the 'crs' property, and always advertize CRS84 Doc OAPIF: update link to official standard ParseGMLCoordinates(): fix typo in error message [ci skip] Doc: fix broken link MSGN: avoid unsigned integer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18233 frmts/hdf5/makefile.vc: unbreak Windows build IGNFHeightASCIIGrid: fix to read RASPM2018.mnt grid OGRSimpleCurve::setNumPoints(): avoid potential int overflow on 32 bit on big values of nNewPointCount. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18092 PDF: fix reading polygon with holes and Bezier curves (fixes #1932) PDF: add a GDAL_PDF_LAUNDER_LAYER_NAMES configuration option Selafin: temptative fix for stack read overflow of https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18342 MRF: do not hardcode GDAL internal zlib header (from https://github.com/opengisch/OSGeo4A/blob/master/recipes/gdal/patches/gdal-zlib.patch) PDF vector: fix reading strings with escape sequences ParseGMLCoordinates(): avoid multiple realloc and potential performance issues, especially under fuzzing instrumentation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18349 OGRSpatialReference::importFromESRI(): avoid potential performance issue on hostile input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18033 HFA: avoid potential performance issue on hostile input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18088 Fix false positive MSVC warning about variable potentially not initialized OGR_SDTS: avoid potential performance issue on hostile input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17740 BNA: code cleanup and hopefully fix performance issue of https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17572 BNA: fix MSVC warning due to previous commit DXF: rework generation of features in INSERT blocks to avoid too much processing time/memory usage. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17282 DXF: ResetReading(): properly reset insert state (follow-up previous commit) XLSX: avoid potential performance issue on hostile input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17425 FlatGeoBuf: fix writing SRS without authority, and on reading avoid null pointer dereference if missing authority ogr_flatgeobuf.py: use a code that works with earlier PROJ 6 versions DXF writer: do not error out if trying to create a OGR_STYLE field (refs https://github.com/OSGeo/gdal/issues/835#issuecomment-544297557) FlatGeoBuf: use envelope from header to implement GetExtent() FlatGeoBuf: fix reading M-only geometries OGRSimpleCurve::getPoints() with XYZM: fix wrong stride used for M array OGRSimpleCurve: remove getPoints(XYZ) method to use getPoints(XYZM) with arguments with default value FlatGeoBuf: avoid crash on empty geometries; small cleanups OGRSimpleCurve: fix reversePoints() and addSubLineString() to take into account M dimension Fix Doxygen warnings of recent commit Fix expected results due to commit ad9178b63f501e9840d86a3220c9e5100c6e3fa2 importFromWkt(): limit size of input to avoid later potential performance problems. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18421 PDF: fix build against Poppler 0.82.0dev Internal libtiff: resync with upstream JPEG: fix further calls to RasterIO after reading full image at full resolution (fixes #1947) Internal libtiff: fix warning caused by previous commit FlatGeobuf: fixes for big-endian hosts. Tested on a PowerPC VM FlatGeobuf: avoid memleak when reading DateTime, and potential integer overflows on hostile input Add OGR_G_CreateGeometryFromEsriJson() and map it to SWIG FlatGeoBuf: fix potential nullptr dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18506 VICAR: support reading more data types and pixel organization (refs #1855) VICAR: fix reading of EOL labels with non BSQ organizations, and possible confusion with LBLSIZE of EOL overwriting the main one Move IEEE <--> VAX double conversion from DGN driver to CPL, so they can be reused by the VICAR driver VICAR: do not error out if the EOL keyword is missing. It is not mandatory VICAR (and RawRasterBand and VRTRawRasterBand): add support for VAX order for floating point data types VICAR: improvements in label reading, and report label in a json:VICAR metadata doma…
This fixes an issue hit by GDAL, and that is found in netcdf 4.6.3
and 4.7.0
git bisect pointed the problem to have started with
where nc_get_vara_double() started using nc4_get_vars() underneath.
It turns out that nc4_get_vars() was buggy in the situation exercised by GDAL.
This can be reproduced with the following simple test case:
Running it under Valgrind without this patch leads to