JPEGLib: Backing store not supported in 1.5.2 #162
Comments
|
Backing store is a legacy feature of libjpeg that essentially implemented a kind of virtual memory on old Messy-DOS machines. That feature has never been available with libjpeg-turbo, since our library only uses the malloc()/free() (jmemnobs) memory manager from libjpeg. However, what that error message really means is that the application configured a memory limit by setting For instance, this is what happens when I try to compress a file that is 18.9 MB in size: (succeeds because (fails because 18.9 MB > 18000 kB, the memory limit defined using the (succeeds because 18.9 MB < 19000 kB) The default is unlimited, but unfortunately libtiff is explicitly setting the limit to 10 MB: https://github.com/vadz/libtiff/blob/master/libtiff/tif_jpeg.c#L2434-L2443 and doesn't seem to provide a way of overriding that. In libjpeg-turbo, the Closing as SEP, since this is really a libtiff problem that was merely exposed by the fact that libjpeg-turbo decided to start honoring |
rouault
added a commit
to MapServer/MapServer
that referenced
this issue
Oct 17, 2017
kwrobot
pushed a commit
to aashish24/gdal-svn
that referenced
this issue
Oct 17, 2017
…_use (cf libjpeg-turbo/libjpeg-turbo#162) git-svn-id: https://svn.osgeo.org/gdal/trunk/gdal@40472 f0d54148-0727-0410-94bb-9a71ac55c965
|
@dcommander > libtiff needs to raise its memory limit and/or provide a way for the user to override that limit. |
rouault
added a commit
to OSGeo/gdal
that referenced
this issue
Oct 17, 2017
…_use (cf libjpeg-turbo/libjpeg-turbo#162) git-svn-id: https://svn.osgeo.org/gdal/trunk@40472 f0d54148-0727-0410-94bb-9a71ac55c965
rouault
added a commit
to OSGeo/gdal
that referenced
this issue
Oct 17, 2017
…_use (cf libjpeg-turbo/libjpeg-turbo#162) git-svn-id: https://svn.osgeo.org/gdal/branches/2.2@40473 f0d54148-0727-0410-94bb-9a71ac55c965
|
@rouault that would be the responsibility of @Malvineous, who was experiencing the issue. I don't personally use libtiff. |
|
FYI I asked my distro maintainers to do this as I couldn't find libtiff contact details at the time. @rouault: Are you willing to contact libtiff or would you prefer I do? |
I'm a libtiff maintainer. I've already fixed the issue in libtiff CVS. This should reflect in the https://github.com/vadz/libtiff/commits/master mirror within a day or so |
vadz
pushed a commit
to vadz/libtiff
that referenced
this issue
Oct 18, 2017
honours max_memory_to_use > 0. Cf libjpeg-turbo/libjpeg-turbo#162
netbsd-srcmastr
pushed a commit
to NetBSD/pkgsrc
that referenced
this issue
Feb 19, 2018
* Add Makefile.common to prepare for geography/py-gdal
Changelog:
= GDAL/OGR 2.2.3 Release Notes =
The 2.2.3 release is a bug fix release.
== Build ==
* nmake.opt: Ensure PDB is included in release DLL if WITH_PDB requested (#7055)
== Port ==
* /vsicurl/ and derived filesystems: redirect ReadDir() to ReadDirEx() (#7045)
* /vsicurl/: enable redirection optimization on signed URLs of Google Cloud Storage. Helps for the PLScenes driver (#7067)
* /vsicurl/: fix 2.2 regression regarding retrieval of file size of FTP file (#7088)
* /vsis3/: fix to avoid invalid content to be sent if Write() writes more than 50 MB in a single call (#7058)
* /vsis3/: fix Seek(Tell(), SEEK_SET) fails if current position is not 0 (#7062)
* /vsis3/: fix support of non-ASCII characters in keys (#7143, #7146)
* Fix CPLCopyTree() that doesn't properly on MSVC 2015 (and possibly other platforms) (#7070)
== Algorithms ==
* RPC transformer: set output coordinates to HUGE_VAL when failure occurs, so that a following coordinate transformation can detect the error too (#7090)
* GDALGrid() with linear algorithm: avoid assertions/segmentation fault when GDALTriangulationFindFacetDirected() fails (#7101)
* GDALComputeProximity(): fix int32 overflow when computing distances on large input datasets (#7102)
* GDALResampleChunk32R_Gauss: fix potential out of bounds access (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4056)
== GDAL utilities ==
* gdalwarp: fix issue with GDALAdjustValueToDataType(Float32, +/- inf) that didn't preserve infinity, which affected gdalwarp -dstnodata inf (#7097)
* gdalwarp -crop_to_cutline: reduce number of iterations to find the appropriate densification (#7119)
* gdal_contour: return with non-0 code if field creation or contour generation failed (#7147)
== GDAL drivers ==
GeoRaster driver:
* fix int overflow (#6999)
GPKG driver:
* speed-up statistics retrieval on non-Byte datasets (#7096)
GRIB driver:
* if GRIB_ADJUST_LONGITUDE_RANGE config option is set to YES, adjust the longitude range to be close to [-180,180] when possible for products whose left origin is close to 180deg. (#7103)
GSAG driver:
* avoid assertion on some files with nul characters (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3726)
GTiff driver:
* make sure that -co PHOTOMETRIC=RGB overrides the color interpretation of the first 3 bands of the source datasets (#7064)
* on reading use GeogTOWGS84GeoKey to override the defaults TOWGS84 values coming from EPSG code (#7144)
* fix potential crash when reading in RGBA mode with internal libtiff (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4157)
* fix potential crash with old-jpeg compression and internal libtiff (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4180)
HTTP driver:
* do not open the underlying dataset with GDAL_OF_SHARED, to avoid later assertion
JP2KAK driver:
* fix unix build with Kakadu 7.A and later (#7048, #7081)
JPEG driver:
* Add compatibility with libjpeg-turbo 1.5.2 that honours max_memory_to_use (cf https://github.com/libjpeg-turbo/libjpeg-turbo/issues/162)
JP2OpenJPEG driver:
* Add support for openjpeg 2.3 (#7074)
netCDF driver:
* fix raster read as nodata with Byte datatype, (valid_range={0,255} or _Unsigned = True) and negative _FillValue (#7069)
* be more tolerant on the formatting of standard parallel (space separated instead of {x,y,...} syntax), and accept up to 2/1000 error on spacing to consider a regular grid, to be able to read files provided by the national weather institute of Netherlands (KNMI) (#7086)
PDF driver:
* round to upper integer when computing a DPI such that page size remains within limits accepted by Acrobat (#7083)
Sentinel2 driver:
* add support for direct opening of .zip files of new safe_compact L1C products (#7085)
VRT driver:
* avoid error being emitted when opening a VRTRawRasterBand in a .zip files (#7056)
== OGR core ==
* Fix OGR[Curve]Polygon::Intersects(OGRPoint*) to return true when point is on polygon boundary (#7091)
* importFromWkt(): fix import of GEOMETRYCOLLECTION ending with POINT EMPTY or LINESTRING EMPTY (#7128, 2.1 regression)
== OGR utilities ==
* ogrtindex: fix crash when using -f SQLITE -t_srs XXXX (#7053)
== OGR drivers ==
Amigocloud driver:
* Fixed data field types (https://github.com/OSGeo/gdal/pull/246)
* Output list of datasets if dataset id is not provided.
* Implemented waiting for job to complete on the server. This will improve write to AmigoCloud reliability.
* Updated documentation.
FileGDB driver:
* remove erroneous ODsCCreateGeomFieldAfterCreateLayer capability declaration (https://github.com/OSGeo/gdal/pull/247)
GeoJSON driver:
* ESRIJson: recognize documents that lack geometry fields (#7071)
* ESRIJson: recognize documents starting with a very long fieldAliases list (#7107)
GeoRSS driver:
* fix detection of field type (#7108)
GML driver:
* fix FORCE_SRS_DETECTION=YES effect on feature count and SRS reporting on gml files with .gfs (#7046)
* do not try to open kml files (#7061)
* GML geometry parsing: fix robustness issue with gml:PolyhedralSurface
* do not report gml:name / gml:description of features as layer metadata
GPKG driver:
* do not try to update extent on gpkg_contents after GetExtent() on a empty layer of a datasource opened in read-only mode
GTM driver:
* fix null pointer dereference in case of read error (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4047)
KML / LIBKML drivers:
MDB driver:
* fix multi-thread support (https://issues.qgis.org/issues/16039)
MITAB driver
* do not emit error if the .ind file is missing, just a debug message (#7094)
MySQL:
* fix build with MariaDB 10.2 (#7079)
OCI driver:
* initialize in multi-threaded compatible mode to fix QGIS related crashes (https://issues.qgis.org/issues/17311), and fix a few memleaks
ODS driver:
* avoid using CPLSPrintf() return directly with VSIFOpenL() as the temp buffer might be later recycled (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4050)
PLScenes driver:
* backport data/plscenesconf.json from trunk to add SkySatScene fields
Shapefile driver:
* Fix GetFeatureCount() to properly take into account spatial filter when attribute filter also in effect (#7123)
SQLite/Spatialite driver:
* don't invalidate statistics when running a PRAGMA (https://issues.qgis.org/issues/17424)
* SQLite dialect: support SQLite 3.21, and LIKE, <>, IS NOT, IS NOT NULL, IS NULL and IS operators (#7149)
XLS driver:
* workaround opening filenames with incompatible character set on Windows (https://issues.qgis.org/issues/9301)
XLSX driver:
* fix non working detection of Date/Time fields in some documents (#7073)
* fix opening of documents with x: namespace in xl/workbook.xml (#7110)
* avoid using CPLSPrintf() return directly with VSIFOpenL() as the temp buffer might be later recycled (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4050)
== SWIG bindings ==
* map GRA_Max, GRA_Min, GRA_Med, GRA_Q1 and GRA_Q3 (#7153)
== Python bindings ==
* remove 'from . import _gdal_array' line from gdal_array.py that is not necessary with normal execution of the bindings, and cause errors with PyInstaller (#7044)
= GDAL/OGR 2.2.2 Release Notes =
The 2.2.2 release is a bug fix release.
== Build ==
* Windows build: always build the PDF driver, even when none of poppler/podofo/pdfium are available, in which case it is write-only (#6938)
* Compilation fixes on Ubuntu 16.04 with explicit --std=c++03 (#6952)
== Port ==
* /vsigzip/: make Eof() detect end of stream when receive a Z_BUF_ERROR error. Fixes #6944.
* Fix memleak in VSIGSFSHandler::GetFileList()
* /vsigzip/: avoid trying to write a .gz.properties file on a /vsicurl/ file (#7016)
* Fix registration of /vsicrypt/ file system handler when compiled as a plugin (#7030)
* CPLStrtod(): parse string like '-1.#IND0000000' as NaN instead of -1 (#7031)
== Algorithms ==
* Warper: when operating on single-band, skip target pixels whose source center pixel is nodata (2.2 regression, #7001)
* Warper: avoid blocking when number of rows is 1 and NUM_THREADS > 1 (#7041). Also limit the number of threads so that each one processes at least 65536 pixels
== GDAL core ==
* GDALGCPsToGeoTransform(): add GDAL_GCPS_TO_GEOTRANSFORM_APPROX_OK=YES and GDAL_GCPS_TO_GEOTRANSFORM_APPROX_THRESHOLD=threshold_in_pixel configuration option (#6995)
* Fix issue with GDALProxyRasterBand::FlushCache() not flushing everything required (#7040)
* RawDataset::IRasterIO(): don't assume all bands are RawRasterBand
== GDAL utilities ==
* gdal2tiles.py: fix GDAL 2.2 regression where tilemapresource.xml was no longer generated (#6966)
* gdal_translate/DefaultCreateCopy(): do not destroy target file in case of failed copy wen using -co APPEND_SUBDATASET=YES (#7019)
* gdal_translate: make -b mask[,xx] use the appropriate band data type (#7028)
== GDAL drivers ==
GeoRaster driver:
* add support for GCP (#6973)
GPKG driver:
* do not error out if extent in gpkg_contents is present but invalid (#6976)
* fix opening subdatasets with absolute filenames on Windows (https://issues.qgis.org/issues/16997)
* fix possible assertion / corruption when creating a raster GeoPackage (#7022)
GSAG driver:
* fix reading issue that could cause a spurious 0 value to be read and shift all following values (#6992)
GTiff driver:
* fix reading subsampled JPEG-in-TIFF when block height = 8 (#6988)
* when reading a COMPD_CS (and GTIFF_REPORT_COMPD_CS=YES), set the name from the GTCitationGeoKey (#7011)
GTX driver:
* do not emit error when opening with GDAL_PAM_ENABLED=NO (#6996)
HF2 driver:
* fix reading tiles that are 1-pixel wide (2.1 regression, #6949)
IDRISI driver:
* Fix memleak in IDRISI Open()
ISIS3 driver:
* make sure that -co USE_SRC_HISTORY=NO -co ADD_GDAL_HISTORY=NO results in remove of History section (#6968)
* fix logic to initialize underlying GeoTIFF file in IWriteBlock(), and implement Fill() (#7040)
JPEG2000 driver:
* Fix build failure in C++03 mode on Jasper inclusion in RHEL 6 (#6951)
* Fix build failure with recent Jasper that no longer define uchar
JP2OpenJPEG driver:
* Add support for building against OpenJPEG 2.2 (#7002)
* fix performance issues with small images with very small tile size, such as some Sentinel2 quicklooks (#7012)
* properly use the opj_set_decode_area() API (#7018)
netCDF driver:
* avoid vector vs raster variable confusion that prevents reading Sentinel3 datasets, and avoid invalid geolocation array to be reported (#6974)
PDF driver:
* add support for Poppler 0.58 (#7033)
PDS driver:
* fix parsing of labels with nested list constructs (2.2 regression, #6970)
SRTMHGT driver:
* recognizes the .hgt.gz extension (#7016)
VRT driver:
* avoid stack buffer read overflow with complex data type and scale = 0. (oss-fuzz#2468)
* fix uninitialized buffer in areas without sources when using non pixel packed spacing (#6965)
* Warped VRT: correctly take into account cutline for implicit overviews; also avoid serializing a duplicate CUTLINE warping options in warped .vrt (#6954)
* Warped VRT: fix implicit overview when output geotransform is not the same as the transformer dst geotransform (#6972)
* fix IGetDataCoverageStatus() in the case of non-simple sources, which unbreaks gdalenhance -equalize (#6987)
== OGR core ==
* OGRCompondCurve::addCurveDirectly(): try reversing non contiguous curves (for GML/NAS)
* OGR API SPY: fix the way we map dataset handles to variable name, to avoid invalid reuses of variable still active
* OGRParseDate(): avoid false-positive time detection, in particular for GeoJSON (#7014)
* OGRCurve::get_isClosed(): do not take into account M component (#7017)
* OGRLineString::setPoint() and addPoint() with a OGRPoint* argument. properly takes into account ZM, Z and M dimensions (#7017)
== OGRSpatialReference ==
* Fix OGRSpatialReference::IsSame() to return FALSE when comparing EPSG:3857 (web_mercator) and EPSG:3395 (WGS84 Mercator) (#7029)
* importFromProj4(): implement import of Hotine Oblique Mercator Two Points Natural Origin, and fix OGRSpatialReference::Validate() for that formulation (#7042)
== OGR utilities ==
* ogr2ogr: fix small memory leak when using -limit switch
* ogr2ogr: make -f GMT work again (#6993)
== OGR drivers ==
AVCBin driver:
* fix 2.1 regression regarding attributes fetching (#6950)
DXF driver:
* fix reading files where INSERT is the last entity and DXF_MERGE_BLOCK_GEOMETRIES is false (#7006)
* avoid segfault when creating a DXF file with only the 'blocks' layer (#6998)
* fix potential null pointer dereference in PrepareLineStyle (#7038)
GeoJSON driver:
* fix 2.2 regression regarding lack of setting the FeatureCollection CRS on feature geometries (fixes https://github.com/r-spatial/sf/issues/449#issuecomment-319369945)
* prevent infinite recursion when reading geocouch spatiallist with properties.properties.properties (#7036)
GML driver:
* fix field type detection logic to avoid a field with xsi:nil=true to be identified as a string (#7027)
* JPGIS FGD v4: fix logic so that coordinate order reported is lon/lat (https://github.com/OSGeo/gdal/pull/241)
GMLAS driver:
* fix memleak in swe:DataRecord handling
* avoid false-positive warning about attribute found in document but ignored according to configuration, when the attribute is actually not present in the document but has a default value (#6956)
* (quick-and-not-prefect) fix to avoid a <xs:any> to prevent all other fields from being set (#6957)
* get the srsName when set on the srsName of the gml:pos of a gml:Point (#6962)
* CityGML related fixes: better take into account .xsd whose namespace prefix is not in the document, fix discovery of substitution elements, handle gYear data type (#6975)
GPKG driver:
* fix feature count after SetFeature() with sqlite < 3.7.8 (#6953)
* do not take into account gpkg_contents for the extent of vector layers when it is present but invalid (#6976)
* fix logic to detect FID on SQL result layer when several FID fields are selected (#7026)
KML / LIBKML drivers:
* read documents with an explicit kml prefix (#6981)
MSSQLSpatial driver:
* fix issues on x64 (#6930)
* properly format DELETE statement with schema (#7039)
OCI driver:
* check view with unquoted table name (#5552)
OpenFileGDB driver:
* properly read GeneralPolygon with M component whose all values are set to NaN (#7017)
OSM driver:
* increase string buffer for osm XML files (#6964)
Shapefile driver:
* use VSIStatL() in Create() to properly work with /vsimem/ directories (#6991)
* fix regression affecting GDAL 2.1.3 or later, 2.2.0 or later, when editing the last shape of a file and growing it, and then appending a new shape, which resulted in corruption of both shapes (#7031)
SQLite/Spatialite driver:
* escape integer primary key column name on table creation (#7007)
== C# bindings ==
* Implement RasterIO extensions (#6915)
== Python bindings ==
* fix reference count issue on gdal.VSIStatL() when it returns None, that can cause Python crashes if None refcount drops to zero
* add C-API to create driver instances for use in SWIG (https://trac.osgeo.org/osgeo4w/ticket/466)
== Security oriented fixes ==
Note: this is only a very partial backport of more extensive fixes done in GDAL trunk. Credit to OSS-Fuzz for all of them. (oss-fuzz#XXXX is a shortcut to
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=XXXX)
* NAS: avoid assertion / null ptr deref and possibly other failures on corrupted files. (oss-fuzz#2366, oss-fuzz#2580)
* NTF: avoid null ptr deref. Related to https://oss-fuzz.com/v2/testcase-detail/6527758899347456
* VSIMemHandle::Read(): avoid unwanted unsigned int overflow that cause a later heap buffer overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/5227283000328192.
* MBTiles: fix use after free. (oss-fuzz#2388)
* netCDF: fix stack buffer overflow when building vector layers from netCDF variables if they don't have the expected number of dimensions. (oss-fuzz#2400)
* DXF: prevent infinite loop / excessive memory allocation on truncated file. (#oss-fuzz#2403)
* DXF: avoid excessive memory allocation. (oss-fuzz#2444)
* XYZ: fix write heap-buffer-overflow (oss-fuzz#2426)
* GTiff: make IGetDataCoverageStatus() properly set the current TIFF directory, and only implement it for 'normal' bands. To avoid heap buffer overflows. (oss-fuzz#2438)
* GTiff: avoid null pointer derefrence when requested implicit overviews on corrupted JPEG-compressed file. (oss-fuzz#2441)
* GTiff: avoid potential issue on corrupted files (oss-fuzz#2481)
* GTiff: don't override member nBlocksPerRow member of GTiffJPEGOverviewBand with unrelated value, in case of single-strip case. (oss-fuzz#3020)
* PCIDSK: for band interleave channel, correctly use pixel_offset in case it is different from pixel_size. (oss-fuzz#2440)
* MITAB: fix skipping of blank lines when reading MIF geometries, and avoid potential very loop loop. Fixes https://oss-fuzz.com/v2/testcase-detail/4516730841858048
* CPLKeywordParser: avoid potential heap buffer overflow read. (oss-fuzz#2706)
* CPLKeywordParser: avoid potential infinite loop. Fixes https://oss-fuzz.com/v2/testcase-detail/5733173726019584
* morphFromESRI(): fixes heap-after-free uses. (oss-fuzz#2864)
* OGR_VRT: fix null pointer dereference on GetExtent() on an invalid layer. (oss-fuzz#3017)
* OGR_VRT: avoid crash on cyclic VRT. (oss-fuzz#3123)
* OSM: avoid potential write heap buffer overflow on corrupted PBF files. (oss-fuzz#3022)
* FAST: fix potential read heap buffer overflow. (oss-fuzz#3025)
* CPG: avoid null pointer dereference on failed open. (oss-fuzz#3136)
= GDAL/OGR 2.2.1 Release Notes =
The 2.2.1 release is a bug fix release.
== Build ==
* fix compilation without BIGTIFF_SUPPORT (#6890)
* configure: detect if std::isnan() is available. Helps compilation on some MacOSX setups, combined together --without-cpp11. Refs https://github.com/macports/macports-ports/pull/480
* fix compilation against ancient Linux kernel headers (#6909)
* fix detection of 64bit file API with clang 5 (#6912)
* configure: use .exe extension when building with mingw64* toolchains (fixes #6919)
* mongoDB: compilation fix on Windows
== Port ==
* /vsicurl/: fix occasional inappropriate failures in Read() with some combinations of initial offset, file size and read request size (#6901)
* Add a VSICurlClearCache() function (bound to SWIG as gdal.VSICurlClearCache()) to be able to clear /vsicurl/ related caches (#6937)
== Algorithms ==
* GDALRasterize(): avoid hang in some cases with all_touched option (#5580)
* gdal_rasterize: fix segfault when rasterizing onto a raster with RPC (#6922)
== GDAL utilities ==
* ogr_merge.py: fix '-single -o out.shp in.shp' case (#6888)
== GDAL drivers ==
AIGRID driver:
* fix handling on raw 32-bit AIG blocks
ENVISAT driver:
* fix 2.2 regression in initialization of members of MerisL2FlagBand. (#6929)
GeoRaster driver:
* Fix memory allocation failure (#6884)
* add support for JP2-F in BLOB compression (corrections on geo-reference) (#6861)
GPKG driver:
* avoid corruption of gpkg_tile_matrix when building overviews, down to a level where they are smaller than the tile size (#6932)
GTIFF driver:
* Internal libtiff: fix libtiff 4.0.8 regression regarding creating of single strip uncompressed TIFF files (#6924)
netCDF driver:
* add support for radian and microradian units for geostationnary projection (https://github.com/OSGeo/gdal/pull/220)
NWT_GRC driver:
* Fix handling of alpha values in GRC color table (#6905)
* Handle case of 0-len GRC class names (#6907)
VRT driver:
* speed-up SerializeToXML() in case of big number of bands
XYZ driver:
* fix 2.2 regression where the driver hangs on some dataset with missing samples (#6934)
== OGR utilities ==
* ogr2ogr/GDALVectorTranslate(): fix crash when using -f PDF -a_srs (#6920)
== OGR drivers ==
GeoJSON driver:
* ESRIJson: avoid endless looping on servers that don't support resultOffset (#6895)
* ESRIJson: use 'latestWkid' in priority over 'wkid' when reading 'spatialReference' (https://github.com/OSGeo/gdal/pull/218)
* GeoJSON writer: accept writing ZM or M geometry by dropping the M component (#6935)
GPKG driver:
* make driver robust to difference of cases between table_name in gpkg_contents/gpkg_geometry_columns and name in sqlite_master (#6916)
MITAB driver:
* recognize Reseau_National_Belge_1972 / EPSG:31370 on writing (#6903)
MySQL driver:
* fix compilation issue with Arch Linux and mariadb 10.1.23 (fixes #6899)
PG driver:
* do not be confused by a 'geometry' table in a non-PostGIS enabled database (#6896)
PLScenes:
* remove support for V0. Deprecate V1 API. Only Data V1 is supported ( #6933)
== Perl bindings ==
* Backport the fix to #6142 Install man page according to GDALmake.opt if INSTALL_BASE is set.
* always return something from non-void functions (#6898)
== Python bindings ==
* Accept callback = 0 since SWIG generates it as the default argument of BandRasterIONumPy(). Fixes https://github.com/OSGeo/gdal/pull/219
* Fix 2.2 regression preventing use of callback function in Band.ComputeStatistics() (#6927)
== Security oriented fixes ==
Note: this is only a very partial backport of more extensive fixes done in GDAL trunk. Credit to OSS-Fuzz for all of them (some have been found locally, so no related ticket)
* Fix CPLErrorSetState to ensure it does not write beyond DEFAULT_LAST_ERR_MSG_SIZE and correctly null-terminates last message. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1529.
* Open() and Stat() methods of a number of virtual file systems: check that the filename starts with the full prefix. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1543.
* VRT pixel functions: fix crash with 'complex' when source count is < 2. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1549
* OpenAIR: fix potential out-of-buffer read if we need to ingest 30000 bytes
* Several fixes in importFromWkb() and importFromWkt()
* GDALDataset and GDALRasterBand::ReportError(): fix crash if dataset name has a % character
* NASAKeywordHandler::SkipWhite(): fix out of bounds read
* MITAB: ParseTABFileFields(): fix out of bounds read.
* MITAB: ParseMIFHeader(): fix memory leak and out-of-bounds read on corrupted file
* MITAB: ParseMIFHeader(): fix memory leaks on corrupted files
* MITAB: avoid potentially veryyyy long loop when stroking arcs. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1644
* MITAB: avoid heap-buffer-overflow in MITABCoordSys2TABProjInfo(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1653
* OSARDataset::Open(): fix crash if pOpenInfo->fpL == NULL. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1565
* OGRESRIJSONReadPolygon(): fix crash in error code path
* DXF: prevent null ptr deref and out-of-bounds read on corrupted file
* DXF: TranslateSPLINE(): sanitize integer values read to avoid int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1843
* KML::unregisterLayerIfMatchingThisNode(): use memmove() instead of memcpy(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1580
* KML: fix crash on weird structure with recursively empty folders. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1591
* KML: fix null ptr dereference on corrupted file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1634
* OGRCurveCollection::importBodyFromWkb(): fix potential crash if the curve in the collection has not the same dimension has the collection dimension. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1584
* OGRCompoundCurve::importFromWkb(): avoid potential stack overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/5192348843638784
* TIGER: fix potential stack buffer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1586 and 2020
* TIGER: avoid potential infinite looping. Fixes https://oss-fuzz.com/v2/testcase-detail/4724815883665408
* VFK: avoid out-of-bounds read. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1596 and 2074
* CPLHexToBinary(): avoid reading outside of hex2char array on on-ASCII input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1606
* OGR PDS: avoid int32 overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=160
* GeoRSS: fix null pointer dereference on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1611.
* VSIArchiveFilesystemHandler::SplitFilename(): improve performance. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1597
* OGRGeometryFactory::organizePolygons(): fix crash on empty polygons. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1620
* JML: fix null pointer dereference on invalid files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1622
* Shape: prevent null ptr deref on truncated MultiPointM geometry. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1638
* /vsisubfile/: avoid Tell() to return negative values. And make VSIIngestFile() more robust to unsigned overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1642
* GTM: avoid useless recursive opening of files when provided with a gzip-compressed input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1650
* GTiff: fix heap-buffer-overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1677
* GTiff: avoid heap-buffer-overfow on corrupted State Plane citation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2050
* GTiff: avoid potential stack buffer overflow on corrupted Imagine citation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2105
* GTiff: prevent heap overflow and fix reading of multi-band band-interleaved 16/24-bit float datasets. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2264
* GTiff: fix potential infinite loop when parsing some 24-bit float denormalized numbers. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2315
* Internal libjson-c: fix stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1682
* ILI1/ILI2: fix null pointer dereference when opening filename ','. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1689
* ILI1: fix various crashes on corrupted files (including, but not limited to, https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1760, 1784, 1926)
* ILI2: use proper cast operator. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1813
* ILI2: fix null pointer dereference on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1993
* ILI2: fix crash due to unhandled exception. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2191
* morphFromESRI(): fix heap-use-after-free issue. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1690
* morphFromESRI(): prevent potential null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1783 and 1867
* SEGUKOOA: fix inversion of leap year that caused index-out-of-bound reading on day 366 of leap years (2.2 regression). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1698
* CPLParseXMLString(): make it error out on invalid XML file even under CPLTurnErrorIntoWarning() mode. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1703.
* GML / NAS: fix memory leak in error code path, and potential heap-buffer-read-overflow
* NTF: fix various issues: heap & stack buffer-overflow, null ptr derefs, memory leaks. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1819 , 1820 , 1823, 1833, 1854, 1862, 1910, 1931, 1961, 1990, 1995, 1996, 2003, 2033, 2052, 2077, 2084, 2103, 2130, 2135, 2146, 2166, 2185, 2187, https://oss-fuzz.com/v2/testcase-detail/4696417694121984
* OGRCreateFromMultiPatch(): avoid assertion on NaN coordinates. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1930
* GXF: validate nGType to avoid later out-of-bound read in GXFReadRawScanlineFrom(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1975
* GXF: fix int overflow and avoid excessive memory allocation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2207
* DGN: prevent heap-buffer-overflow read. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1988
* COSAR: fix leak of file descriptor. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2019
* ISO8211: prevent stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2022
* WEBP: prevent int32 overflow and too large memory allocation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2059
* IRIS: fix heap-buffer-overflow in some cases of nDataTypeCode. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2069
* E00GRID: avoid heap and stack buffer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2090 , 2182, 2237, 2327
* VICAR: fix null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2116
* VICAR: avoid use-after-free and heap-buffer-overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/4825577427042304
* VICAR: fix potential endless loop on broken files. Fixes https://oss-fuzz.com/v2/testcase-detail/6261508172414976
* REC: fix nullptr deref
* REC: fix potential stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2165
* GDALGetJPEG2000Structure() / DumpGeoTIFFBox(): fix memory leak.
* DumpGeoTIFFBox(): reject GeoJP2 boxes with a TIFF with band_count > 1
* DumpJPK2CodeStream(): avoid potentially very long loop
* GDALGetJPEG2000Structure(): avoid bad performance on corrupted JP2 boxes. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2214
* GPKG: fix potential heap-buffer overflow in GPkgHeaderFromWKB(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2150
* GPKG: fix potential null ptr deref. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2240
* GPKG: avoid potential division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2253
* SEGUKOOA: prevent read beyond end of buffer. (#6921)
* SRP: avoid potential stack buffer overflow and excessive memory allocation/processing time
* CPLUnixTimeToYMDHMS(): avoid potential infinite loop. Fixes https://oss-fuzz.com/v2/testcase-detail/4949567697059840
* Selafin: fix double frees. Fixes https://oss-fuzz.com/v2/testcase-detail/6429713822121984
* CEOS: fix heap buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2259
* CEOS: fix memleak in error code path. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2279
* FAST: avoid null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2290
* netCDF: avoid stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2302
GDAL/OGR 2.2.0 Release Notes =
== In a nutshell... ==
* New GDAL/raster drivers:
- DERIVED driver: read-support. Expose subdatasets in a a new metadata domain, called DERIVED_SUBDATASETS
- JP2Lura driver: read/create support for JPEG-2000 images using Luratech JP2 Library
- PRF: add read-only support for PHOTOMOD PRF file format driver (github #173)
- RRASTER driver: read-support .grd/.gri files handled by the R 'raster' package (#6249)
* New OGR/vector drivers:
- CAD driver: read support for DWG R2000 files (GSoC 2016 project)
- DGNv8 driver: read-write support for DGN 8.0 format (using Teigha ODA libraries)
- GMLAS driver: read-write support. XML/GML driver driven by Application Schemas.
* New utility script: ogrmerge.py to merge several vector datasets into a single one
* New /vsigs/ and /vsigs_streaming/ virtual file systems to read Google Cloud Storage non-public files
* Significantly improved drivers:
- NWT_GRD: write support (#6533)
- FileGDB/OpenFileGDB: add support to read curve geometries (#5890)
- VRT derived band: add the capability to define pixel functions in Python
- Add read support for RasterLite2 coverages in SQLite driver
- GPKG: implement tiled gridded elevation data extension
- ISIS3: add write support and improve read support
* RFC 63: Add GDALRasterBand::GetDataCoverageStatus() and implement it in GTiff and VRT drivers
https://trac.osgeo.org/gdal/wiki/rfc63_sparse_datasets_improvements
* RFC 64: Triangle, Polyhedral surface and TIN
https://trac.osgeo.org/gdal/wiki/rfc64_triangle_polyhedralsurface_tin
==> this RFC introduces potential backward incompatible behaviour.
Consult MIGRATION_GUIDE.txt
* RFC 66: OGR random layer read/write capabilities
https://trac.osgeo.org/gdal/wiki/rfc66_randomlayerreadwrite
* RFC 67: add null field state for OGR features, in addition to unset fields
https://trac.osgeo.org/gdal/wiki/rfc67_nullfieldvalues
==> this RFC introduces potential backward incompatible behaviour.
Consult MIGRATION_GUIDE.txt
* Upgrade to EPSG database v9.0 (#6772)
* Python bindings: Global Interpreter Lock (GIL) released before entering GDAL native code (for all, in GDAL module and a few ones in ogr like ogr.Open())
* Continued major efforts on sanitization of code base
* Remove bridge and vb6 bindings (#6640)
* GNM built by default
== Installed files ==
* Removed: data/s57attributes_aml.csv data/s57attributes_iw.csv data/s57objectclasses_aml.csv data/s57objectclasses_iw.csv
* Added plscenesconf.json, gmlasconf.xsd
== Backward compatibility issues ==
See MIGRATION_GUIDE.TXT
== GDAL/OGR 2.2.0 - General Changes ==
Build(Unix):
* improve detection of packaged libfyba (SOSI) --with-sosi, as in Ubuntu 16.04 (#6488)
* Sort files in static library to make the build reproducible (#6520)
* fix libqhull include path when it is /usr/local/include/libqhull (#6522)
* FileGDB: compilation fix on Linux in C++11 mode
* configure: make pdfium detection not fail if there are just warnings. And make configure fail if --with-pdfium was required but failed (#6653)
* Make ./configure --with-xerces fail if not found
* Don't install script documentation in INST_BIN (github #157)
* configure: auto-detect webp without requiring explicit --with-webp
* configure: use pkg-config for HDF5 detection so that works out of the box on recent Ubuntu
* auto-detect JDK 8 on Ubuntu
* MDB: allow libjvm.so to be dlopen'd with --with-jvm-lib=dlopen (Unix only, github #177)
* configure: delete temporary directories on the mac
* configure: make sure --with-macosx-framework is correctly defined
* configure: error out if --with-ld-shared is specified (#6769)
* configure: remove bashism.
* configure: fix --without-mrf (#6811)
* configure: take into account CXXFLAGS and LDFLAGS in a few more cases (cryptopp, podofo, libdap)
* Vagrant: all lxc and Hyper-V provider support; use vagrant-cachier for package caching
* configure: update DWG support to work with Teigha libraries
* Internal libgeotiff: hide symbols in --with-hide-internal-symbols mode
* Shape: do not export Shapelib symbols for builds --with-hide-internal-symbols (#6860)
Build(Windows):
* Try to avoid confusion between libqhull io.h and mingw own io.h (#6590)
* update script to generate most recent Visual C++ project files (#6635)
* fix broken and missing dependencies in makefile.vc
* add a way to use an external zlib (github #171)
* Rename makegdal_gen.bat to generate_vcxproj.bat
* generate_vcxproj.bat: Set correct value of PlatformToolset property based on specified Visual C++ version. Add NMAKE command line parameters: MSVC_VER based on specified Visual C++ version, DEBUG=1 WITH_PDB=1 to Debug build configuration.
* generate_vcxproj.bat: generate project for autotest/cpp (Ticket #6815)
* Add WIN64=1 to NMAKE command line options.
* Add HDF4_INCLUDE option (#6805)
* Add MSVC compiler option /MP to build with parallel processes.
* Add ZLIB_LIB missing from EXTERNAL_LIBS
Build(all):
* make Xerces 3.1 the minimal version
* drop support for PostgreSQL client library older than 7.4, or non security maintained releases older than 8.1.4, 8.0.8, 7.4.13, 7.3.15
== GDAL 2.2.0 - Overview of Changes ==
Port:
* Export VSICreateCachedFile() as CPL_DLL so as to enable building JP2KAK as a plugin
* Added possibility to find GDAL_DATA path using INST_DATA definition without execution GDALAllRegister if GDAL_DATA placed in version named directory on Linux (#6543)
* Unix filesystem: make error message about failed open to report the filename (#6545)
* File finder: Remove hardcoded find location (/usr/local/share/gdal) (#6543)
* Win32 filesystem handler: make Truncate() turn on sparse files when doing file extension
* Add VSIFGetRangeStatusL() and VSISupportsSparseFiles()
* GDAL_NO_HARDCODED_FIND compilation option (#6543,#6531) to block file open calls (for sandboxed systems)
* Add VSIMallocAligned(), VSIMallocAlignedAuto() and VSIFreeAligned() APIs
* /vsizip / /vsitar: support alternate syntax /vsitar/{/path/to/the/archive}/path/inside/the/tar/file so as not to be dependent on file extension and enable chaining
* Optimize opening of /vsitar/my.tar.gz/my_single_file
* /vsizip/ : support creating non-ASCII filenames inside a ZIP (#6631)
* VSI Unix file system: fix behaviour in a+ mode that made MRF caching not work on Mac and other BSD systems
* Fix deadlock at CPLWorkerThreadPool destruction (#6646)
* Windows: honour GDAL_FILENAME_IS_UTF8 setting to call LoadLibraryW() (#6650)
* CPLFormFilename(): always use / path separator for /vsimem, even on Windows
* /vsimem/: add trick to limit the file size, so as to be able to test how drivers handle write errors
* /vsimem/: fix potential crash when closing -different- handles pointing to the same file from different threads (#6683)
* CPLHTTPFetch(): add MAX_FILE_SIZE option
* CPLHTTPFetch(): add a CAINFO option to set the path to the CA bundle file. As a fallback also honour the CURL_CA_BUNDLE and SSL_CERT_FILE environment variables used by the curl binary, which makes this setting also available for /vsicurl/, /vsicurl_streaming/, /vsis3/ and /vsis3_streaming/ file systems (#6732)
* CPLHTTPFetch(): don't disable peer certificate verification when doing https (#6734)
* CPLHTTPFetch(): cleanly deal with multiple headers passed with HEADERS and separated with newlines
* CPLHTTPFetch(): add a CONNECTTIMEOUT option
* CPLHTTPFetch(): add a GDAL_HTTP_HEADER_FILE / HEADER_FILE option.
* CPLHTTPSetOptions(): make redirection of POST requests to still be POST requests after redirection (#6849)
* /vsicurl/: take CPL_VSIL_CURL_ALLOWED_EXTENSIONS into account even if GDAL_DISABLE_READDIR_ON_OPEN is defined (#6681)
* /vsicurl/: get modification time if available from GET or HEAD results
* /vsis3/: add a AWS_REQUEST_PAYER=requester configuration option (github #186)
* CPLParseXMLString(): do not reset error state
* Windows: fix GetFreeDiskSpace()
* Fix GetDiskFreeSpace() on 32bit Linux to avoid 32bit overflow when free disk space is above 4 GB (#6750)
* Fix CPLPrintUIntBig() to really print a unsigned value and not a signed one
* Add CPL_HAS_GINT64, GINT64_MIN/MAX, GUINT64_MAX macros (#6747)
* Add CPLGetConfigOptions(), CPLSetConfigOptions(), CPLGetThreadLocalConfigOptions() and CPLSetThreadLocalConfigOptions() to help improving compatibility between osgeo.gdal python bindings and rasterio (related to https://github.com/mapbox/rasterio/pull/969)
* MiniXML serializer: fix potential buffer overflow.
Core:
* Proxy dataset: add consistency checks in (the unlikely) case the proxy and underlying dataset/bands would not share compatible characteristics
* GDALPamDataset::TryLoadXML(): do not reset error context when parsing .aux.xml
* PAM/VRT: only take into account <Entry> elements when deserializing a <ColorTable>
* GDALCopyWords(): add fast copy path when src data type == dst data type == Int16 or UInt16
* GetVirtualMemAuto(): allow USE_DEFAULT_IMPLEMENTATION=NO to prevent the default implementation from being used
* Nodata comparison: fix test when nodata is FLT_MIN or DBL_MIN (#6578)
* GetHistogram() / ComputeRasterMinMax() / ComputeStatistics(): better deal with precision issues of nodata comparison on Float32 data type
* Fast implementation of GDALRasterBand::ComputeStatistics() for GDT_Byte and GDT_UInt16 (including use of SSE2/AVX2)
* Driver manage: If INST_DATA is not requested, do not check the GDAL_DATA variable.
* Make sure that GDALSetCacheMax() initialize the raster block mutex (#6611)
* External overview: fix incorrect overview building when generating from greater overview factors to lower ones, in compressed and single-band case (#6617)
* Speed-up SSE2 implementation of GDALCopy4Words from float to byte/uint16/int16
* Add SSE2 and SSSE3 implementations of GDALCopyWords from Byte with 2,3 or 4 byte stride to packed byte
* GDALCopyWords(): SSE2-accelerated Byte->Int32 and Byte->Float32 packed conversions
* Fix GDALRasterBand::IRasterIO() on a VRT dataset that has resampled sources, on requests such as nXSize == nBufXSize but nXSize != dfXSize
* GDALRasterBand::IRasterIO(): add small epsilon to floating-point srcX and srcY to avoid some numeric precision issues when rounding.
* Add GDALRasterBand::GetActualBlockSize() (#1233)
* Fix potential deadlock in multithreaded writing scenarios (#6661)
* Fix thread-unsafe behaviour when using GetLockedBlock()/MarkDirty()/DropLock() lower level interfaces (#6665)
* Fix multi-threading issues in read/write scenarios (#6684)
* Resampled RasterIO(): so as to get consistent results, use band datatype as intermediate type if it is different from the buffer type
* Add GDALIdentifyDriverEx() function (github #152)
* GDALOpenInfo: add a papszAllowedDrivers member and fill it in GDALOpenEx()
* GDALDefaultOverviews::BuildOverviews(): improve progress report
* Average and mode overview/rasterio resampling: correct source pixel computation due to numerical precision issues when downsampling by an integral factor, and also in oversampling use cases (github #156)
* Overview building: add experimental GDAL_OVR_PROPAGATE_NODATA config option that can be set to YES so that a nodata value in source samples will cause the target pixel to be zeroed. Only implemented for AVERAGE resampling right now
* GDALValidateOptions(): fix check of min/max values
* GMLJP2 v2: update to 2.0.1 corrigendum and add capability to set gml:RectifiedGrid/gmlcov:rangeType content. Set SRSNAME_FORMAT=OGC_URL by default when converting to GML. Add gml:boundedBy in gmljp2:GMLJP2RectifiedGridCoverage
* GMLJP2 v2: ensure KML root node id unicity when converting annotations on the fly. When generating GML features, make sure that PREFIX and TARGET_NAMESPACE are unique when specifying several documents.
Algorithms:
* RPC transformer: speed-up DEM extraction by requesting and caching a larger buffer, instead of doing many queries of just a few pixels that can be costly with VRT for example
* GDALDeserializeRPCTransformer(): for consistency, use the same default value as in GDALCreateRPCTransformer() if <PixErrThreshold> is missing (so use 0.1 instead of 0.25 as before)
* TPS solver: when Armadillo fails sometimes, fallback to old method
* GDALCreateGenImgProjTransformer2(): add SRC_APPROX_ERROR_IN_SRS_UNIT, SRC_APPROX_ERROR_IN_PIXEL, DST_APPROX_ERROR_IN_SRS_UNIT, DST_APPROX_ERROR_IN_PIXEL, REPROJECTION_APPROX_ERROR_IN_SRC_SRS_UNIT and REPROJECTION_APPROX_ERROR_IN_DST_SRS_UNIT transformer options, so as to be able to have approximate sub-transformers
* Fix GDAL_CG_Create() to call GDALContourGenerator::Init() (#6491)
* GDALContourGenerate(): handle the case where the nodata value is NaN (#6519)
* GDALGridCreate(): fix hang in multi-threaded case when pfnProgress is NULL or GDALDummyProgress (#6552)
* GDAL contour: fix incorrect oriented contour lines in some rare cases (#6563)
* Warper: multiple performance improvements for cubic interpolation and uint16 data type
* Warper: add SRC_ALPHA_MAX and DST_ALPHA_MAX warp options to control the maximum value of the alpha channel. Set now to 65535 for UInt16 (and 32767 for Int16), or to 2^NBITS-1. 255 used for other cases as before
* Warper: avoid undefined behaviour when doing floating point to int conversion, that may trigger exception with some compilers (LLVM 8) (#6753)
* OpenCL warper: update cubicConvolution to use same formula as CPU case (#6664)
* OpenCL warper: fix compliance to the spec. Fix issues with NVidia opencl (#6624, #6669)
* OpenCL warper: use GPU based over CPU based implementation when possible, use non-Intel OpenCL implementation when possible. Add BLACKLISTED_OPENCL_VENDOR and PREFERRED_OPENCL_VENDOR to customize choice of implementation
Utilities:
* gdalinfo -json: fix order of points in wgs84Extent.coordinates (github #166)
* gdalwarp: do not densify cutlines by default when CUTLINE_BLEND_DIST is used (#6507)
* gdalwarp: when -to RPC_DEM is specified, make -et default to 0 as documented (#6608)
* gdalwarp: improve detection of source alpha band and auto-setting of target alpha band. Automatically set PHOTOMETRIC=RGB on target GeoTIFF when input colors are RGB
* gdalwarp: add a -nosrcalpha option to wrap the alpha band as a regular band and not as the alpha band
* gdalwarp: avoid cutline densification when no transform at all is involved (related to #6648)
* gdalwarp: fix failure with cutline on a layer of touching polygons (#6694)
* gdalwarp: allow to set UNIFIED_SRC_NODATA=NO to override the default that set it to YES
* gdalwarp: fix -to SRC_METHOD=NO_GEOTRANSFORM -to DST_METHOD=NO_GEOTRANSFORM mode (#6721)
* gdalwarp: add support for shifting the values of input DEM when source and/or target SRS references a proj.4 vertical datum shift grid
* gdalwarp: fix crash when -multi and -to RPC_DEM are used together (#6869)
* gdal_translate: when using -projwin with default nearest neighbour resampling, align on integer source pixels (#6610)
* gdal_translate & gdalwarp: lower the default value of GDAL_MAX_DATASET_POOL_SIZE to 100 on MacOSX (#6604)
* gdal_translate: avoid useless directory scanning on GeoTIFF files
* gdal_translate: make "-a_nodata inf -ot Float32" work without warning
* gdal_translate: set nodata value on vrtsource on scale / unscale / expand cases (github #199)
* GDALTranslate(): make it possible to create a anonymous target VRT from a (anonymous) memory source
* gdaldem: speed-up computations for src type = Byte/Int16/UInt16 and particularly for hillshade
* gdaldem hillshade: add a -multidirectional option
* GDALDEMProcessing() API: fix -alt support (#6847)
* gdal_polygonize.py: explicitly set output layer geometry type to be polygon (#6530)
* gdal_polygonize.py: add support for -b mask[,band_number] option to polygonize a mask band
* gdal_rasterize: make sure -3d, -burn and -a are exclusive
* gdal_rasterize: fix segfaults when rasterizing into an ungeoreferenced raster, or when doing 'gdal_rasterize my.shp my.tif' with a non existing my.tif (#6738)
* gdal_rasterize: fix crash when rasterizing empty polygon (#6844)
* gdal_grid: add a smoothing parameter to invdistnn algorithm (github #196)
* gdal_retile.py: add a -overlap switch
* gdal2tiles.py: do not crash on empty tiles generation (#6057)
* gdal2tiles.py: handle requested tile at too low zoom to get any data (#6795)
* gdal2tiles: fix handling of UTF-8 filenames (#6794)
* gdal2xyz: use %d formatting for Int32/UInt32 data types (#6644)
* gdal_edit.py: add -scale and -offset switches (#6833)
* gdaltindex: emit warning in -src_srs_format WKT when WKT is too large
* gdalbuildvrt: add a -oo switch to specify dataset open options
Python samples:
* add validate_cloud_optimized_geotiff.py
* add validate_gpkg.py
Multi-driver:
* Add GEOREF_SOURCES open option / GDAL_GEOREF_SOURCES config. option to all JPEG2000 drivers and GTiff to control which sources of georeferencing can be used and their respective priority
AIGRID driver:
* fix 2.1.0 regression when reading statistics (.sta) file with only 3 values, and fix <2.1 behaviour to read them in LSB order (#6633)
AAIGRID driver:
* auto-detect Float64 when the nodata value is not representable in the Float32 range
ADRG driver:
* handle north and south polar zones (ZNA 9 and 18) (#6783)
ASRP driver:
* fix georeferencing of polar arc zone images (#6560)
BPG driver:
* declare GDALRegister_BPG as C exported for building as a plugin (#6693)
DIMAP driver:
* DIMAP: for DIMAP 2, read RPC from RPC_xxxxx.XML file (#6539)
* DIMAP/Pleiades metadata reader: take into tiling to properly shift RPC (#6293)
* add support for tiled DIMAP 2 datasets (#6293)
DODS driver:
* fix crash on URL that are not DODS servers (#6718)
DTED driver:
* correctly create files at latitudes -80, -75, -70 and -50 (#6859)
ECW driver:
* Add option ECW_ALWAYS_UPWARD=TRUE/FALSE to work around issues with "Downward" oriented images (#6516).
ENVI driver:
* on closing, pad image file with trailing nul bytes if needed (#6662)
* add read/write support for rotated geotransform (#1778)
GeoRaster driver:
* fix report of rotation (#6593)
* support for JP2-F compression (#6861)
* support direct loading of JPEG-F when blocking=no (#6861)
* default blocking increased from 256x256 to 512x512 (#6861)
GPKG driver:
* implement tiled gridded elevation data extension
* add VERSION creation option
* check if transaction COMMIT is successful (#6667)
* fix crash on overview building on big overview factor (#6668)
* fix crash when opening an empty raster with USE_TILE_EXTENT=YES
* fix gpkg_zoom_other registration
GTiff driver:
* support SPARSE_OK=YES in CreateCopy() mode (and in update mode with the SPARSE_OK=YES open option), by actively detecting blocks filled with 0/nodata about to be written
* When writing missing blocks (i.e. non SPARSE case), use the nodata value when defined. Otherwise fallback to 0 as before.
* in FillEmptyTiles() (i.e. in the TIFF non-sparse mode), avoid writing zeroes to file so as to speed up file creation when filesystem supports ... sparse files
* add write support for half-precision floating point (Float32 with NBITS=16)
* handle storing (and reading) band color interpretation in GDAL internal metadata when it doesn't match the capabilities of the TIFF format, such as B,G,R ordering (#6651)
* Fix RasterIO() reported when downsampling a RGBA JPEG compressed TIFF file (#6943)
* Switch search order in GTIFGetOGISDefn() - Look for gdal_datum.csv before datum.csv (#6531)
* optimize IWriteBlock() to avoid reloading tile/strip from disk in multiband contig/pixel-interleave layouts when all blocks are dirty
* fix race between empty block filling logic and background compression threads when using Create() interface and NUM_THREADS creation option (#6582)
* use VSIFTruncateL() to do file extension
* optimize reading and writing of 1-bit rasters
* fix detection of blocks larger than 2GB on opening on 32-bit builds
* fix saving and loading band description (#6592)
* avoid reading external metadata file that could be related to the target filename when using Create() or CreateCopy() (#6594)
* do not generate erroneous ExtraSamples tag when translating from a RGB UInt16, without explicit PHOTOMETRIC=RGB (#6647)
* do not set a PCSCitationGeoKey = 'LUnits = ...' as the PROJCS citation on reading
* fix creating an image with the Create() interface with BLOCKYSIZE > image height (#6743)
* fix so that GDAL_DISABLE_READDIR_ON_OPEN = NO / EMPTY_DIR is properly honoured and doesn't cause a useless directory listing
* make setting GCPs when geotransform is already set work (with warning about unsetting the geotransform), and vice-versa) (#6751)
* correctly detect error return of TIFFReadRGBATile() and TIFFReadRGBAStrip()
* in the YCBCR RGBA interface case, only expose RGB bands, as the alpha is always 255
* don't check free disk space when outputting to /vsistdout/ (#6768)
* make GetUnitType() use VERT_CS unit as a fallback (#6675)
* in COPY_SRC_OVERVIEWS=YES mode, set nodata value on overview bands
* read GCPs in ESRI <GeodataXform> .aux.xml
* explicitly write YCbCrSubsampling tag, so as to avoid (latest version of) libtiff to try reading the first strip to guess it. Helps performance for cloud optimized geotiffs
* map D_North_American_1927 datum citation name to OGC North_American_Datum_1927 so that datum is properly recognized (#6863)
* Internal libtiff. Resync with CVS (post 4.0.7)
* Internal libtiff: fix 1.11 regression that prevents from reading one-strip files that have no StripByteCounts tag (#6490)
GRASS driver:
* plugin configure: add support for GRASS 7.2 (#6785)
* plugin makefile: do not clone datum tables and drivers (#2953)
* use Rast_get_window/Rast_set_window for GRASS 7 (#6853)
GRIB driver:
* Add (minimalistic) support for template 4.15 needed to read Wide Area Forecast System (WAFS) products (#5768)
* **Partial** resynchronization with degrib-2.0.3, mostly to get updated tables (related to #5768)
* adds MRMS grib2 decoder table (http://www.nssl.noaa.gov/projects/mrms/operational/tables.php) (github #160)
* enable PNG decoding on Unix (#5661, github #160)
* remove explicitly JPEG2000 decompression through Jasper and use generic GDAL code so that other drivers can be triggered
* fix a few crashes on malformed files
GTX driver:
* add a SHIFT_ORIGIN_IN_MINUS_180_PLUS_180 open option
HDF4 driver:
* Fixed erroneous type casting in HDF4Dataset::AnyTypeToDouble() that breaks reading georeferencing and other metadata
HDF5 driver:
* correct number of GCPs to avoid dummy trailing (0,0)->(0,0,0) and remove +180 offset applied to GCP longitude. Add instead a heuristics to determine if the product is crossing the antimeridian, and a HDF5_SHIFT_GCPX_BY_180 config option to be able to override the heuristics (#6666)
HFA driver:
* fix reading and writing of TOWGS84 parameters (github #132)
* export overview type from HFA files to GDAL metadata as OVERVIEWS_ALGORITHM (github #135)
* make .ige initialization use VSIFTruncateL() to be faster on Windows
* add support for TMSO and HOM Variant A projections (#6615)
* Add elevation units read from HFA files metadata (github #169)
* set binning type properly according to layerType being thematic or not (#6854)
Idrisi driver:
* use geotransform of source dataset even if it doesn't have a SRS (#6727)
* make Create() zero-initialize the .rst file (#6873)
ILWIS driver:
* avoid IniFile::Load() to set the bChanged flag, so as to avoid a rewrite of files when just opening datasets
ISCE driver:
* fix computation of line offset for multi-band BIP files, and warn if detecting a wrong file produced by GDAL 2.1.0 (#6556)
* fix misbehaviour on big endian hosts
* add support for reading and writing georeferencing (#6630, #6634)
* make parsing of properties case insensitive (#6637)
ISIS3 driver:
* add write support
* add mask band support on read
* get label in json:ISIS3 metadata domain
JPEGLS driver:
JP2ECW driver:
* fix crash when translating a Float64 raster (at least with SDK 3.3)
JP2KAK driver:
* add support for Kakadu v7.9. v7.8 should not be used. It has a bug fixed in v7.9
* catch exceptions in jp2_out.write_header()
JP2OpenJPEG driver:
* add a USE_TILE_AS_BLOCK=YES open option that can help with whole image conversion
* prevent endless looping in openjpeg in case of short write
* for single-line organized images, such as found in some GRIB2 JPEG2000 images, use a Wx1 block size to avoid huge performance issues (#6719)
* ignore warnings related to empty tag-trees.
JPIPKAK driver:
* fix random crashes JPIP in multi-tread environment (#6809)
KEA driver:
* Add support for Get/SetLinearBinning (#6855)
KMLSuperOverlay driver:
* recognize simple document made of GroundOverlay (#6712)
* Add FORMAT=AUTO option. Uses PNG for semi-transparent areas, else JPG. (#4745)
LAN driver:
* remove wrong byte-swapping for big-endian hosts
MAP driver:
* change logic to detect image file when its path is not absolute
MBTiles driver:
* on opening if detecting 3 bands, expose 4 bands since there might be transparent border tiles (#6836)
* fix setting of minzoom when computing overviews out of order
* do not open .mbtiles that contain vector tiles, which are not supported by the driver
MEM driver:
* disable R/W mutex for tiny performance increase in resampled RasterIO
* add support for overviews
* add support for mask bands
MRF driver:
* bug fix in PNG and JPEG codecs
* Fixing a problem with setting NoData for MRFs generated with Create
* fix plugin building (#6498)
* rename CS variable so as to avoid build failure on Solaris 11 (#6559)
* Allow MRF to write the data file directly to an S3 bucket.
* Allow relative paths when MRF is open via the metadata string.
* Add support for spacing (unused space) between tiles. Defaults to none.
* Read a single LERC block as an MRF file.
MSG driver:
* fix incorrect georeference calculation for msg datasets (github #129)
NetCDF driver:
* add support for reading SRS from srid attribute when it exists and has content like urn:ogc:def:crs:EPSG::XXXX (#6613)
* fix crash on datasets with 1D variable with 0 record (#6645)
* fix erroneous detection of a non-longitude X axis as a longitude axis that caused a shift of 360m on the georeferencing (#6759)
* read/write US_survey_foot unit for linear units of projection
* apply 'add_offset' and 'scale_factor' on x and y variables when present, such as in files produced by NOAA from the new GOES-16 (GOES-R) satellite (github #200)
* add a HONOUR_VALID_RANGE=YES/NO open option to control whether pixel values outside of the validity range should be set to the nodata value (#6857)
* fix crash on int64/uint64 dimensions and variables, and add support for them (#6870)
NITF driver:
* add support for writing JPEG2000 compressed images with JP2OpenJPEG driver
* fix writing with JP2KAK driver (emit codestream only instead of JP2 format)
* fix setting of NBPR/NBPC/NPPBH/NPPBV fields for JPEG2000 (fixes #4322); in JP2ECW case, make sure that the default PROFILE=NPJE implies 1024 block size at the NITF level
* implement creation of RPC00B TRE for RPC metadata in CreateCopy() mode
* add support for reading&writing _rpc.txt files
* nitf_spec.xml: Add support for MTIRPB TRE in NITF image segment. Also makes minor change to BLOCKA to include default values (github #127)
* nitf_spec.xml: add IMASDA and IMRFCA TREs
* GetFileList(): Small optimization to avoid useless file probing.
NWT_GRD:
* detect short writes
OpenJPEG driver:
* support direct extracting of GeoRaster JP2-F BLOB (#6861)
PCIDSK driver:
* handle Exceptions returned from destructor and check access rights in setters (github #183)
PDF driver:
* implement loading/saving of metadata from/into PAM (#6600)
* implement reading from/writing to PAM for geotransform and projection (#6603)
* prevent crashes on dataset reopening in case of short write
PLScenes driver:
* add a METADATA open option
PostgisRaster driver:
* fix potential crash when one tile has a lower number of bands than the max of the table (#6267)
R driver:
* fix out-of-memory (oom) with corrupt R file
Raw drivers:
* prevent crashes on dataset closing in case of short write
RMF driver:
* fix wrong counter decrement that caused compressed RMF to be incorrectly decompressed (github #153)
* fix load/store inversion of cm and dm units in MTW files (github #162)
* fix reading nodata for non-double data type (github #174)
ROIPAC driver:
* add support for reading/writing .flg files (#6504)
* fix computation of line offset for multi-band BIP files, and warn if detecting a wrong file produced by GDAL >= 2.0.0 (#6591)
* fix for big endian hosts
RS2 driver:
* add support for reading RPC from product.xml
SAFE driver:
* fix handling of SLC Products by providing access to measurements as subdatasets (#6514)
Sentinel2 driver:
* add support for new "Safe Compact" encoding of L1C products (fixes #6745)
SQLite driver:
* Add read support for RasterLite2 coverages in SQLite driver
SRTMHGT driver:
* open directly .hgt.zip files
* accept filenames like NXXEYYY.SRTMGL1.hgt (#6614)
* handle files for latitude >= 50 (#6840)
VRT driver:
* add default pixel functions: real, imag, complex, mod, phase, conj, etc... for complex data types (github #141)
* avoid useless floating point values in SrcRect / DstRect (#6568)
* avoid buffer initialization in RasterIO() when possible (replace ancient and likely broken concept of bEqualAreas)
* make CheckCompatibleForDatasetIO() return FALSE on VRTDerivedRasterBands (#6599)
* VRT warp: fix issue with partial blocks at the right/bottom and dest nodata values that are different per band (#6581)
* fix performance issue when nodata set at band level and non-nearest resampling used (#6628)
* VRTComplexSource: do temp computations on double to avoid precision issues when band data type is Int32/UInt32/CInt32/Float64/CFloat64 (#6642)
* VRT derived band: add the capability to define pixel functions in Python
* CreateCopy(): detect short writes
* Fix linking error on VRTComplexSource::RasterIOInternal<float>() (#6748)
* avoid recursion in xml:VRT metadata (#6767)
* prevent 'Destination buffer too small' error when calling GetMetadata('xml:VRT') on a in-memory VRT copied from a VRT
* fix 2.1 regression that can cause crash in VRTSimpleSource::GetFileList() (#6802)
WMS driver:
* Added support for open options to WMS minidrivers
* Refactored the multi-http code to make it possible to do range requests.
* Added a minidriver_mrf, which reads from remote MRFs using range requests.
* Made the minidriver_arcgis work with an ImageService, not only MapService.
* Added static cache of server response.
* Allow tiledWMS to work in off-line mode by including the server response in the .wms file itself.
* honour GDAL_HTTP_USERAGENT config option when it is set and <UserAgent> is missing (#6825)
* WMS/WMTS: better deal with tiles with different band count (grayscale, gray+alpha, palette, rgb, rgba) (github #208)
* Make HTTPS options apply to tWMS minidriver init, better HTTP error reporting
WMTS driver:
* do not take into account WGS84BoundingBox/BoundingBox that would be the result of the densified reprojection of the bbox of the most precise tile matrix
* add TILEMATRIX / ZOOM_LEVEL open options
* accept tiles of small dimensions (github #210)
XYZ driver:
== OGR 2.2.0 - Overview of Changes ==
Core:
* Layer algebra: Add KEEP_LOWER_DIMENSION_GEOMETRIES=YES/NO option to Intersection, Union and Identity.
Default is yes, but it is set to no unless result layer is of geom type unknown.
If set to no, result features which would have lower dim geoms are skipped
if operating on two layers with same geom dim.
* Fix crash/corrupted values when running importFromWkb() on POLYGON M/POLYGON ZM geometries (#6562)
* Add OGR_WKT_PRECISION config option that defaults to 15 to select the number of decimals when outputting to WKT
* Make OGRFeature::SetField(string) accept JSon serialized arrays for the String/Integer/Integer64/RealList types; add reciprocal OGRFeature::GetFieldAsSerializedJSon() for those types
* OGRGeometryFactory::transformWithOptions(): better deal with reprojection from polar projection to WGS84, and projections crossing the antimeridian to WGS84, by splitting geometries prior to reprojection (#6705)
* LinearRing transformTo(): force last point to be identical to first one in case it is not.
* GML geometry parsing: avoid 'Cannot add a compound curve inside a compound curve' error (#6777)
* OGR SQL: fix IN filtering on MapInfo indexed columns (#6798)
* OGR SQL: add support for LIMIT and OFFSET keywords
* OGR SQL: add comparisons on date / datetime (#6810)
* OGR SQL: increase efficiency of DISTINCT operator
* OGREnvelope: change initialization to fix issue when getting MULTIPOINT(0 0,1 1) envelope (#6841)
* OGRParse: fix parsing logic to avoid false positive detection of string as datetime (#6867)
OGRSpatialReference:
* Upgrade to EPSG database v9.0 (#6772)
* OGRCT: upgrade LIBNAME of mingw and cygwin to libproj-9.dll and cygproj-9.dll to be up-to-date with proj 4.9.X (recommended method is using ./configure --with-static-proj4 instead) (#6501)
* importFromESRI(): fix import of multi line MERCATOR SRS (#6523)
* morphToESRI(): correctly compute standard_parallel_1 of Mercator(2SP) projection from scale factor of Mercator(1SP) (#6456, #4861)
* exportToProj4(): recognize explicit OSR_USE_ETMERC=NO to avoid using etmerc with proj >= 4.9.3
* importFromProj4(): do not set a AUTHORITY node for strings like '+init=epsg:XXXX +some_param=val'
* importFromProj4(): be robust with missing proj.4 epsg dictionary when importing '+init=epsg:xxxx +other_parm=value'
* AutoIdentifyEPSG(): add identification of EPSG:3995 (Arctic Polar Stereographic on WGS84) and EPSG:3031 (Antarctic Polar Stereographic on WGS84)
* OGRCoordinateTransformation: avoid potential bugs in proj.4 on NaN input
* importFromEPSG(): take into account DX,DY,DZ,RX,RY,RZ,DS columns in pcs.csv to add per-PCS TOWGS84 overrides (geotiff #52)
* Coordinate transformation: prevent unnecessary coordinate transformations (github #184, #185)
Utilities:
* ogr2ogr: do not return error on ogr2ogr --utility_version
* ogr2ogr: keep -append and -overwrite when -update follows
* ogr2ogr: fix heuristics to detect likely abs…
netbsd-srcmastr
pushed a commit
to NetBSD/pkgsrc
that referenced
this issue
Feb 24, 2018
* Add Makefile.common to prepare for geography/py-gdal
Changelog:
= GDAL/OGR 2.2.3 Release Notes =
The 2.2.3 release is a bug fix release.
== Build ==
* nmake.opt: Ensure PDB is included in release DLL if WITH_PDB requested (#7055)
== Port ==
* /vsicurl/ and derived filesystems: redirect ReadDir() to ReadDirEx() (#7045)
* /vsicurl/: enable redirection optimization on signed URLs of Google Cloud Storage. Helps for the PLScenes driver (#7067)
* /vsicurl/: fix 2.2 regression regarding retrieval of file size of FTP file (#7088)
* /vsis3/: fix to avoid invalid content to be sent if Write() writes more than 50 MB in a single call (#7058)
* /vsis3/: fix Seek(Tell(), SEEK_SET) fails if current position is not 0 (#7062)
* /vsis3/: fix support of non-ASCII characters in keys (#7143, #7146)
* Fix CPLCopyTree() that doesn't properly on MSVC 2015 (and possibly other platforms) (#7070)
== Algorithms ==
* RPC transformer: set output coordinates to HUGE_VAL when failure occurs, so that a following coordinate transformation can detect the error too (#7090)
* GDALGrid() with linear algorithm: avoid assertions/segmentation fault when GDALTriangulationFindFacetDirected() fails (#7101)
* GDALComputeProximity(): fix int32 overflow when computing distances on large input datasets (#7102)
* GDALResampleChunk32R_Gauss: fix potential out of bounds access (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4056)
== GDAL utilities ==
* gdalwarp: fix issue with GDALAdjustValueToDataType(Float32, +/- inf) that didn't preserve infinity, which affected gdalwarp -dstnodata inf (#7097)
* gdalwarp -crop_to_cutline: reduce number of iterations to find the appropriate densification (#7119)
* gdal_contour: return with non-0 code if field creation or contour generation failed (#7147)
== GDAL drivers ==
GeoRaster driver:
* fix int overflow (#6999)
GPKG driver:
* speed-up statistics retrieval on non-Byte datasets (#7096)
GRIB driver:
* if GRIB_ADJUST_LONGITUDE_RANGE config option is set to YES, adjust the longitude range to be close to [-180,180] when possible for products whose left origin is close to 180deg. (#7103)
GSAG driver:
* avoid assertion on some files with nul characters (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3726)
GTiff driver:
* make sure that -co PHOTOMETRIC=RGB overrides the color interpretation of the first 3 bands of the source datasets (#7064)
* on reading use GeogTOWGS84GeoKey to override the defaults TOWGS84 values coming from EPSG code (#7144)
* fix potential crash when reading in RGBA mode with internal libtiff (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4157)
* fix potential crash with old-jpeg compression and internal libtiff (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4180)
HTTP driver:
* do not open the underlying dataset with GDAL_OF_SHARED, to avoid later assertion
JP2KAK driver:
* fix unix build with Kakadu 7.A and later (#7048, #7081)
JPEG driver:
* Add compatibility with libjpeg-turbo 1.5.2 that honours max_memory_to_use (cf https://github.com/libjpeg-turbo/libjpeg-turbo/issues/162)
JP2OpenJPEG driver:
* Add support for openjpeg 2.3 (#7074)
netCDF driver:
* fix raster read as nodata with Byte datatype, (valid_range={0,255} or _Unsigned = True) and negative _FillValue (#7069)
* be more tolerant on the formatting of standard parallel (space separated instead of {x,y,...} syntax), and accept up to 2/1000 error on spacing to consider a regular grid, to be able to read files provided by the national weather institute of Netherlands (KNMI) (#7086)
PDF driver:
* round to upper integer when computing a DPI such that page size remains within limits accepted by Acrobat (#7083)
Sentinel2 driver:
* add support for direct opening of .zip files of new safe_compact L1C products (#7085)
VRT driver:
* avoid error being emitted when opening a VRTRawRasterBand in a .zip files (#7056)
== OGR core ==
* Fix OGR[Curve]Polygon::Intersects(OGRPoint*) to return true when point is on polygon boundary (#7091)
* importFromWkt(): fix import of GEOMETRYCOLLECTION ending with POINT EMPTY or LINESTRING EMPTY (#7128, 2.1 regression)
== OGR utilities ==
* ogrtindex: fix crash when using -f SQLITE -t_srs XXXX (#7053)
== OGR drivers ==
Amigocloud driver:
* Fixed data field types (https://github.com/OSGeo/gdal/pull/246)
* Output list of datasets if dataset id is not provided.
* Implemented waiting for job to complete on the server. This will improve write to AmigoCloud reliability.
* Updated documentation.
FileGDB driver:
* remove erroneous ODsCCreateGeomFieldAfterCreateLayer capability declaration (https://github.com/OSGeo/gdal/pull/247)
GeoJSON driver:
* ESRIJson: recognize documents that lack geometry fields (#7071)
* ESRIJson: recognize documents starting with a very long fieldAliases list (#7107)
GeoRSS driver:
* fix detection of field type (#7108)
GML driver:
* fix FORCE_SRS_DETECTION=YES effect on feature count and SRS reporting on gml files with .gfs (#7046)
* do not try to open kml files (#7061)
* GML geometry parsing: fix robustness issue with gml:PolyhedralSurface
* do not report gml:name / gml:description of features as layer metadata
GPKG driver:
* do not try to update extent on gpkg_contents after GetExtent() on a empty layer of a datasource opened in read-only mode
GTM driver:
* fix null pointer dereference in case of read error (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4047)
KML / LIBKML drivers:
MDB driver:
* fix multi-thread support (https://issues.qgis.org/issues/16039)
MITAB driver
* do not emit error if the .ind file is missing, just a debug message (#7094)
MySQL:
* fix build with MariaDB 10.2 (#7079)
OCI driver:
* initialize in multi-threaded compatible mode to fix QGIS related crashes (https://issues.qgis.org/issues/17311), and fix a few memleaks
ODS driver:
* avoid using CPLSPrintf() return directly with VSIFOpenL() as the temp buffer might be later recycled (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4050)
PLScenes driver:
* backport data/plscenesconf.json from trunk to add SkySatScene fields
Shapefile driver:
* Fix GetFeatureCount() to properly take into account spatial filter when attribute filter also in effect (#7123)
SQLite/Spatialite driver:
* don't invalidate statistics when running a PRAGMA (https://issues.qgis.org/issues/17424)
* SQLite dialect: support SQLite 3.21, and LIKE, <>, IS NOT, IS NOT NULL, IS NULL and IS operators (#7149)
XLS driver:
* workaround opening filenames with incompatible character set on Windows (https://issues.qgis.org/issues/9301)
XLSX driver:
* fix non working detection of Date/Time fields in some documents (#7073)
* fix opening of documents with x: namespace in xl/workbook.xml (#7110)
* avoid using CPLSPrintf() return directly with VSIFOpenL() as the temp buffer might be later recycled (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4050)
== SWIG bindings ==
* map GRA_Max, GRA_Min, GRA_Med, GRA_Q1 and GRA_Q3 (#7153)
== Python bindings ==
* remove 'from . import _gdal_array' line from gdal_array.py that is not necessary with normal execution of the bindings, and cause errors with PyInstaller (#7044)
= GDAL/OGR 2.2.2 Release Notes =
The 2.2.2 release is a bug fix release.
== Build ==
* Windows build: always build the PDF driver, even when none of poppler/podofo/pdfium are available, in which case it is write-only (#6938)
* Compilation fixes on Ubuntu 16.04 with explicit --std=c++03 (#6952)
== Port ==
* /vsigzip/: make Eof() detect end of stream when receive a Z_BUF_ERROR error. Fixes #6944.
* Fix memleak in VSIGSFSHandler::GetFileList()
* /vsigzip/: avoid trying to write a .gz.properties file on a /vsicurl/ file (#7016)
* Fix registration of /vsicrypt/ file system handler when compiled as a plugin (#7030)
* CPLStrtod(): parse string like '-1.#IND0000000' as NaN instead of -1 (#7031)
== Algorithms ==
* Warper: when operating on single-band, skip target pixels whose source center pixel is nodata (2.2 regression, #7001)
* Warper: avoid blocking when number of rows is 1 and NUM_THREADS > 1 (#7041). Also limit the number of threads so that each one processes at least 65536 pixels
== GDAL core ==
* GDALGCPsToGeoTransform(): add GDAL_GCPS_TO_GEOTRANSFORM_APPROX_OK=YES and GDAL_GCPS_TO_GEOTRANSFORM_APPROX_THRESHOLD=threshold_in_pixel configuration option (#6995)
* Fix issue with GDALProxyRasterBand::FlushCache() not flushing everything required (#7040)
* RawDataset::IRasterIO(): don't assume all bands are RawRasterBand
== GDAL utilities ==
* gdal2tiles.py: fix GDAL 2.2 regression where tilemapresource.xml was no longer generated (#6966)
* gdal_translate/DefaultCreateCopy(): do not destroy target file in case of failed copy wen using -co APPEND_SUBDATASET=YES (#7019)
* gdal_translate: make -b mask[,xx] use the appropriate band data type (#7028)
== GDAL drivers ==
GeoRaster driver:
* add support for GCP (#6973)
GPKG driver:
* do not error out if extent in gpkg_contents is present but invalid (#6976)
* fix opening subdatasets with absolute filenames on Windows (https://issues.qgis.org/issues/16997)
* fix possible assertion / corruption when creating a raster GeoPackage (#7022)
GSAG driver:
* fix reading issue that could cause a spurious 0 value to be read and shift all following values (#6992)
GTiff driver:
* fix reading subsampled JPEG-in-TIFF when block height = 8 (#6988)
* when reading a COMPD_CS (and GTIFF_REPORT_COMPD_CS=YES), set the name from the GTCitationGeoKey (#7011)
GTX driver:
* do not emit error when opening with GDAL_PAM_ENABLED=NO (#6996)
HF2 driver:
* fix reading tiles that are 1-pixel wide (2.1 regression, #6949)
IDRISI driver:
* Fix memleak in IDRISI Open()
ISIS3 driver:
* make sure that -co USE_SRC_HISTORY=NO -co ADD_GDAL_HISTORY=NO results in remove of History section (#6968)
* fix logic to initialize underlying GeoTIFF file in IWriteBlock(), and implement Fill() (#7040)
JPEG2000 driver:
* Fix build failure in C++03 mode on Jasper inclusion in RHEL 6 (#6951)
* Fix build failure with recent Jasper that no longer define uchar
JP2OpenJPEG driver:
* Add support for building against OpenJPEG 2.2 (#7002)
* fix performance issues with small images with very small tile size, such as some Sentinel2 quicklooks (#7012)
* properly use the opj_set_decode_area() API (#7018)
netCDF driver:
* avoid vector vs raster variable confusion that prevents reading Sentinel3 datasets, and avoid invalid geolocation array to be reported (#6974)
PDF driver:
* add support for Poppler 0.58 (#7033)
PDS driver:
* fix parsing of labels with nested list constructs (2.2 regression, #6970)
SRTMHGT driver:
* recognizes the .hgt.gz extension (#7016)
VRT driver:
* avoid stack buffer read overflow with complex data type and scale = 0. (oss-fuzz#2468)
* fix uninitialized buffer in areas without sources when using non pixel packed spacing (#6965)
* Warped VRT: correctly take into account cutline for implicit overviews; also avoid serializing a duplicate CUTLINE warping options in warped .vrt (#6954)
* Warped VRT: fix implicit overview when output geotransform is not the same as the transformer dst geotransform (#6972)
* fix IGetDataCoverageStatus() in the case of non-simple sources, which unbreaks gdalenhance -equalize (#6987)
== OGR core ==
* OGRCompondCurve::addCurveDirectly(): try reversing non contiguous curves (for GML/NAS)
* OGR API SPY: fix the way we map dataset handles to variable name, to avoid invalid reuses of variable still active
* OGRParseDate(): avoid false-positive time detection, in particular for GeoJSON (#7014)
* OGRCurve::get_isClosed(): do not take into account M component (#7017)
* OGRLineString::setPoint() and addPoint() with a OGRPoint* argument. properly takes into account ZM, Z and M dimensions (#7017)
== OGRSpatialReference ==
* Fix OGRSpatialReference::IsSame() to return FALSE when comparing EPSG:3857 (web_mercator) and EPSG:3395 (WGS84 Mercator) (#7029)
* importFromProj4(): implement import of Hotine Oblique Mercator Two Points Natural Origin, and fix OGRSpatialReference::Validate() for that formulation (#7042)
== OGR utilities ==
* ogr2ogr: fix small memory leak when using -limit switch
* ogr2ogr: make -f GMT work again (#6993)
== OGR drivers ==
AVCBin driver:
* fix 2.1 regression regarding attributes fetching (#6950)
DXF driver:
* fix reading files where INSERT is the last entity and DXF_MERGE_BLOCK_GEOMETRIES is false (#7006)
* avoid segfault when creating a DXF file with only the 'blocks' layer (#6998)
* fix potential null pointer dereference in PrepareLineStyle (#7038)
GeoJSON driver:
* fix 2.2 regression regarding lack of setting the FeatureCollection CRS on feature geometries (fixes https://github.com/r-spatial/sf/issues/449#issuecomment-319369945)
* prevent infinite recursion when reading geocouch spatiallist with properties.properties.properties (#7036)
GML driver:
* fix field type detection logic to avoid a field with xsi:nil=true to be identified as a string (#7027)
* JPGIS FGD v4: fix logic so that coordinate order reported is lon/lat (https://github.com/OSGeo/gdal/pull/241)
GMLAS driver:
* fix memleak in swe:DataRecord handling
* avoid false-positive warning about attribute found in document but ignored according to configuration, when the attribute is actually not present in the document but has a default value (#6956)
* (quick-and-not-prefect) fix to avoid a <xs:any> to prevent all other fields from being set (#6957)
* get the srsName when set on the srsName of the gml:pos of a gml:Point (#6962)
* CityGML related fixes: better take into account .xsd whose namespace prefix is not in the document, fix discovery of substitution elements, handle gYear data type (#6975)
GPKG driver:
* fix feature count after SetFeature() with sqlite < 3.7.8 (#6953)
* do not take into account gpkg_contents for the extent of vector layers when it is present but invalid (#6976)
* fix logic to detect FID on SQL result layer when several FID fields are selected (#7026)
KML / LIBKML drivers:
* read documents with an explicit kml prefix (#6981)
MSSQLSpatial driver:
* fix issues on x64 (#6930)
* properly format DELETE statement with schema (#7039)
OCI driver:
* check view with unquoted table name (#5552)
OpenFileGDB driver:
* properly read GeneralPolygon with M component whose all values are set to NaN (#7017)
OSM driver:
* increase string buffer for osm XML files (#6964)
Shapefile driver:
* use VSIStatL() in Create() to properly work with /vsimem/ directories (#6991)
* fix regression affecting GDAL 2.1.3 or later, 2.2.0 or later, when editing the last shape of a file and growing it, and then appending a new shape, which resulted in corruption of both shapes (#7031)
SQLite/Spatialite driver:
* escape integer primary key column name on table creation (#7007)
== C# bindings ==
* Implement RasterIO extensions (#6915)
== Python bindings ==
* fix reference count issue on gdal.VSIStatL() when it returns None, that can cause Python crashes if None refcount drops to zero
* add C-API to create driver instances for use in SWIG (https://trac.osgeo.org/osgeo4w/ticket/466)
== Security oriented fixes ==
Note: this is only a very partial backport of more extensive fixes done in GDAL trunk. Credit to OSS-Fuzz for all of them. (oss-fuzz#XXXX is a shortcut to
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=XXXX)
* NAS: avoid assertion / null ptr deref and possibly other failures on corrupted files. (oss-fuzz#2366, oss-fuzz#2580)
* NTF: avoid null ptr deref. Related to https://oss-fuzz.com/v2/testcase-detail/6527758899347456
* VSIMemHandle::Read(): avoid unwanted unsigned int overflow that cause a later heap buffer overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/5227283000328192.
* MBTiles: fix use after free. (oss-fuzz#2388)
* netCDF: fix stack buffer overflow when building vector layers from netCDF variables if they don't have the expected number of dimensions. (oss-fuzz#2400)
* DXF: prevent infinite loop / excessive memory allocation on truncated file. (#oss-fuzz#2403)
* DXF: avoid excessive memory allocation. (oss-fuzz#2444)
* XYZ: fix write heap-buffer-overflow (oss-fuzz#2426)
* GTiff: make IGetDataCoverageStatus() properly set the current TIFF directory, and only implement it for 'normal' bands. To avoid heap buffer overflows. (oss-fuzz#2438)
* GTiff: avoid null pointer derefrence when requested implicit overviews on corrupted JPEG-compressed file. (oss-fuzz#2441)
* GTiff: avoid potential issue on corrupted files (oss-fuzz#2481)
* GTiff: don't override member nBlocksPerRow member of GTiffJPEGOverviewBand with unrelated value, in case of single-strip case. (oss-fuzz#3020)
* PCIDSK: for band interleave channel, correctly use pixel_offset in case it is different from pixel_size. (oss-fuzz#2440)
* MITAB: fix skipping of blank lines when reading MIF geometries, and avoid potential very loop loop. Fixes https://oss-fuzz.com/v2/testcase-detail/4516730841858048
* CPLKeywordParser: avoid potential heap buffer overflow read. (oss-fuzz#2706)
* CPLKeywordParser: avoid potential infinite loop. Fixes https://oss-fuzz.com/v2/testcase-detail/5733173726019584
* morphFromESRI(): fixes heap-after-free uses. (oss-fuzz#2864)
* OGR_VRT: fix null pointer dereference on GetExtent() on an invalid layer. (oss-fuzz#3017)
* OGR_VRT: avoid crash on cyclic VRT. (oss-fuzz#3123)
* OSM: avoid potential write heap buffer overflow on corrupted PBF files. (oss-fuzz#3022)
* FAST: fix potential read heap buffer overflow. (oss-fuzz#3025)
* CPG: avoid null pointer dereference on failed open. (oss-fuzz#3136)
= GDAL/OGR 2.2.1 Release Notes =
The 2.2.1 release is a bug fix release.
== Build ==
* fix compilation without BIGTIFF_SUPPORT (#6890)
* configure: detect if std::isnan() is available. Helps compilation on some MacOSX setups, combined together --without-cpp11. Refs https://github.com/macports/macports-ports/pull/480
* fix compilation against ancient Linux kernel headers (#6909)
* fix detection of 64bit file API with clang 5 (#6912)
* configure: use .exe extension when building with mingw64* toolchains (fixes #6919)
* mongoDB: compilation fix on Windows
== Port ==
* /vsicurl/: fix occasional inappropriate failures in Read() with some combinations of initial offset, file size and read request size (#6901)
* Add a VSICurlClearCache() function (bound to SWIG as gdal.VSICurlClearCache()) to be able to clear /vsicurl/ related caches (#6937)
== Algorithms ==
* GDALRasterize(): avoid hang in some cases with all_touched option (#5580)
* gdal_rasterize: fix segfault when rasterizing onto a raster with RPC (#6922)
== GDAL utilities ==
* ogr_merge.py: fix '-single -o out.shp in.shp' case (#6888)
== GDAL drivers ==
AIGRID driver:
* fix handling on raw 32-bit AIG blocks
ENVISAT driver:
* fix 2.2 regression in initialization of members of MerisL2FlagBand. (#6929)
GeoRaster driver:
* Fix memory allocation failure (#6884)
* add support for JP2-F in BLOB compression (corrections on geo-reference) (#6861)
GPKG driver:
* avoid corruption of gpkg_tile_matrix when building overviews, down to a level where they are smaller than the tile size (#6932)
GTIFF driver:
* Internal libtiff: fix libtiff 4.0.8 regression regarding creating of single strip uncompressed TIFF files (#6924)
netCDF driver:
* add support for radian and microradian units for geostationnary projection (https://github.com/OSGeo/gdal/pull/220)
NWT_GRC driver:
* Fix handling of alpha values in GRC color table (#6905)
* Handle case of 0-len GRC class names (#6907)
VRT driver:
* speed-up SerializeToXML() in case of big number of bands
XYZ driver:
* fix 2.2 regression where the driver hangs on some dataset with missing samples (#6934)
== OGR utilities ==
* ogr2ogr/GDALVectorTranslate(): fix crash when using -f PDF -a_srs (#6920)
== OGR drivers ==
GeoJSON driver:
* ESRIJson: avoid endless looping on servers that don't support resultOffset (#6895)
* ESRIJson: use 'latestWkid' in priority over 'wkid' when reading 'spatialReference' (https://github.com/OSGeo/gdal/pull/218)
* GeoJSON writer: accept writing ZM or M geometry by dropping the M component (#6935)
GPKG driver:
* make driver robust to difference of cases between table_name in gpkg_contents/gpkg_geometry_columns and name in sqlite_master (#6916)
MITAB driver:
* recognize Reseau_National_Belge_1972 / EPSG:31370 on writing (#6903)
MySQL driver:
* fix compilation issue with Arch Linux and mariadb 10.1.23 (fixes #6899)
PG driver:
* do not be confused by a 'geometry' table in a non-PostGIS enabled database (#6896)
PLScenes:
* remove support for V0. Deprecate V1 API. Only Data V1 is supported ( #6933)
== Perl bindings ==
* Backport the fix to #6142 Install man page according to GDALmake.opt if INSTALL_BASE is set.
* always return something from non-void functions (#6898)
== Python bindings ==
* Accept callback = 0 since SWIG generates it as the default argument of BandRasterIONumPy(). Fixes https://github.com/OSGeo/gdal/pull/219
* Fix 2.2 regression preventing use of callback function in Band.ComputeStatistics() (#6927)
== Security oriented fixes ==
Note: this is only a very partial backport of more extensive fixes done in GDAL trunk. Credit to OSS-Fuzz for all of them (some have been found locally, so no related ticket)
* Fix CPLErrorSetState to ensure it does not write beyond DEFAULT_LAST_ERR_MSG_SIZE and correctly null-terminates last message. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1529.
* Open() and Stat() methods of a number of virtual file systems: check that the filename starts with the full prefix. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1543.
* VRT pixel functions: fix crash with 'complex' when source count is < 2. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1549
* OpenAIR: fix potential out-of-buffer read if we need to ingest 30000 bytes
* Several fixes in importFromWkb() and importFromWkt()
* GDALDataset and GDALRasterBand::ReportError(): fix crash if dataset name has a % character
* NASAKeywordHandler::SkipWhite(): fix out of bounds read
* MITAB: ParseTABFileFields(): fix out of bounds read.
* MITAB: ParseMIFHeader(): fix memory leak and out-of-bounds read on corrupted file
* MITAB: ParseMIFHeader(): fix memory leaks on corrupted files
* MITAB: avoid potentially veryyyy long loop when stroking arcs. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1644
* MITAB: avoid heap-buffer-overflow in MITABCoordSys2TABProjInfo(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1653
* OSARDataset::Open(): fix crash if pOpenInfo->fpL == NULL. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1565
* OGRESRIJSONReadPolygon(): fix crash in error code path
* DXF: prevent null ptr deref and out-of-bounds read on corrupted file
* DXF: TranslateSPLINE(): sanitize integer values read to avoid int overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1843
* KML::unregisterLayerIfMatchingThisNode(): use memmove() instead of memcpy(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1580
* KML: fix crash on weird structure with recursively empty folders. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1591
* KML: fix null ptr dereference on corrupted file. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1634
* OGRCurveCollection::importBodyFromWkb(): fix potential crash if the curve in the collection has not the same dimension has the collection dimension. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1584
* OGRCompoundCurve::importFromWkb(): avoid potential stack overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/5192348843638784
* TIGER: fix potential stack buffer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1586 and 2020
* TIGER: avoid potential infinite looping. Fixes https://oss-fuzz.com/v2/testcase-detail/4724815883665408
* VFK: avoid out-of-bounds read. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1596 and 2074
* CPLHexToBinary(): avoid reading outside of hex2char array on on-ASCII input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1606
* OGR PDS: avoid int32 overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=160
* GeoRSS: fix null pointer dereference on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1611.
* VSIArchiveFilesystemHandler::SplitFilename(): improve performance. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1597
* OGRGeometryFactory::organizePolygons(): fix crash on empty polygons. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1620
* JML: fix null pointer dereference on invalid files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1622
* Shape: prevent null ptr deref on truncated MultiPointM geometry. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1638
* /vsisubfile/: avoid Tell() to return negative values. And make VSIIngestFile() more robust to unsigned overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1642
* GTM: avoid useless recursive opening of files when provided with a gzip-compressed input. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1650
* GTiff: fix heap-buffer-overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1677
* GTiff: avoid heap-buffer-overfow on corrupted State Plane citation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2050
* GTiff: avoid potential stack buffer overflow on corrupted Imagine citation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2105
* GTiff: prevent heap overflow and fix reading of multi-band band-interleaved 16/24-bit float datasets. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2264
* GTiff: fix potential infinite loop when parsing some 24-bit float denormalized numbers. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2315
* Internal libjson-c: fix stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1682
* ILI1/ILI2: fix null pointer dereference when opening filename ','. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1689
* ILI1: fix various crashes on corrupted files (including, but not limited to, https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1760, 1784, 1926)
* ILI2: use proper cast operator. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1813
* ILI2: fix null pointer dereference on corrupted files. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1993
* ILI2: fix crash due to unhandled exception. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2191
* morphFromESRI(): fix heap-use-after-free issue. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1690
* morphFromESRI(): prevent potential null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1783 and 1867
* SEGUKOOA: fix inversion of leap year that caused index-out-of-bound reading on day 366 of leap years (2.2 regression). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1698
* CPLParseXMLString(): make it error out on invalid XML file even under CPLTurnErrorIntoWarning() mode. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1703.
* GML / NAS: fix memory leak in error code path, and potential heap-buffer-read-overflow
* NTF: fix various issues: heap & stack buffer-overflow, null ptr derefs, memory leaks. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1819 , 1820 , 1823, 1833, 1854, 1862, 1910, 1931, 1961, 1990, 1995, 1996, 2003, 2033, 2052, 2077, 2084, 2103, 2130, 2135, 2146, 2166, 2185, 2187, https://oss-fuzz.com/v2/testcase-detail/4696417694121984
* OGRCreateFromMultiPatch(): avoid assertion on NaN coordinates. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1930
* GXF: validate nGType to avoid later out-of-bound read in GXFReadRawScanlineFrom(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1975
* GXF: fix int overflow and avoid excessive memory allocation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2207
* DGN: prevent heap-buffer-overflow read. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1988
* COSAR: fix leak of file descriptor. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2019
* ISO8211: prevent stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2022
* WEBP: prevent int32 overflow and too large memory allocation. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2059
* IRIS: fix heap-buffer-overflow in some cases of nDataTypeCode. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2069
* E00GRID: avoid heap and stack buffer overflows. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2090 , 2182, 2237, 2327
* VICAR: fix null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2116
* VICAR: avoid use-after-free and heap-buffer-overflow. Fixes https://oss-fuzz.com/v2/testcase-detail/4825577427042304
* VICAR: fix potential endless loop on broken files. Fixes https://oss-fuzz.com/v2/testcase-detail/6261508172414976
* REC: fix nullptr deref
* REC: fix potential stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2165
* GDALGetJPEG2000Structure() / DumpGeoTIFFBox(): fix memory leak.
* DumpGeoTIFFBox(): reject GeoJP2 boxes with a TIFF with band_count > 1
* DumpJPK2CodeStream(): avoid potentially very long loop
* GDALGetJPEG2000Structure(): avoid bad performance on corrupted JP2 boxes. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2214
* GPKG: fix potential heap-buffer overflow in GPkgHeaderFromWKB(). Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2150
* GPKG: fix potential null ptr deref. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2240
* GPKG: avoid potential division by zero. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2253
* SEGUKOOA: prevent read beyond end of buffer. (#6921)
* SRP: avoid potential stack buffer overflow and excessive memory allocation/processing time
* CPLUnixTimeToYMDHMS(): avoid potential infinite loop. Fixes https://oss-fuzz.com/v2/testcase-detail/4949567697059840
* Selafin: fix double frees. Fixes https://oss-fuzz.com/v2/testcase-detail/6429713822121984
* CEOS: fix heap buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2259
* CEOS: fix memleak in error code path. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2279
* FAST: avoid null pointer dereference. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2290
* netCDF: avoid stack buffer overflow. Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2302
GDAL/OGR 2.2.0 Release Notes =
== In a nutshell... ==
* New GDAL/raster drivers:
- DERIVED driver: read-support. Expose subdatasets in a a new metadata domain, called DERIVED_SUBDATASETS
- JP2Lura driver: read/create support for JPEG-2000 images using Luratech JP2 Library
- PRF: add read-only support for PHOTOMOD PRF file format driver (github #173)
- RRASTER driver: read-support .grd/.gri files handled by the R 'raster' package (#6249)
* New OGR/vector drivers:
- CAD driver: read support for DWG R2000 files (GSoC 2016 project)
- DGNv8 driver: read-write support for DGN 8.0 format (using Teigha ODA libraries)
- GMLAS driver: read-write support. XML/GML driver driven by Application Schemas.
* New utility script: ogrmerge.py to merge several vector datasets into a single one
* New /vsigs/ and /vsigs_streaming/ virtual file systems to read Google Cloud Storage non-public files
* Significantly improved drivers:
- NWT_GRD: write support (#6533)
- FileGDB/OpenFileGDB: add support to read curve geometries (#5890)
- VRT derived band: add the capability to define pixel functions in Python
- Add read support for RasterLite2 coverages in SQLite driver
- GPKG: implement tiled gridded elevation data extension
- ISIS3: add write support and improve read support
* RFC 63: Add GDALRasterBand::GetDataCoverageStatus() and implement it in GTiff and VRT drivers
https://trac.osgeo.org/gdal/wiki/rfc63_sparse_datasets_improvements
* RFC 64: Triangle, Polyhedral surface and TIN
https://trac.osgeo.org/gdal/wiki/rfc64_triangle_polyhedralsurface_tin
==> this RFC introduces potential backward incompatible behaviour.
Consult MIGRATION_GUIDE.txt
* RFC 66: OGR random layer read/write capabilities
https://trac.osgeo.org/gdal/wiki/rfc66_randomlayerreadwrite
* RFC 67: add null field state for OGR features, in addition to unset fields
https://trac.osgeo.org/gdal/wiki/rfc67_nullfieldvalues
==> this RFC introduces potential backward incompatible behaviour.
Consult MIGRATION_GUIDE.txt
* Upgrade to EPSG database v9.0 (#6772)
* Python bindings: Global Interpreter Lock (GIL) released before entering GDAL native code (for all, in GDAL module and a few ones in ogr like ogr.Open())
* Continued major efforts on sanitization of code base
* Remove bridge and vb6 bindings (#6640)
* GNM built by default
== Installed files ==
* Removed: data/s57attributes_aml.csv data/s57attributes_iw.csv data/s57objectclasses_aml.csv data/s57objectclasses_iw.csv
* Added plscenesconf.json, gmlasconf.xsd
== Backward compatibility issues ==
See MIGRATION_GUIDE.TXT
== GDAL/OGR 2.2.0 - General Changes ==
Build(Unix):
* improve detection of packaged libfyba (SOSI) --with-sosi, as in Ubuntu 16.04 (#6488)
* Sort files in static library to make the build reproducible (#6520)
* fix libqhull include path when it is /usr/local/include/libqhull (#6522)
* FileGDB: compilation fix on Linux in C++11 mode
* configure: make pdfium detection not fail if there are just warnings. And make configure fail if --with-pdfium was required but failed (#6653)
* Make ./configure --with-xerces fail if not found
* Don't install script documentation in INST_BIN (github #157)
* configure: auto-detect webp without requiring explicit --with-webp
* configure: use pkg-config for HDF5 detection so that works out of the box on recent Ubuntu
* auto-detect JDK 8 on Ubuntu
* MDB: allow libjvm.so to be dlopen'd with --with-jvm-lib=dlopen (Unix only, github #177)
* configure: delete temporary directories on the mac
* configure: make sure --with-macosx-framework is correctly defined
* configure: error out if --with-ld-shared is specified (#6769)
* configure: remove bashism.
* configure: fix --without-mrf (#6811)
* configure: take into account CXXFLAGS and LDFLAGS in a few more cases (cryptopp, podofo, libdap)
* Vagrant: all lxc and Hyper-V provider support; use vagrant-cachier for package caching
* configure: update DWG support to work with Teigha libraries
* Internal libgeotiff: hide symbols in --with-hide-internal-symbols mode
* Shape: do not export Shapelib symbols for builds --with-hide-internal-symbols (#6860)
Build(Windows):
* Try to avoid confusion between libqhull io.h and mingw own io.h (#6590)
* update script to generate most recent Visual C++ project files (#6635)
* fix broken and missing dependencies in makefile.vc
* add a way to use an external zlib (github #171)
* Rename makegdal_gen.bat to generate_vcxproj.bat
* generate_vcxproj.bat: Set correct value of PlatformToolset property based on specified Visual C++ version. Add NMAKE command line parameters: MSVC_VER based on specified Visual C++ version, DEBUG=1 WITH_PDB=1 to Debug build configuration.
* generate_vcxproj.bat: generate project for autotest/cpp (Ticket #6815)
* Add WIN64=1 to NMAKE command line options.
* Add HDF4_INCLUDE option (#6805)
* Add MSVC compiler option /MP to build with parallel processes.
* Add ZLIB_LIB missing from EXTERNAL_LIBS
Build(all):
* make Xerces 3.1 the minimal version
* drop support for PostgreSQL client library older than 7.4, or non security maintained releases older than 8.1.4, 8.0.8, 7.4.13, 7.3.15
== GDAL 2.2.0 - Overview of Changes ==
Port:
* Export VSICreateCachedFile() as CPL_DLL so as to enable building JP2KAK as a plugin
* Added possibility to find GDAL_DATA path using INST_DATA definition without execution GDALAllRegister if GDAL_DATA placed in version named directory on Linux (#6543)
* Unix filesystem: make error message about failed open to report the filename (#6545)
* File finder: Remove hardcoded find location (/usr/local/share/gdal) (#6543)
* Win32 filesystem handler: make Truncate() turn on sparse files when doing file extension
* Add VSIFGetRangeStatusL() and VSISupportsSparseFiles()
* GDAL_NO_HARDCODED_FIND compilation option (#6543,#6531) to block file open calls (for sandboxed systems)
* Add VSIMallocAligned(), VSIMallocAlignedAuto() and VSIFreeAligned() APIs
* /vsizip / /vsitar: support alternate syntax /vsitar/{/path/to/the/archive}/path/inside/the/tar/file so as not to be dependent on file extension and enable chaining
* Optimize opening of /vsitar/my.tar.gz/my_single_file
* /vsizip/ : support creating non-ASCII filenames inside a ZIP (#6631)
* VSI Unix file system: fix behaviour in a+ mode that made MRF caching not work on Mac and other BSD systems
* Fix deadlock at CPLWorkerThreadPool destruction (#6646)
* Windows: honour GDAL_FILENAME_IS_UTF8 setting to call LoadLibraryW() (#6650)
* CPLFormFilename(): always use / path separator for /vsimem, even on Windows
* /vsimem/: add trick to limit the file size, so as to be able to test how drivers handle write errors
* /vsimem/: fix potential crash when closing -different- handles pointing to the same file from different threads (#6683)
* CPLHTTPFetch(): add MAX_FILE_SIZE option
* CPLHTTPFetch(): add a CAINFO option to set the path to the CA bundle file. As a fallback also honour the CURL_CA_BUNDLE and SSL_CERT_FILE environment variables used by the curl binary, which makes this setting also available for /vsicurl/, /vsicurl_streaming/, /vsis3/ and /vsis3_streaming/ file systems (#6732)
* CPLHTTPFetch(): don't disable peer certificate verification when doing https (#6734)
* CPLHTTPFetch(): cleanly deal with multiple headers passed with HEADERS and separated with newlines
* CPLHTTPFetch(): add a CONNECTTIMEOUT option
* CPLHTTPFetch(): add a GDAL_HTTP_HEADER_FILE / HEADER_FILE option.
* CPLHTTPSetOptions(): make redirection of POST requests to still be POST requests after redirection (#6849)
* /vsicurl/: take CPL_VSIL_CURL_ALLOWED_EXTENSIONS into account even if GDAL_DISABLE_READDIR_ON_OPEN is defined (#6681)
* /vsicurl/: get modification time if available from GET or HEAD results
* /vsis3/: add a AWS_REQUEST_PAYER=requester configuration option (github #186)
* CPLParseXMLString(): do not reset error state
* Windows: fix GetFreeDiskSpace()
* Fix GetDiskFreeSpace() on 32bit Linux to avoid 32bit overflow when free disk space is above 4 GB (#6750)
* Fix CPLPrintUIntBig() to really print a unsigned value and not a signed one
* Add CPL_HAS_GINT64, GINT64_MIN/MAX, GUINT64_MAX macros (#6747)
* Add CPLGetConfigOptions(), CPLSetConfigOptions(), CPLGetThreadLocalConfigOptions() and CPLSetThreadLocalConfigOptions() to help improving compatibility between osgeo.gdal python bindings and rasterio (related to https://github.com/mapbox/rasterio/pull/969)
* MiniXML serializer: fix potential buffer overflow.
Core:
* Proxy dataset: add consistency checks in (the unlikely) case the proxy and underlying dataset/bands would not share compatible characteristics
* GDALPamDataset::TryLoadXML(): do not reset error context when parsing .aux.xml
* PAM/VRT: only take into account <Entry> elements when deserializing a <ColorTable>
* GDALCopyWords(): add fast copy path when src data type == dst data type == Int16 or UInt16
* GetVirtualMemAuto(): allow USE_DEFAULT_IMPLEMENTATION=NO to prevent the default implementation from being used
* Nodata comparison: fix test when nodata is FLT_MIN or DBL_MIN (#6578)
* GetHistogram() / ComputeRasterMinMax() / ComputeStatistics(): better deal with precision issues of nodata comparison on Float32 data type
* Fast implementation of GDALRasterBand::ComputeStatistics() for GDT_Byte and GDT_UInt16 (including use of SSE2/AVX2)
* Driver manage: If INST_DATA is not requested, do not check the GDAL_DATA variable.
* Make sure that GDALSetCacheMax() initialize the raster block mutex (#6611)
* External overview: fix incorrect overview building when generating from greater overview factors to lower ones, in compressed and single-band case (#6617)
* Speed-up SSE2 implementation of GDALCopy4Words from float to byte/uint16/int16
* Add SSE2 and SSSE3 implementations of GDALCopyWords from Byte with 2,3 or 4 byte stride to packed byte
* GDALCopyWords(): SSE2-accelerated Byte->Int32 and Byte->Float32 packed conversions
* Fix GDALRasterBand::IRasterIO() on a VRT dataset that has resampled sources, on requests such as nXSize == nBufXSize but nXSize != dfXSize
* GDALRasterBand::IRasterIO(): add small epsilon to floating-point srcX and srcY to avoid some numeric precision issues when rounding.
* Add GDALRasterBand::GetActualBlockSize() (#1233)
* Fix potential deadlock in multithreaded writing scenarios (#6661)
* Fix thread-unsafe behaviour when using GetLockedBlock()/MarkDirty()/DropLock() lower level interfaces (#6665)
* Fix multi-threading issues in read/write scenarios (#6684)
* Resampled RasterIO(): so as to get consistent results, use band datatype as intermediate type if it is different from the buffer type
* Add GDALIdentifyDriverEx() function (github #152)
* GDALOpenInfo: add a papszAllowedDrivers member and fill it in GDALOpenEx()
* GDALDefaultOverviews::BuildOverviews(): improve progress report
* Average and mode overview/rasterio resampling: correct source pixel computation due to numerical precision issues when downsampling by an integral factor, and also in oversampling use cases (github #156)
* Overview building: add experimental GDAL_OVR_PROPAGATE_NODATA config option that can be set to YES so that a nodata value in source samples will cause the target pixel to be zeroed. Only implemented for AVERAGE resampling right now
* GDALValidateOptions(): fix check of min/max values
* GMLJP2 v2: update to 2.0.1 corrigendum and add capability to set gml:RectifiedGrid/gmlcov:rangeType content. Set SRSNAME_FORMAT=OGC_URL by default when converting to GML. Add gml:boundedBy in gmljp2:GMLJP2RectifiedGridCoverage
* GMLJP2 v2: ensure KML root node id unicity when converting annotations on the fly. When generating GML features, make sure that PREFIX and TARGET_NAMESPACE are unique when specifying several documents.
Algorithms:
* RPC transformer: speed-up DEM extraction by requesting and caching a larger buffer, instead of doing many queries of just a few pixels that can be costly with VRT for example
* GDALDeserializeRPCTransformer(): for consistency, use the same default value as in GDALCreateRPCTransformer() if <PixErrThreshold> is missing (so use 0.1 instead of 0.25 as before)
* TPS solver: when Armadillo fails sometimes, fallback to old method
* GDALCreateGenImgProjTransformer2(): add SRC_APPROX_ERROR_IN_SRS_UNIT, SRC_APPROX_ERROR_IN_PIXEL, DST_APPROX_ERROR_IN_SRS_UNIT, DST_APPROX_ERROR_IN_PIXEL, REPROJECTION_APPROX_ERROR_IN_SRC_SRS_UNIT and REPROJECTION_APPROX_ERROR_IN_DST_SRS_UNIT transformer options, so as to be able to have approximate sub-transformers
* Fix GDAL_CG_Create() to call GDALContourGenerator::Init() (#6491)
* GDALContourGenerate(): handle the case where the nodata value is NaN (#6519)
* GDALGridCreate(): fix hang in multi-threaded case when pfnProgress is NULL or GDALDummyProgress (#6552)
* GDAL contour: fix incorrect oriented contour lines in some rare cases (#6563)
* Warper: multiple performance improvements for cubic interpolation and uint16 data type
* Warper: add SRC_ALPHA_MAX and DST_ALPHA_MAX warp options to control the maximum value of the alpha channel. Set now to 65535 for UInt16 (and 32767 for Int16), or to 2^NBITS-1. 255 used for other cases as before
* Warper: avoid undefined behaviour when doing floating point to int conversion, that may trigger exception with some compilers (LLVM 8) (#6753)
* OpenCL warper: update cubicConvolution to use same formula as CPU case (#6664)
* OpenCL warper: fix compliance to the spec. Fix issues with NVidia opencl (#6624, #6669)
* OpenCL warper: use GPU based over CPU based implementation when possible, use non-Intel OpenCL implementation when possible. Add BLACKLISTED_OPENCL_VENDOR and PREFERRED_OPENCL_VENDOR to customize choice of implementation
Utilities:
* gdalinfo -json: fix order of points in wgs84Extent.coordinates (github #166)
* gdalwarp: do not densify cutlines by default when CUTLINE_BLEND_DIST is used (#6507)
* gdalwarp: when -to RPC_DEM is specified, make -et default to 0 as documented (#6608)
* gdalwarp: improve detection of source alpha band and auto-setting of target alpha band. Automatically set PHOTOMETRIC=RGB on target GeoTIFF when input colors are RGB
* gdalwarp: add a -nosrcalpha option to wrap the alpha band as a regular band and not as the alpha band
* gdalwarp: avoid cutline densification when no transform at all is involved (related to #6648)
* gdalwarp: fix failure with cutline on a layer of touching polygons (#6694)
* gdalwarp: allow to set UNIFIED_SRC_NODATA=NO to override the default that set it to YES
* gdalwarp: fix -to SRC_METHOD=NO_GEOTRANSFORM -to DST_METHOD=NO_GEOTRANSFORM mode (#6721)
* gdalwarp: add support for shifting the values of input DEM when source and/or target SRS references a proj.4 vertical datum shift grid
* gdalwarp: fix crash when -multi and -to RPC_DEM are used together (#6869)
* gdal_translate: when using -projwin with default nearest neighbour resampling, align on integer source pixels (#6610)
* gdal_translate & gdalwarp: lower the default value of GDAL_MAX_DATASET_POOL_SIZE to 100 on MacOSX (#6604)
* gdal_translate: avoid useless directory scanning on GeoTIFF files
* gdal_translate: make "-a_nodata inf -ot Float32" work without warning
* gdal_translate: set nodata value on vrtsource on scale / unscale / expand cases (github #199)
* GDALTranslate(): make it possible to create a anonymous target VRT from a (anonymous) memory source
* gdaldem: speed-up computations for src type = Byte/Int16/UInt16 and particularly for hillshade
* gdaldem hillshade: add a -multidirectional option
* GDALDEMProcessing() API: fix -alt support (#6847)
* gdal_polygonize.py: explicitly set output layer geometry type to be polygon (#6530)
* gdal_polygonize.py: add support for -b mask[,band_number] option to polygonize a mask band
* gdal_rasterize: make sure -3d, -burn and -a are exclusive
* gdal_rasterize: fix segfaults when rasterizing into an ungeoreferenced raster, or when doing 'gdal_rasterize my.shp my.tif' with a non existing my.tif (#6738)
* gdal_rasterize: fix crash when rasterizing empty polygon (#6844)
* gdal_grid: add a smoothing parameter to invdistnn algorithm (github #196)
* gdal_retile.py: add a -overlap switch
* gdal2tiles.py: do not crash on empty tiles generation (#6057)
* gdal2tiles.py: handle requested tile at too low zoom to get any data (#6795)
* gdal2tiles: fix handling of UTF-8 filenames (#6794)
* gdal2xyz: use %d formatting for Int32/UInt32 data types (#6644)
* gdal_edit.py: add -scale and -offset switches (#6833)
* gdaltindex: emit warning in -src_srs_format WKT when WKT is too large
* gdalbuildvrt: add a -oo switch to specify dataset open options
Python samples:
* add validate_cloud_optimized_geotiff.py
* add validate_gpkg.py
Multi-driver:
* Add GEOREF_SOURCES open option / GDAL_GEOREF_SOURCES config. option to all JPEG2000 drivers and GTiff to control which sources of georeferencing can be used and their respective priority
AIGRID driver:
* fix 2.1.0 regression when reading statistics (.sta) file with only 3 values, and fix <2.1 behaviour to read them in LSB order (#6633)
AAIGRID driver:
* auto-detect Float64 when the nodata value is not representable in the Float32 range
ADRG driver:
* handle north and south polar zones (ZNA 9 and 18) (#6783)
ASRP driver:
* fix georeferencing of polar arc zone images (#6560)
BPG driver:
* declare GDALRegister_BPG as C exported for building as a plugin (#6693)
DIMAP driver:
* DIMAP: for DIMAP 2, read RPC from RPC_xxxxx.XML file (#6539)
* DIMAP/Pleiades metadata reader: take into tiling to properly shift RPC (#6293)
* add support for tiled DIMAP 2 datasets (#6293)
DODS driver:
* fix crash on URL that are not DODS servers (#6718)
DTED driver:
* correctly create files at latitudes -80, -75, -70 and -50 (#6859)
ECW driver:
* Add option ECW_ALWAYS_UPWARD=TRUE/FALSE to work around issues with "Downward" oriented images (#6516).
ENVI driver:
* on closing, pad image file with trailing nul bytes if needed (#6662)
* add read/write support for rotated geotransform (#1778)
GeoRaster driver:
* fix report of rotation (#6593)
* support for JP2-F compression (#6861)
* support direct loading of JPEG-F when blocking=no (#6861)
* default blocking increased from 256x256 to 512x512 (#6861)
GPKG driver:
* implement tiled gridded elevation data extension
* add VERSION creation option
* check if transaction COMMIT is successful (#6667)
* fix crash on overview building on big overview factor (#6668)
* fix crash when opening an empty raster with USE_TILE_EXTENT=YES
* fix gpkg_zoom_other registration
GTiff driver:
* support SPARSE_OK=YES in CreateCopy() mode (and in update mode with the SPARSE_OK=YES open option), by actively detecting blocks filled with 0/nodata about to be written
* When writing missing blocks (i.e. non SPARSE case), use the nodata value when defined. Otherwise fallback to 0 as before.
* in FillEmptyTiles() (i.e. in the TIFF non-sparse mode), avoid writing zeroes to file so as to speed up file creation when filesystem supports ... sparse files
* add write support for half-precision floating point (Float32 with NBITS=16)
* handle storing (and reading) band color interpretation in GDAL internal metadata when it doesn't match the capabilities of the TIFF format, such as B,G,R ordering (#6651)
* Fix RasterIO() reported when downsampling a RGBA JPEG compressed TIFF file (#6943)
* Switch search order in GTIFGetOGISDefn() - Look for gdal_datum.csv before datum.csv (#6531)
* optimize IWriteBlock() to avoid reloading tile/strip from disk in multiband contig/pixel-interleave layouts when all blocks are dirty
* fix race between empty block filling logic and background compression threads when using Create() interface and NUM_THREADS creation option (#6582)
* use VSIFTruncateL() to do file extension
* optimize reading and writing of 1-bit rasters
* fix detection of blocks larger than 2GB on opening on 32-bit builds
* fix saving and loading band description (#6592)
* avoid reading external metadata file that could be related to the target filename when using Create() or CreateCopy() (#6594)
* do not generate erroneous ExtraSamples tag when translating from a RGB UInt16, without explicit PHOTOMETRIC=RGB (#6647)
* do not set a PCSCitationGeoKey = 'LUnits = ...' as the PROJCS citation on reading
* fix creating an image with the Create() interface with BLOCKYSIZE > image height (#6743)
* fix so that GDAL_DISABLE_READDIR_ON_OPEN = NO / EMPTY_DIR is properly honoured and doesn't cause a useless directory listing
* make setting GCPs when geotransform is already set work (with warning about unsetting the geotransform), and vice-versa) (#6751)
* correctly detect error return of TIFFReadRGBATile() and TIFFReadRGBAStrip()
* in the YCBCR RGBA interface case, only expose RGB bands, as the alpha is always 255
* don't check free disk space when outputting to /vsistdout/ (#6768)
* make GetUnitType() use VERT_CS unit as a fallback (#6675)
* in COPY_SRC_OVERVIEWS=YES mode, set nodata value on overview bands
* read GCPs in ESRI <GeodataXform> .aux.xml
* explicitly write YCbCrSubsampling tag, so as to avoid (latest version of) libtiff to try reading the first strip to guess it. Helps performance for cloud optimized geotiffs
* map D_North_American_1927 datum citation name to OGC North_American_Datum_1927 so that datum is properly recognized (#6863)
* Internal libtiff. Resync with CVS (post 4.0.7)
* Internal libtiff: fix 1.11 regression that prevents from reading one-strip files that have no StripByteCounts tag (#6490)
GRASS driver:
* plugin configure: add support for GRASS 7.2 (#6785)
* plugin makefile: do not clone datum tables and drivers (#2953)
* use Rast_get_window/Rast_set_window for GRASS 7 (#6853)
GRIB driver:
* Add (minimalistic) support for template 4.15 needed to read Wide Area Forecast System (WAFS) products (#5768)
* **Partial** resynchronization with degrib-2.0.3, mostly to get updated tables (related to #5768)
* adds MRMS grib2 decoder table (http://www.nssl.noaa.gov/projects/mrms/operational/tables.php) (github #160)
* enable PNG decoding on Unix (#5661, github #160)
* remove explicitly JPEG2000 decompression through Jasper and use generic GDAL code so that other drivers can be triggered
* fix a few crashes on malformed files
GTX driver:
* add a SHIFT_ORIGIN_IN_MINUS_180_PLUS_180 open option
HDF4 driver:
* Fixed erroneous type casting in HDF4Dataset::AnyTypeToDouble() that breaks reading georeferencing and other metadata
HDF5 driver:
* correct number of GCPs to avoid dummy trailing (0,0)->(0,0,0) and remove +180 offset applied to GCP longitude. Add instead a heuristics to determine if the product is crossing the antimeridian, and a HDF5_SHIFT_GCPX_BY_180 config option to be able to override the heuristics (#6666)
HFA driver:
* fix reading and writing of TOWGS84 parameters (github #132)
* export overview type from HFA files to GDAL metadata as OVERVIEWS_ALGORITHM (github #135)
* make .ige initialization use VSIFTruncateL() to be faster on Windows
* add support for TMSO and HOM Variant A projections (#6615)
* Add elevation units read from HFA files metadata (github #169)
* set binning type properly according to layerType being thematic or not (#6854)
Idrisi driver:
* use geotransform of source dataset even if it doesn't have a SRS (#6727)
* make Create() zero-initialize the .rst file (#6873)
ILWIS driver:
* avoid IniFile::Load() to set the bChanged flag, so as to avoid a rewrite of files when just opening datasets
ISCE driver:
* fix computation of line offset for multi-band BIP files, and warn if detecting a wrong file produced by GDAL 2.1.0 (#6556)
* fix misbehaviour on big endian hosts
* add support for reading and writing georeferencing (#6630, #6634)
* make parsing of properties case insensitive (#6637)
ISIS3 driver:
* add write support
* add mask band support on read
* get label in json:ISIS3 metadata domain
JPEGLS driver:
JP2ECW driver:
* fix crash when translating a Float64 raster (at least with SDK 3.3)
JP2KAK driver:
* add support for Kakadu v7.9. v7.8 should not be used. It has a bug fixed in v7.9
* catch exceptions in jp2_out.write_header()
JP2OpenJPEG driver:
* add a USE_TILE_AS_BLOCK=YES open option that can help with whole image conversion
* prevent endless looping in openjpeg in case of short write
* for single-line organized images, such as found in some GRIB2 JPEG2000 images, use a Wx1 block size to avoid huge performance issues (#6719)
* ignore warnings related to empty tag-trees.
JPIPKAK driver:
* fix random crashes JPIP in multi-tread environment (#6809)
KEA driver:
* Add support for Get/SetLinearBinning (#6855)
KMLSuperOverlay driver:
* recognize simple document made of GroundOverlay (#6712)
* Add FORMAT=AUTO option. Uses PNG for semi-transparent areas, else JPG. (#4745)
LAN driver:
* remove wrong byte-swapping for big-endian hosts
MAP driver:
* change logic to detect image file when its path is not absolute
MBTiles driver:
* on opening if detecting 3 bands, expose 4 bands since there might be transparent border tiles (#6836)
* fix setting of minzoom when computing overviews out of order
* do not open .mbtiles that contain vector tiles, which are not supported by the driver
MEM driver:
* disable R/W mutex for tiny performance increase in resampled RasterIO
* add support for overviews
* add support for mask bands
MRF driver:
* bug fix in PNG and JPEG codecs
* Fixing a problem with setting NoData for MRFs generated with Create
* fix plugin building (#6498)
* rename CS variable so as to avoid build failure on Solaris 11 (#6559)
* Allow MRF to write the data file directly to an S3 bucket.
* Allow relative paths when MRF is open via the metadata string.
* Add support for spacing (unused space) between tiles. Defaults to none.
* Read a single LERC block as an MRF file.
MSG driver:
* fix incorrect georeference calculation for msg datasets (github #129)
NetCDF driver:
* add support for reading SRS from srid attribute when it exists and has content like urn:ogc:def:crs:EPSG::XXXX (#6613)
* fix crash on datasets with 1D variable with 0 record (#6645)
* fix erroneous detection of a non-longitude X axis as a longitude axis that caused a shift of 360m on the georeferencing (#6759)
* read/write US_survey_foot unit for linear units of projection
* apply 'add_offset' and 'scale_factor' on x and y variables when present, such as in files produced by NOAA from the new GOES-16 (GOES-R) satellite (github #200)
* add a HONOUR_VALID_RANGE=YES/NO open option to control whether pixel values outside of the validity range should be set to the nodata value (#6857)
* fix crash on int64/uint64 dimensions and variables, and add support for them (#6870)
NITF driver:
* add support for writing JPEG2000 compressed images with JP2OpenJPEG driver
* fix writing with JP2KAK driver (emit codestream only instead of JP2 format)
* fix setting of NBPR/NBPC/NPPBH/NPPBV fields for JPEG2000 (fixes #4322); in JP2ECW case, make sure that the default PROFILE=NPJE implies 1024 block size at the NITF level
* implement creation of RPC00B TRE for RPC metadata in CreateCopy() mode
* add support for reading&writing _rpc.txt files
* nitf_spec.xml: Add support for MTIRPB TRE in NITF image segment. Also makes minor change to BLOCKA to include default values (github #127)
* nitf_spec.xml: add IMASDA and IMRFCA TREs
* GetFileList(): Small optimization to avoid useless file probing.
NWT_GRD:
* detect short writes
OpenJPEG driver:
* support direct extracting of GeoRaster JP2-F BLOB (#6861)
PCIDSK driver:
* handle Exceptions returned from destructor and check access rights in setters (github #183)
PDF driver:
* implement loading/saving of metadata from/into PAM (#6600)
* implement reading from/writing to PAM for geotransform and projection (#6603)
* prevent crashes on dataset reopening in case of short write
PLScenes driver:
* add a METADATA open option
PostgisRaster driver:
* fix potential crash when one tile has a lower number of bands than the max of the table (#6267)
R driver:
* fix out-of-memory (oom) with corrupt R file
Raw drivers:
* prevent crashes on dataset closing in case of short write
RMF driver:
* fix wrong counter decrement that caused compressed RMF to be incorrectly decompressed (github #153)
* fix load/store inversion of cm and dm units in MTW files (github #162)
* fix reading nodata for non-double data type (github #174)
ROIPAC driver:
* add support for reading/writing .flg files (#6504)
* fix computation of line offset for multi-band BIP files, and warn if detecting a wrong file produced by GDAL >= 2.0.0 (#6591)
* fix for big endian hosts
RS2 driver:
* add support for reading RPC from product.xml
SAFE driver:
* fix handling of SLC Products by providing access to measurements as subdatasets (#6514)
Sentinel2 driver:
* add support for new "Safe Compact" encoding of L1C products (fixes #6745)
SQLite driver:
* Add read support for RasterLite2 coverages in SQLite driver
SRTMHGT driver:
* open directly .hgt.zip files
* accept filenames like NXXEYYY.SRTMGL1.hgt (#6614)
* handle files for latitude >= 50 (#6840)
VRT driver:
* add default pixel functions: real, imag, complex, mod, phase, conj, etc... for complex data types (github #141)
* avoid useless floating point values in SrcRect / DstRect (#6568)
* avoid buffer initialization in RasterIO() when possible (replace ancient and likely broken concept of bEqualAreas)
* make CheckCompatibleForDatasetIO() return FALSE on VRTDerivedRasterBands (#6599)
* VRT warp: fix issue with partial blocks at the right/bottom and dest nodata values that are different per band (#6581)
* fix performance issue when nodata set at band level and non-nearest resampling used (#6628)
* VRTComplexSource: do temp computations on double to avoid precision issues when band data type is Int32/UInt32/CInt32/Float64/CFloat64 (#6642)
* VRT derived band: add the capability to define pixel functions in Python
* CreateCopy(): detect short writes
* Fix linking error on VRTComplexSource::RasterIOInternal<float>() (#6748)
* avoid recursion in xml:VRT metadata (#6767)
* prevent 'Destination buffer too small' error when calling GetMetadata('xml:VRT') on a in-memory VRT copied from a VRT
* fix 2.1 regression that can cause crash in VRTSimpleSource::GetFileList() (#6802)
WMS driver:
* Added support for open options to WMS minidrivers
* Refactored the multi-http code to make it possible to do range requests.
* Added a minidriver_mrf, which reads from remote MRFs using range requests.
* Made the minidriver_arcgis work with an ImageService, not only MapService.
* Added static cache of server response.
* Allow tiledWMS to work in off-line mode by including the server response in the .wms file itself.
* honour GDAL_HTTP_USERAGENT config option when it is set and <UserAgent> is missing (#6825)
* WMS/WMTS: better deal with tiles with different band count (grayscale, gray+alpha, palette, rgb, rgba) (github #208)
* Make HTTPS options apply to tWMS minidriver init, better HTTP error reporting
WMTS driver:
* do not take into account WGS84BoundingBox/BoundingBox that would be the result of the densified reprojection of the bbox of the most precise tile matrix
* add TILEMATRIX / ZOOM_LEVEL open options
* accept tiles of small dimensions (github #210)
XYZ driver:
== OGR 2.2.0 - Overview of Changes ==
Core:
* Layer algebra: Add KEEP_LOWER_DIMENSION_GEOMETRIES=YES/NO option to Intersection, Union and Identity.
Default is yes, but it is set to no unless result layer is of geom type unknown.
If set to no, result features which would have lower dim geoms are skipped
if operating on two layers with same geom dim.
* Fix crash/corrupted values when running importFromWkb() on POLYGON M/POLYGON ZM geometries (#6562)
* Add OGR_WKT_PRECISION config option that defaults to 15 to select the number of decimals when outputting to WKT
* Make OGRFeature::SetField(string) accept JSon serialized arrays for the String/Integer/Integer64/RealList types; add reciprocal OGRFeature::GetFieldAsSerializedJSon() for those types
* OGRGeometryFactory::transformWithOptions(): better deal with reprojection from polar projection to WGS84, and projections crossing the antimeridian to WGS84, by splitting geometries prior to reprojection (#6705)
* LinearRing transformTo(): force last point to be identical to first one in case it is not.
* GML geometry parsing: avoid 'Cannot add a compound curve inside a compound curve' error (#6777)
* OGR SQL: fix IN filtering on MapInfo indexed columns (#6798)
* OGR SQL: add support for LIMIT and OFFSET keywords
* OGR SQL: add comparisons on date / datetime (#6810)
* OGR SQL: increase efficiency of DISTINCT operator
* OGREnvelope: change initialization to fix issue when getting MULTIPOINT(0 0,1 1) envelope (#6841)
* OGRParse: fix parsing logic to avoid false positive detection of string as datetime (#6867)
OGRSpatialReference:
* Upgrade to EPSG database v9.0 (#6772)
* OGRCT: upgrade LIBNAME of mingw and cygwin to libproj-9.dll and cygproj-9.dll to be up-to-date with proj 4.9.X (recommended method is using ./configure --with-static-proj4 instead) (#6501)
* importFromESRI(): fix import of multi line MERCATOR SRS (#6523)
* morphToESRI(): correctly compute standard_parallel_1 of Mercator(2SP) projection from scale factor of Mercator(1SP) (#6456, #4861)
* exportToProj4(): recognize explicit OSR_USE_ETMERC=NO to avoid using etmerc with proj >= 4.9.3
* importFromProj4(): do not set a AUTHORITY node for strings like '+init=epsg:XXXX +some_param=val'
* importFromProj4(): be robust with missing proj.4 epsg dictionary when importing '+init=epsg:xxxx +other_parm=value'
* AutoIdentifyEPSG(): add identification of EPSG:3995 (Arctic Polar Stereographic on WGS84) and EPSG:3031 (Antarctic Polar Stereographic on WGS84)
* OGRCoordinateTransformation: avoid potential bugs in proj.4 on NaN input
* importFromEPSG(): take into account DX,DY,DZ,RX,RY,RZ,DS columns in pcs.csv to add per-PCS TOWGS84 overrides (geotiff #52)
* Coordinate transformation: prevent unnecessary coordinate transformations (github #184, #185)
Utilities:
* ogr2ogr: do not return error on ogr2ogr --utility_version
* ogr2ogr: keep -append and -overwrite when -update follows
* ogr2ogr: fix heuristics to detect likely abs…
|
It's back. But I don't see any libtiff changes since 2017. Why on Earth would one ever cap jpg to ~7mb in the year 2022? Certainly another accidental oversight that is now affecting distros like Fedora, which can no longer deal with jpgs, which is going to result in a lot of confused bug reports. |
|
@JohannPopper This is the libjpeg-turbo bug tracker, but this issue is not in libjpeg-turbo. I would suggest filing a new issue here: https://gitlab.com/libtiff/libtiff. |
|
@dcommander Ok. Sorry. Thanks. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Just got upgraded to v1.5.2 in Arch Linux, and
tiff2pdfcan no longer produce PDF files:Reverting back to 1.5.1 fixes the problem.
The text was updated successfully, but these errors were encountered: