Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3.10 backports #34607

Merged
merged 49 commits into from
Feb 26, 2020
Merged

3.10 backports #34607

merged 49 commits into from
Feb 26, 2020

Conversation

nyalldawson
Copy link
Collaborator

Backports which were delayed to after 3.10.3 (+some other fixes which weren't backported)

Notably includes the transform fallback work and shapefile encoding rework

For merge early in the 3.10.4 cycle for widest testing and feedback

elpaso and others added 30 commits February 22, 2020 14:31
because actions() does something else

(cherry picked from commit bcf0e48)
(cherry picked from commit f71e04e)
Fixes qgis#34552

probably fixes other unreported issues.

(cherry picked from commit 1bb6a53)
try falling back to a default PROJ-provided operation to transform the
coordinates

This allows **some** transformation of coordinates to occur in the situation:
- user has selected a coordinate operation utilising a grid shift file
- one or more points which fall OUTSIDE the grid file are transformed

Not for merge -- likely needs some form of user-visible-warning when
this occurs

Refs qgis#33929

(cherry picked from commit ec78431)
(cherry picked from commit 1684ff1)
failures, so manually scan for inf values to detect failures which
should result in the fallback transform usage

(cherry picked from commit fd534f6)
…nsforms

for a particular QgsCoordinateTransform object

This can be set for transforms where we definitively know that a "ballpark"
result is acceptable, e.g. when transforming global layer extents to
a localized area-of-use CRS

Refs qgis#33929

(cherry picked from commit 8bb1c47)
(cherry picked from commit 890fe57)
We can't populate this, because it's not a singular proj operation

(cherry picked from commit 9883e18)
instead correctly use the desired operation for the handler

(cherry picked from commit ee4774a)
This is too costly to calculate, and results in a lot of processing
time when raster rendering transform errors occur (since they occur
on a one-by-one basis and in large quantities)

(cherry picked from commit 5a60a4c)
…_errno

It's giving us incorrect error numbers (such as "failed to load datum shift file",
which is definitely incorrect for a default proj created operation!)
so we resort to testing values ourselves...

(cherry picked from commit 4da584d)
(cherry picked from commit 9d7355b)
Hopefully giving enough clues for users to diagnose the error

(cherry picked from commit 6637d45)
…ansform whether a fallback operation occurred

(cherry picked from commit 0b48f0a)
of to the main window message bar

More processing algorithms will require this, but it's a template for
the others to be adapted

(cherry picked from commit ef06fb7)
(cherry picked from commit 62e6e9e)
ballpark transforms for a source/dest CRS pair

(cherry picked from commit 61e6ab0)
on a transform-by-transform basis, populate correctly from context

(cherry picked from commit e3b1649)
…propriate

and setAllowFallbackTransforms and improve dox

setBallparkTransformsAreAppropriate overrules the setAllowFallbackTransforms
setting

(cherry picked from commit d4d6f79)
- Don't spew a whole lot of "rethrowing expection" log messages on the
default log level
- Don't output tons of duplicate transform error messages - instead
only output the message if its different to the last error encountered
(avoids flooding the console when transform failures occur during
raster projection, resulting in quasi-hangs)

(cherry picked from commit f749ba5)
(cherry picked from commit a5b6290)
(cherry picked from commit e0556cf)
We no longer support building with these versions

(cherry picked from commit b2f37bb)
Require 6.3.1 for critical fixes

(cherry picked from commit 38e57db)
nyalldawson and others added 19 commits February 22, 2020 14:41
(cherry picked from commit 9ba4fb9)
(cherry picked from commit 97486ee)
(cherry picked from commit efeadcf)
(cherry picked from commit 6a0266c)
(cherry picked from commit 342e6db)
that only a single transform is possible between a CRS pair

(since we've already tried that transform, it's pointless to re-try
it and expect different results)

Avoids superious "fallback transform used" warnings when only one
operation is possible between a CRS pair

(cherry picked from commit 1c8aacf)
…ation

Ported from the GDAL logic, which unfortunately is not available
for re-use directly from GDAL.

(cherry picked from commit bb71b71)
And instead always do the decoding on QGIS' side. This unifies
the encoding handling whether or not we are using the underlying
shapefile declared encoding (e.g. via LDID or .cpg file) OR
are overriding it manually by a user-set encoding.

Why?

- if we DON'T disable GDAL side encoding support, then there's NO
way to change the encoding used when reading shapefiles. And
unfortunately the embedded encoding (which is read by GDAL) is
sometimes wrong (because shapefiles!), so we need to expose support
for users to be able to change and correct this

- we can't change this setting on-the-fly. If we don't set it
upfront, we can't reverse this decision later when a user does
want/need to manually specify the encoding

This also removes a lot of confusing code logic in the provider!

Fixes qgis#21264, user frustration on mailing lists e.g.

http://osgeo-org.1560.x6.nabble.com/Shapefile-with-file-cpg-codepage-td5275106.html
http://osgeo-org.1560.x6.nabble.com/QGIS-ignore-the-cpg-files-when-loading-shapefiles-td5348021.html
(cherry picked from commit f36bd8f)
This was previously a band-aid fix applied in order to get shapefiles
to load using the encoding specified in the data source manager
encoding option.

But the same settings key is used for lots of different things, e.g.
saving and restoring the last used encoding when creating a new layer.

By forcing all layers to this encoding leads to lots of nasty issues,
e.g. shapefiles loaded through the QGIS api will get a quasi-random
encoding assigned to them (unless the API caller manually follows up
with a setEncoding() call...)

It's no longer required now, in any case

(cherry picked from commit c0581ff)
(cherry picked from commit e93bacd)
(cherry picked from commit 5b63955)
via drag and drop or browser, instead of always picking the System encoding

(cherry picked from commit a74b259)
through the Data Source Manager

This allows the embedded (LDID/CPG based) Shapefile encoding to
be used by default, but still offers users a way to override this.

(cherry picked from commit 301d2ba)
(cherry picked from commit 9bd81de)
Avoids code duplication, correct behavior with vsi* sources

(cherry picked from commit 54eba4a)
@nyalldawson nyalldawson added the Backport Is a backport of another pull request label Feb 22, 2020
@nyalldawson nyalldawson added this to the 3.10.4 milestone Feb 22, 2020
@nyalldawson
Copy link
Collaborator Author

Thanks @nirvn!

@nyalldawson nyalldawson merged commit 5cb8573 into qgis:release-3_10 Feb 26, 2020
@nyalldawson nyalldawson deleted the backports310 branch February 26, 2020 01:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Backport Is a backport of another pull request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants