QGIS 3.0: Break compatibility with 2.x processing models

[nyalldawson]

QGIS Enhancement: Break compatibility with 2.x processing models

Date 2017/06/12

Author Nyall Dawson @nyalldawson

Contact nyall dot dawson at gmail dot com

maintainer @volaya @alexbruy

Version QGIS 3.0

Summary

Currently, processing is being heavily reworked for 3.0 with the port of numerous core processing functions across to c++ classes. Additionally, numerous algorithms have been heavily reworked for 3.0.

This QEP proposes to drop compatibility with processing 2.x models. The rationale here is:

• an algorithm which was available in 2.x was removed. This breaks existing models which used that algorithm. (Specifically qgis:eliminatesliverpolygons)
• we have a bunch of "duplicate" native algorithms, some of which have been "deprecated" in 3.0. Deprecation just hides them from the toolbox - it doesn't break existing models. This is a HUGE amount of baggage to carry around and maintain, for little benefit. Examples include the grid creation algorithms (condensed to single 'ultimate' algorithms), the field type specific basic stats algorithms (replaced with a universal 'basic stats' for any field type), polygon centroids (replaced with an algorithm which calculates the centroid for any geometry type), split lines with lines (replaced by SplitWithLines).
• any custom user scripts will be broken, because they use the old processing API, PyQGIS API, python 2 + Qt4.
• the remaining algs still need a lot of cleanup. We have a lot of duplicate functionality spread across the QGIS algs, which would be good to address for 3.0. The organic growth of processing algorithms has led to non-optimal arrangement of algorithms (One example is the "polygon from layer extent" algorithm, which has a redunant option to output extent for individual features. This is better covered by the bounding boxes algorithm. Another is that the dissolve algorithm has a redundant check box for "dissolve all", which should just be decided based on the presence or absence of the dissolve by fields). We can't clean up these inconsistencies without breaking old models.
• 3.0 is a major release. Some changes to user workflows and existing projects are expected. Better to make the break now then in 3.2/other future 3.x releases.
• 3.0 can be installed side-by-side with 2.x, allowing users to open existing models in 2.x and step by step recreate their models in 3.0 world.

Proposed Solution

Break compatibility with existing 2.x models, and note this in the release notes. Clean up the existing algorithms and have a nice clean, logical base of algorithms for 3.x.

Backwards Compatibility

This QEP explicitly BREAKS backwards compatibility ;)

Votes

(required)

[alexbruy]

+1 from me

[3nids]

Can you provide some thoughts about an automatic migration (porting projects)?
i.e. why it is not feasible? / is this too complicated with respect to the impact? missing funding? etc.

[pcav]

+1

[ghtmtt]

What about having some pop-up informing the user that the model won't work because of the algorithm X? So maybe it could be easier for the users to find the other algorithms and change them with the new ones

[nirvn]

@nyalldawson , +1

[mbernasocchi]

+1

[mach0]

cleaner solutions are always better
+1

[nyalldawson]

@3nids Can you provide some thoughts about an automatic migration (porting projects)?
i.e. why it is not feasible? / is this too complicated with respect to the impact? missing funding? etc.

processing 2.x used json.dumps/json.loads to serialize models - this makes things very tricky, because the corresponding classes don't exist anymore, using json.loads to restore 2.x models won't work. At the same time, the json structure could lend itself to a separate "upgrading" script which just reads the json as a dict and then attempts to recreate a new model by matching the old algorithms across to new processing algs. It's not straightforward though.

Another issue with the current set of algorithms is the inconsistency in parameter naming. We use a bunch of different names for the same parameter - 'INPUT'/'INPUT_LAYER', 'OUTPUT'/'OUTPUT_LAYER', etc. This makes for confusing use. E.g.

processing.run("qgis:densifygeometries", {'INPUT':'some_layer,'VERTICES':12,'OUTPUT':'a.shp'})

vs

processing.run("qgis:boundary", {'INPUT_LAYER':'some_layer,'OUTPUT_LAYER':'a.shp'})

I'd like to standardise these, but that will further break 2.x model compatibility.

[3nids]

Thanks for the detailed explanation.
A wholeheartedly 👍

[nyalldawson]

@NathanW2 can we lock this as accepted? There's been a wide approval from devs and non-devs alike here.

[NathanW2]

Yes good with me. Generally it would be good to support translation but I can see that is super hard at the moment.

…

On Wed, 21 Jun 2017, 3:20 PM Nyall Dawson ***@***.***> wrote: @NathanW2 <https://github.com/nathanw2> can we lock this as accepted? There's been a wide approval from devs and non-devs alike here. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#95 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXS3HAzre_QUlPUjjZrKbrtTiHiuch8ks5sGKgzgaJpZM4N2zr2> .

[pcav]

Good for me. Non standardized output names bothered me a lot in the past. Thanks.