Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More processing parameters in modeller #6504

Merged
merged 8 commits into from Mar 1, 2018

Conversation

m-kuhn
Copy link
Member

@m-kuhn m-kuhn commented Mar 1, 2018

Opens up the processing parameter system more. Instead of having hardcoded lists of parameters in various places, there is a registry of available parameters and an API for plugins to expose their own parameters.

This makes the fields mapping parameter for refactor fields available in the modeller.

screenshot from 2018-03-01 08-18-32

As a nice side effect, the list of available parameters in the modeller dialog is now also translatable.

At the current state, all available parameters are exposed to the modeller. Many of them are not prepared (i.e. they don't have a combobox to select from available input parameter definitions inside an algorithm, so cannot be connected). I'll probably disable those for now, they can then incrementally be added later on.

@alexbruy
Copy link
Contributor

alexbruy commented Mar 1, 2018

Wow! Super nice addition

@nirvn
Copy link
Contributor

nirvn commented Mar 1, 2018

Nice.

BTW, side thought, I think ultimately we want the parameters to be added through a dedicated toolbar instead of a tree list widget. That'd make the experience feel a lot more like our layout designer.

@m-kuhn m-kuhn requested a review from alexbruy March 1, 2018 14:21
@m-kuhn
Copy link
Member Author

m-kuhn commented Mar 1, 2018

@alexbruy could you give this a review?

@m-kuhn m-kuhn added this to the 3.2 milestone Mar 1, 2018
@m-kuhn
Copy link
Member Author

m-kuhn commented Mar 1, 2018

BTW, side thought, I think ultimately we want the parameters to be added through a dedicated toolbar instead of a tree list widget. That'd make the experience feel a lot more like our layout designer.

I like the treeview with filter possibilities, but I think they should be in the same treeview as the algorithms.

@m-kuhn
Copy link
Member Author

m-kuhn commented Mar 1, 2018

Ready for review

@alexbruy
Copy link
Contributor

alexbruy commented Mar 1, 2018

Looks good to me.

Only wondering if we should have parameters registry in the core instead of having it in Python.

@alexbruy
Copy link
Contributor

alexbruy commented Mar 1, 2018

+1 to have parameters in the treeview with filter. With custom parameters we very quickly will get cluttered toolbar.

@nirvn
Copy link
Contributor

nirvn commented Mar 1, 2018

Would there be any use for a basic parameters toolbar to live alongside the treeview.

@m-kuhn
Copy link
Member Author

m-kuhn commented Mar 1, 2018

Only wondering if we should have parameters registry in the core instead of having it in Python.

It's kind of a mixed thing between widgets (which are defined in python) and core parameters (which are in C++). And wasn't sure how to expose classes (the param['parameter'] thing) from C++ side. So I'm not unhappy with this as a first solution. If the requirements come up or someone can think of a clean solution, it should be easy to do that down the road.

Would there be any use for a basic parameters toolbar to live alongside the treeview

I'd do it like QtCreator. Add elements from the treeview and have a toolbar for layout things and other operations.

@alexbruy
Copy link
Contributor

alexbruy commented Mar 1, 2018

Yep, I also think that as first solution it looks awesome. We can improve things later when needed

@m-kuhn m-kuhn merged commit 4f4b831 into qgis:master Mar 1, 2018
@nyalldawson
Copy link
Collaborator

Late comment (very small merge window here!) But why not add a flag for parameters as to whether they are exposed to the modeler and filter on this? Then we have a method to hide parameters which CANNOT be used as model parameters.

Processing.REGISTERED_PARAMETERS = dict()

@staticmethod
def registerParameter(id, name, parameter, metadata=dict(), description=None, exposeToModeller=True):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move this to the C++ processing registry please, and add unit tests for it?

@m-kuhn
Copy link
Member Author

m-kuhn commented Mar 1, 2018

But why not add a flag for parameters as to whether they are exposed to the modeler and filter on this? Then we have a method to hide parameters which CANNOT be used as model parameters.

That's in there already.

Can we move this to the C++ processing registry please, and add unit tests for it?

See #6504 (comment)

@m-kuhn
Copy link
Member Author

m-kuhn commented Mar 1, 2018

And sorry for the quick merge, happy for any review to still land now !

@nyalldawson
Copy link
Collaborator

nyalldawson commented Mar 1, 2018

But why not add a flag for parameters as to whether they are exposed to the modeler and filter on this? Then we have a method to hide parameters which CANNOT be used as model parameters.

That's in there already.

Ahh - I was thinking of adding a new flag to QgsProcessingParameterDefinition.Flag, but I see now that that won't work since you'd need an instance of the definition to check the flags.

My concern with adding this in Python is that it becomes part of the stable API - so it's locked in for the next 3-4 months (4.0 is June-ish, right?*). (We could potentially add some bridge from this API to a future c++ core parameter registry)

It's kind of a mixed thing between widgets (which are defined in python) and core parameters (which are in C++). And wasn't sure how to expose classes (the param['parameter'] thing) from C++ side. So I'm not unhappy with this as a first solution. If the requirements come up or someone can think of a clean solution, it should be easy to do that down the road.

I think the c++ solution would be very similar to how renderers, symbol layers, layout items, etc are registered. You'd create a metadata object with virtual methods for creating the parameter and creating a config widget (with an enum specifying whether it's for toolbox/modeler/batch).

** a total lie

@nyalldawson
Copy link
Collaborator

By the way -- don't get me wrong, this is a great feature and much appreciated!

@m-kuhn
Copy link
Member Author

m-kuhn commented Mar 1, 2018

My concern with adding this in Python is that it becomes part of the stable API - so it's locked in for the next 3-4 months

Right now we don't promis much API stability within processing, or that shouldn't happen (and I think it should happen). I guess at some point we'll need to start tagging methods and attributes in processing as API stable.

Ahh - I was thinking of adding a new flag to QgsProcessingParameterDefinition.Flag, but I see now that that won't work since you'd need an instance of the definition to check the flags.

Yeah, it's the "MetaParameter" object that is missing. And which would result in writing a QgsProcessingParameterRegistry and QgsProcessingParameterFactories, pretty much like what the editor widgets (and all the others you mentioned...)

And I only wanted to have the fieldsmapper available...

@nyalldawson
Copy link
Collaborator

Ok, how about:

  • some unit tests are added (since core processing is sitting at pretty dang close to 100% test coverage)
  • add a note to the docs "these methods are private and unsupported. They are likely to change in future releases and should not be used by plugins or scripts"

How's that sound?

@m-kuhn
Copy link
Member Author

m-kuhn commented Mar 2, 2018

Unit tests for methods that are not stable API sounds not like what I want to spend my time on to be honest... I totally understand the point. But if I do something from now on, I'd much rather implement a real solution than spending more time on the current implementation (which b.t.w. I think is an improvement already).

@nyalldawson
Copy link
Collaborator

Unit tests for methods that are not stable API sounds not like what I want to spend my time on to be honest... I totally understand the point.

Not trying to be a pain here, because you know I love your work and I understand the reluctance in spending more time on this, but that argument doesn't really hold up. A regression is a regression regardless of whether the underlying code is in stable api or not! 😉

But if I do something from now on, I'd much rather implement a real solution than spending more time on the current implementation (which b.t.w. I think is an improvement already).

Let me know if you do want to go down this route... I've got ideas on how we can slowly start to transition processing gui base classes to c++. I think I've even got an ancient branch floating around somewhere with some first steps...

@m-kuhn
Copy link
Member Author

m-kuhn commented Mar 3, 2018

Let me know if you do want to go down this route...

https://github.com/qgis/QGIS/compare/master...m-kuhn:procparams?expand=1

@nyalldawson
Copy link
Collaborator

It's beautiful 😭

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants