Spatial references in Expressions (refFunctions goes core)

[m-kuhn]

QGIS Enhancement: Spatial references in Expressions

Date 2019/05/15

Author Matthias Kuhn (@m-kuhn) / Enrico Ferreguti (@enricofer)

Contact matthias@opengis.ch

maintainer @m-kuhn

Version QGIS 3.10

Summary

The aim of this proposal is to add a set of new spatial reference functions to the expressions engine.

This new feature will allow the re-use of feature attributes and geometry located on other layers but spatially related with the currently evaluated feature.

Expressions with spatially related functions are particularly useful for defining complex symbologies, layouts and labels without the need to create new data from scratch or to run specific algorithms.

At the moment spatial reference in expressions can be performed with refFunctions plugin, available in the QGIS plugins repository, that adds some new spatial reference functions in the expression builder and field calculator. This python solution has preformance and reliability limits. This functionality can also be considered base functionality of a QGIS system. It would therefore be preferable to have a stronger implementation in core.

Proposed Solution

Adding a new function with the following syntax:

geometry_overlay(_LAYER_, _METHOD_, _OUTPUT_)

with these arguments:

• _LAYER_: type "layer" (id, layer name, QgsVectorLayer), required: the layer on which to perform spatial reference

• _METHOD_: type "String", required: spatial condition to verify

  • "intersects"
  • "within"
  • "contains"
  • ...

• _OUTPUT_: type "String", an optional Expression:

  • if _OUTPUT_ argument is not provided or left empty the command returns a boolean value depending whether or not the provided spatial condition is satisfied.
  • Otherwise the _OUTPUT_ contains an expression to be performed in the target layer context for extracting related informations such as "target_layer_field_name" , $geometry, $area or COALESCE("field", 0)

A target value can be retrieved from the output list using an array function, for example array_first, array_cat, array_get, array_lenght or summarized with other aggregates functions ( sum, count, etc...).

At this point existing Array functions group lacks of many geometry and number aggregation functions so it would be useful to extend the aggregate functions group, which takes features from a layer source, using them for array too either by adding a new aggregate_array(array, aggregate_method [, filter]) or by allowing the existing one to take an array as first parameter (instead of only layers). This is not subject of this QEP in particular, but will be treated in the same project.

Next to the function geometry_overlay, to improve discoverability by autocompletion search more self explained aliases will be created:

• geometry_overlay_intersects(_LAYER_, _OUTPUT_)

• geometry_overlay_touches(_LAYER_, _OUTPUT_)

• geometry_overlay_within(_LAYER_, _OUTPUT_)

• geometry_overlay_contains(_LAYER_, _OUTPUT_)

• and so on...

Example(s)

geometry_overlay('layer_e5313dca_be5e_4d28_95f9_8c58b41ef4ad','intersects')

returns true or false (this method covers the most used case, can act as spatial modifier in rule symbology

array_sum(geometry_overlay_contains('layer_e5313dca_be5e_4d28_95f9_8c58b41ef4ad', "population"))

returns the sum of the values of field population of the features satisfying spatial condition

array_lenght(geometry_overlay('layer_e5313dca_be5e_4d28_95f9_8c58b41ef4ad','contains',"fid"))

returns the count of the features satisfying spatial condition (in this case the field parameter is needed to return an array of results to be counted)

array_first(geometry_overlay_touches('layer_e5313dca_be5e_4d28_95f9_8c58b41ef4ad',$geometry))

returns the first geometry from the features satisfying the "touch" spatial condition

array_first(geometry_overlay_intersects('layer_e5313dca_be5e_4d28_95f9_8c58b41ef4ad',buffer($geometry,20)))

returns the first buffered geometry from the features satisfying the "intersects" spatial condition

other aggregation modes would be possible taking vantage of aggregate function, extending them to apply to arrays as well as for layer features.

Affected Files

core/expression/qgsexpressionfunction.cpp

Performance Implications

(required if known at design time)

Further Considerations/Improvements

(optional)

Backwards Compatibility

(required)

Issue Tracking ID(s)

(optional)

Votes

(required)

[luipir]

+1

[nyalldawson]

if OUTPUT argument is not provided or left empty the command returns a boolean value depending whether or not the provided spatial condition is satisfied.
Otherwise the OUTPUT contains an expression to be performed in the target layer context for extracting related informations such as "target_layer_field_name" , $geometry, $area or COALESCE("field", 0)

I wonder if this could be simplified by splitting into two functions:

• geometry_overlay_test (or test_geometry_overlay): only layer/predicate arguments, always returns true or false
• geometry_overlay: always returns an array. If the third argument is not specified, the returned array is an array of the matching features

aggregate_array

Speaking of aggregate array, it would be nice to keep similar functionality here -- i.e. have you considered adding an optional "filter" argument to geometry_overlay?

array_first(geometry_overlay_touch('layer_e5313dca_be5e_4d28_95f9_8c58b41ef4ad','$geometry'))

This is a rather inefficient way to do this, and would certainly have performance issues on large tables. What about adding an optional "limit" argument instead?

array_sum(geometry_overlay_contains('layer_e5313dca_be5e_4d28_95f9_8c58b41ef4ad', ' "population" '))

Is there a typo here for the 3rd argument? Like aggregates, it should be a sub expression, not a string literal, right?

In general, I am very much in favour of providing a stable, optimised solution to this as an alternative to refFunctions. I see that that plugin is very widely used, so there's clearly a demonstrated need for this. Yet the approaches used by refFunctions are very inefficient, and also quite dangerous (not thread safe), so this work would be extremely valuable!

[roya0045]

+1 also

[m-kuhn]

Thanks for the feedback @nyalldawson

I wonder if this could be simplified by splitting into two functions:

geometry_overlay_test (or test_geometry_overlay): only layer/predicate arguments, always returns true or false
geometry_overlay: always returns an array. If the third argument is not specified, the returned array is an array of the matching features

I am 50/50 on that one. While it's technically / API wise a bit more concise because of the static return type, it decreases discoverability.

Speaking of aggregate array, it would be nice to keep similar functionality here -- i.e. have you considered adding an optional "filter" argument to geometry_overlay?

Interesting thought indeed. That would make it possible to filter results earlier and improve performance.
If I'm not mistaken, that's something that can be added at any time without changing the base concept, correct?

This is a rather inefficient way to do this, and would certainly have performance issues on large tables. What about adding an optional "limit" argument instead?

Sounds like a very good proposal.
I think that's something that can be added as an extension at any time without changing the concept here at all, so it's just something to keep in mind and do "if reasonably possible", correct?

Is there a typo here for the 3rd argument? Like aggregates, it should be a sub expression, not a string literal, right?

Thanks, corrected.

[M-Rick]

Will this include the geomnearest feature as well?

[thymaro]

Is this under consideration or in development? I use the plugin for this and stumbled upon this open issue. I don't know that the functionalities of the refFunctions can be used without the plugin, so that's why I'm asking.

Also, there's a typo in array_lenght. It should be array_length. I don't know whether that's relevant.

[m-kuhn]

Ping @olivierdalang

[olivierdalang]

@thymaro Yes it's under development, and will hopefully land in 3.16