New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] layer refresh and trigger actions on provider notification #5179

Merged
merged 1 commit into from Sep 22, 2017

Conversation

Projects
None yet
@vmora
Contributor

vmora commented Sep 12, 2017

[needs-docs]

In vector layer properties, in the rendering tab, a
"Refresh layer on notification" checkbox has been added to refresh layer
on provider notification.

For a postgres datasource, if a "NOTIFY qgis;" command is issued by one of the database clients,
a refresh of the layer will occur.

If the "Only if message is" checkbox is checked and the notification will trigger the refresh only
if the message contend is the one specified, e.g. if the user enters
"refresh" in the box right next to the "Only if message is" checkbox,
then a "NOTIFY qgis, 'refresh';" command in the datatabase will trigger
a layer refresh, but "NOTIFY qgis;" or "NOTIFY qgis, 'something else';"
won't.

IMPLEMENTATION DETAILS:

A notify signal has been added to the abstract QgsVectorDataProvider
along with a setListening function that enables/disble the notification
mechanism.

For the moment, this notification is only used to refresh/redraw a vector
layer.

For the moment only the postgres provider implements the notification.

The notification notion extends beyond SRGBD servers (postgres and oracle at
least have the notify) and the "watch file" in the delimitedtext
provider could also benefit from this interface.

For the postgres provider a thread is created with a second connection
to the database. This thread is responsible for listening postgres
notifications.

It would be nice to avoid the creation of one listening chanel per
provider in the case transaction groups are enabled.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 12, 2017

Contributor

@haubourg can you try this out please ?

Contributor

vmora commented Sep 12, 2017

@haubourg can you try this out please ?

@haubourg

This comment has been minimized.

Show comment
Hide comment
@haubourg

haubourg Sep 12, 2017

Contributor

Tested now, I like it :)
See https://youtu.be/yLcRmeqiD2E

Contributor

haubourg commented Sep 12, 2017

Tested now, I like it :)
See https://youtu.be/yLcRmeqiD2E

@Gustry

This comment has been minimized.

Show comment
Hide comment
@Gustry

Gustry Sep 12, 2017

Contributor

Nice feature, I didn't know that!

Contributor

Gustry commented Sep 12, 2017

Nice feature, I didn't know that!

@luipir

This comment has been minimized.

Show comment
Hide comment
@luipir

luipir Sep 12, 2017

Contributor

@vmora Is there a reason to reduce action to only refresh instead of registered layer actions?

Contributor

luipir commented Sep 12, 2017

@vmora Is there a reason to reduce action to only refresh instead of registered layer actions?

@luipir

This comment has been minimized.

Show comment
Hide comment
@luipir

luipir Sep 12, 2017

Contributor

btw useful feature :)

Contributor

luipir commented Sep 12, 2017

btw useful feature :)

@luipir

This comment has been minimized.

Show comment
Hide comment
@luipir

luipir Sep 12, 2017

Contributor

@vmora have you planned to add tests for this new feature?

Contributor

luipir commented Sep 12, 2017

@vmora have you planned to add tests for this new feature?

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 12, 2017

Contributor

@luipir:

Is there a reason to reduce action to only refresh instead of registered layer actions?

Nope, it was just the first thing to do that came to mind.

There is no interface yet to connect the signal to an action. But that's a good idea, both simple and powerful.

have you planned to add tests for this new feature ?

Of course it's on my todo list, should be simple enough based on an existing postgres test.

Contributor

vmora commented Sep 12, 2017

@luipir:

Is there a reason to reduce action to only refresh instead of registered layer actions?

Nope, it was just the first thing to do that came to mind.

There is no interface yet to connect the signal to an action. But that's a good idea, both simple and powerful.

have you planned to add tests for this new feature ?

Of course it's on my todo list, should be simple enough based on an existing postgres test.

@luipir

This comment has been minimized.

Show comment
Hide comment
@luipir

luipir Sep 12, 2017

Contributor

great, tnx

Contributor

luipir commented Sep 12, 2017

great, tnx

@andreasneumann

This comment has been minimized.

Show comment
Hide comment
@andreasneumann

andreasneumann Sep 12, 2017

Member

Interesting feature.

It would be interesting to extend this feature to QGIS server. There are push notifications (https://developers.google.com/web/updates/2015/03/push-notifications-on-the-open-web) that allow a registered clients to receive data from the server on the servers demand. That way QGIS server could tell a web client that new data is available and that the client should reload the map image. Would be interesting for dynamic data ...

Just thinking out loud. Not asking for this to be implemented, because I don't have such dynamic data anyway ... but may be a useful improvement for others who have such data.

Member

andreasneumann commented Sep 12, 2017

Interesting feature.

It would be interesting to extend this feature to QGIS server. There are push notifications (https://developers.google.com/web/updates/2015/03/push-notifications-on-the-open-web) that allow a registered clients to receive data from the server on the servers demand. That way QGIS server could tell a web client that new data is available and that the client should reload the map image. Would be interesting for dynamic data ...

Just thinking out loud. Not asking for this to be implemented, because I don't have such dynamic data anyway ... but may be a useful improvement for others who have such data.

@haubourg

This comment has been minimized.

Show comment
Hide comment
@haubourg

haubourg Sep 12, 2017

Contributor

@andreasneumann interesting use case. In a web server context that would be somewhat different. The web client is not directly discussing with postgres in a transaction. So we would need the client to be able to receive "push" notifications (That is what web sockets do, if I get it well).
On the server side, a qgis server plugin could connect itself to the QT signal triggered by the notification, so it's a wide range of possible uses.

Contributor

haubourg commented Sep 12, 2017

@andreasneumann interesting use case. In a web server context that would be somewhat different. The web client is not directly discussing with postgres in a transaction. So we would need the client to be able to receive "push" notifications (That is what web sockets do, if I get it well).
On the server side, a qgis server plugin could connect itself to the QT signal triggered by the notification, so it's a wide range of possible uses.

Show outdated Hide outdated src/core/qgsvectordataprovider.h
Show outdated Hide outdated src/core/qgsvectordataprovider.h
Show outdated Hide outdated src/core/qgsvectorlayer.cpp
Show outdated Hide outdated src/core/qgsvectorlayer.cpp
Show outdated Hide outdated src/core/qgsvectorlayer.cpp
Show outdated Hide outdated src/core/qgsvectorlayer.h
Show outdated Hide outdated src/providers/postgres/qgspostgreslistener.h
@m-kuhn

This comment has been minimized.

Show comment
Hide comment
@m-kuhn

m-kuhn Sep 13, 2017

Member

Nice work @vmora.
Are we aware of which changes we introduced ourselves and which are coming from outside? I think this could be a very valuable information given that with a notification from outside we need to kill every cache we have (e.g. attribute table) while for changes introduced from our own side we are tracking the changes.

@andreasneumann The webserver use case certainly sounds interesting. The main thing to do there is to have some sort of session management where user(s) connect to a project which is kept alive (not only for a single request but for the duration of the session). I guess this is a bigger topic in which the whole server team should be involved (and which could magically resolve all the caching issues with which we had troubles in the past ;) ).

Member

m-kuhn commented Sep 13, 2017

Nice work @vmora.
Are we aware of which changes we introduced ourselves and which are coming from outside? I think this could be a very valuable information given that with a notification from outside we need to kill every cache we have (e.g. attribute table) while for changes introduced from our own side we are tracking the changes.

@andreasneumann The webserver use case certainly sounds interesting. The main thing to do there is to have some sort of session management where user(s) connect to a project which is kept alive (not only for a single request but for the duration of the session). I guess this is a bigger topic in which the whole server team should be involved (and which could magically resolve all the caching issues with which we had troubles in the past ;) ).

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 13, 2017

Contributor

Are we aware of which changes we introduced ourselves and which are coming from outside?

@m-kuhn to me the notification is something coming from outside, but I don't think I understand what you have in mind.

Contributor

vmora commented Sep 13, 2017

Are we aware of which changes we introduced ourselves and which are coming from outside?

@m-kuhn to me the notification is something coming from outside, but I don't think I understand what you have in mind.

@m-kuhn

This comment has been minimized.

Show comment
Hide comment
@m-kuhn

m-kuhn Sep 13, 2017

Member

Are we aware of which changes we introduced ourselves and which are coming from outside?

@m-kuhn to me the notification is something coming from outside, but I don't think I understand what you have in mind.

Which ones have been caused by an update, insert or delete triggered by our local QGIS instance. And which ones by "the outside world™"

Member

m-kuhn commented Sep 13, 2017

Are we aware of which changes we introduced ourselves and which are coming from outside?

@m-kuhn to me the notification is something coming from outside, but I don't think I understand what you have in mind.

Which ones have been caused by an update, insert or delete triggered by our local QGIS instance. And which ones by "the outside world™"

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 13, 2017

Contributor

Which ones have been caused by an update, insert or delete triggered by our local QGIS instance. And which ones by "the outside world™"

To me, outside transaction groups, each layers lives in its own world.

Maybe you can achieve something with the postgres connection process id
select pg_notify('qgis', 'pid'||pg_backend_pid());. This way you have the process id in the notification message.

Contributor

vmora commented Sep 13, 2017

Which ones have been caused by an update, insert or delete triggered by our local QGIS instance. And which ones by "the outside world™"

To me, outside transaction groups, each layers lives in its own world.

Maybe you can achieve something with the postgres connection process id
select pg_notify('qgis', 'pid'||pg_backend_pid());. This way you have the process id in the notification message.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 13, 2017

Contributor

@luipir for your idea of connecting the notification to an action: the interface with the two checkboxes plus the message is a bit clumsy to add in the actions list. Moreover, for actions, I think it's a bit dangerous to react on any notification (e.g. if action triggers a notification, we have a nice infinite loop).

What I'd go for is an additional column for actions with a string to filter messages (could be a regex) if there is a specified string, we enable listening and trigger action if the notification matches. What do you think ?

Contributor

vmora commented Sep 13, 2017

@luipir for your idea of connecting the notification to an action: the interface with the two checkboxes plus the message is a bit clumsy to add in the actions list. Moreover, for actions, I think it's a bit dangerous to react on any notification (e.g. if action triggers a notification, we have a nice infinite loop).

What I'd go for is an additional column for actions with a string to filter messages (could be a regex) if there is a specified string, we enable listening and trigger action if the notification matches. What do you think ?

@m-kuhn

This comment has been minimized.

Show comment
Hide comment
@m-kuhn

m-kuhn Sep 13, 2017

Member

Reading your statement I realize that the main use-case you have in mind is probably side-effects caused by triggers etc - and not concurrent editing by different users.

That's actually another very interesting scenario, but I guess there it's even harder to know which of those changes are already known to our own qgis instance (because it's a result of directly editing and saving a layer) and which ones are side-effects.

So in the end it's up to the database side to emit notifications for the right layers at the right time?
And possibly we can have some sort of additional local filtering by adding the pid or transaction id to the notification signal (future addition)?

Member

m-kuhn commented Sep 13, 2017

Reading your statement I realize that the main use-case you have in mind is probably side-effects caused by triggers etc - and not concurrent editing by different users.

That's actually another very interesting scenario, but I guess there it's even harder to know which of those changes are already known to our own qgis instance (because it's a result of directly editing and saving a layer) and which ones are side-effects.

So in the end it's up to the database side to emit notifications for the right layers at the right time?
And possibly we can have some sort of additional local filtering by adding the pid or transaction id to the notification signal (future addition)?

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 13, 2017

Contributor

not concurrent editing by different users

Well, for me, a different user or a different layer (no transaction group) is the same thing a far a the database is concerned: it's a different connection (or am I mistaken?)

The refresh is the first use case for the notification, so side-effects (triggers) are somewhat related, but also insertion by sensors connected to the database, or updates by another users... But the notification mechanism has a lot more to offer.

As you say "in the end it's up to the database side to emit notifications for the right layers at the right time".

Contributor

vmora commented Sep 13, 2017

not concurrent editing by different users

Well, for me, a different user or a different layer (no transaction group) is the same thing a far a the database is concerned: it's a different connection (or am I mistaken?)

The refresh is the first use case for the notification, so side-effects (triggers) are somewhat related, but also insertion by sensors connected to the database, or updates by another users... But the notification mechanism has a lot more to offer.

As you say "in the end it's up to the database side to emit notifications for the right layers at the right time".

@luipir

This comment has been minimized.

Show comment
Hide comment
@luipir

luipir Sep 13, 2017

Contributor

@vmora I really like the implementation and I agree with your proposal to avoid infinite loop.

Speculating more (and probably outside the scope of this PR), probably we need a base class able to implement a generic Publisher/Subscriber cross network protocol, like ZeroMQ (https://github.com/HBPVIS/ZeroEQ). I use PubSub python lib when I've to add async behaviour in functions that does not use QObject/event infrastrucure.
There are many possible usecases and one is the telemetry plugin or an application to monitor underwater or flying UAV telemetry.
Just a note: https://github.com/SpiderOak/skeeter is an example to bridge PG listen/notify event to ZeroMQ. The advantage is that we can aslo publish events listened by the DB.

Contributor

luipir commented Sep 13, 2017

@vmora I really like the implementation and I agree with your proposal to avoid infinite loop.

Speculating more (and probably outside the scope of this PR), probably we need a base class able to implement a generic Publisher/Subscriber cross network protocol, like ZeroMQ (https://github.com/HBPVIS/ZeroEQ). I use PubSub python lib when I've to add async behaviour in functions that does not use QObject/event infrastrucure.
There are many possible usecases and one is the telemetry plugin or an application to monitor underwater or flying UAV telemetry.
Just a note: https://github.com/SpiderOak/skeeter is an example to bridge PG listen/notify event to ZeroMQ. The advantage is that we can aslo publish events listened by the DB.

@NathanW2

This comment has been minimized.

Show comment
Hide comment
@NathanW2

NathanW2 Sep 13, 2017

Member
Member

NathanW2 commented Sep 13, 2017

@m-kuhn

This comment has been minimized.

Show comment
Hide comment
@m-kuhn

m-kuhn Sep 13, 2017

Member

Well, for me, a different user or a different layer (no transaction group) is the same thing a far a the database is concerned: it's a different connection (or am I mistaken?)

The question I wanted to ask is, if we can avoid cache invalidation (which happens unconditionally if dataChanged() is emitted) when a notification is emitted for actions of which we are already aware. I don't think transaction groups need special attention.

Member

m-kuhn commented Sep 13, 2017

Well, for me, a different user or a different layer (no transaction group) is the same thing a far a the database is concerned: it's a different connection (or am I mistaken?)

The question I wanted to ask is, if we can avoid cache invalidation (which happens unconditionally if dataChanged() is emitted) when a notification is emitted for actions of which we are already aware. I don't think transaction groups need special attention.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 13, 2017

Contributor

avoid cache invalidation

OK, got it now. Does the triggerRepaint invalidate cache ? Because I kinda copied the behavior from the timer refresh (triggerRepaint, not dataChanged as pointed out by @nyalldawson).

With the connection on actions I'm about to add, you can do whatever you want with the signal, does this address your concerns ?

Contributor

vmora commented Sep 13, 2017

avoid cache invalidation

OK, got it now. Does the triggerRepaint invalidate cache ? Because I kinda copied the behavior from the timer refresh (triggerRepaint, not dataChanged as pointed out by @nyalldawson).

With the connection on actions I'm about to add, you can do whatever you want with the signal, does this address your concerns ?

@m-kuhn

This comment has been minimized.

Show comment
Hide comment
@m-kuhn

m-kuhn Sep 13, 2017

Member

triggerRepaint invalidates the cached, painted image, it's a trivial one because no feature ids or cached attributes are involved.

What do you think how should be dealt with

  • Attribute tables which are open
  • The snapping cache
  • The feature count
  • ... other things in core
  • ... many more things in plugins

I think there's a general need for a signal that says "hey, something on the layer changed but we don't have enough information to tell you exactly what, if you want to stay on the safe side reload whatever you have" (that's what the dataChanged() signal is meant for).

I can't offer a complete solution, I guess it's just not a trivial topic. Just bringing a braindump to the table here.

Member

m-kuhn commented Sep 13, 2017

triggerRepaint invalidates the cached, painted image, it's a trivial one because no feature ids or cached attributes are involved.

What do you think how should be dealt with

  • Attribute tables which are open
  • The snapping cache
  • The feature count
  • ... other things in core
  • ... many more things in plugins

I think there's a general need for a signal that says "hey, something on the layer changed but we don't have enough information to tell you exactly what, if you want to stay on the safe side reload whatever you have" (that's what the dataChanged() signal is meant for).

I can't offer a complete solution, I guess it's just not a trivial topic. Just bringing a braindump to the table here.

@m-kuhn

This comment has been minimized.

Show comment
Hide comment
@m-kuhn

m-kuhn Sep 13, 2017

Member

What could be done is to add a map with extended information to the dataChanged( information ) signal which points out if

  • feature ids (might) have been changed
  • geometries (might) have been changed
  • features (might) have been added
  • features (might) have been deleted

Other information groups like these may be better. However, I think some kind of "hints" about what (potentially could have) changed will be useful for big projects with many users working in parallel to allow slots to filter what they are interested in.

Member

m-kuhn commented Sep 13, 2017

What could be done is to add a map with extended information to the dataChanged( information ) signal which points out if

  • feature ids (might) have been changed
  • geometries (might) have been changed
  • features (might) have been added
  • features (might) have been deleted

Other information groups like these may be better. However, I think some kind of "hints" about what (potentially could have) changed will be useful for big projects with many users working in parallel to allow slots to filter what they are interested in.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 13, 2017

Contributor

@m-kuhn to be on the safe side, we would be better of with a dataChanged signal in this case (same holds for timer refresh I guess).

We could also have a switch to chose which signal is to be emitted... but that is not user-friendly IMO

Custom signaling could also be achieved in the layer actions.

I like the added checkboxes in the rendering tab, because it advertises the notification from provider feature. The "actions on notify" will be more flexible, bit less discoverable, so it's nice to have both options.

Concerning the granularity of "dataChanged"... it's a good idea, but beyond the scope of what I'm trying to do here.

Contributor

vmora commented Sep 13, 2017

@m-kuhn to be on the safe side, we would be better of with a dataChanged signal in this case (same holds for timer refresh I guess).

We could also have a switch to chose which signal is to be emitted... but that is not user-friendly IMO

Custom signaling could also be achieved in the layer actions.

I like the added checkboxes in the rendering tab, because it advertises the notification from provider feature. The "actions on notify" will be more flexible, bit less discoverable, so it's nice to have both options.

Concerning the granularity of "dataChanged"... it's a good idea, but beyond the scope of what I'm trying to do here.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 13, 2017

Contributor

@luipir you have the GUI part for "notification triggered action" in the last commit. Here is what it looks like

screenshot

If someone can give me an OK for the addition of notification message to QgsAction, I'll finish implementation.

Contributor

vmora commented Sep 13, 2017

@luipir you have the GUI part for "notification triggered action" in the last commit. Here is what it looks like

screenshot

If someone can give me an OK for the addition of notification message to QgsAction, I'll finish implementation.

@luipir

This comment has been minimized.

Show comment
Hide comment
@luipir

luipir Sep 13, 2017

Contributor

great @vmora, but if actions can be triggered, it still make sense to have a refresh config parameter instead of a standard repaint/refresh action?
BTW, +1 for the GUI to setup triggerign message. Make sense to have it as a QStringList of possibile listened topics that activate the action? e.g. insert, modified

I really like the fine granularity that can be managed creating specific topics... e.g. an action can listen a specific record field modification if a specific NOTFY topic is set.

Contributor

luipir commented Sep 13, 2017

great @vmora, but if actions can be triggered, it still make sense to have a refresh config parameter instead of a standard repaint/refresh action?
BTW, +1 for the GUI to setup triggerign message. Make sense to have it as a QStringList of possibile listened topics that activate the action? e.g. insert, modified

I really like the fine granularity that can be managed creating specific topics... e.g. an action can listen a specific record field modification if a specific NOTFY topic is set.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 13, 2017

Contributor

it still make sense to have a refresh config parameter instead of a standard repaint/refresh action?

See my comment above concerning the advertisement of the feature. I think it still make some sense.

Contributor

vmora commented Sep 13, 2017

it still make sense to have a refresh config parameter instead of a standard repaint/refresh action?

See my comment above concerning the advertisement of the feature. I think it still make some sense.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 13, 2017

Contributor

/!\ the commit has not solved the test failure which is, I think, timing dependent. I tried to sync the listener but I'm not sure I succeeded. So I may have introduced an intermittent test failure with my unit test.

Contributor

vmora commented Sep 13, 2017

/!\ the commit has not solved the test failure which is, I think, timing dependent. I tried to sync the listener but I'm not sure I succeeded. So I may have introduced an intermittent test failure with my unit test.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 13, 2017

Contributor

@haubourg if you want to make another demo video, you can now do virtually anything (see example below)

capture d ecran_2017-09-13_21-17-22

@luipir thanks for the action idea.

Contributor

vmora commented Sep 13, 2017

@haubourg if you want to make another demo video, you can now do virtually anything (see example below)

capture d ecran_2017-09-13_21-17-22

@luipir thanks for the action idea.

@vmora vmora changed the title from [FEATURE] refresh layer on provider notification to [FEATURE] layer refresh and trigger actions on provider notification Sep 14, 2017

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 14, 2017

Contributor

@nyalldawson the test should be ok now (loop+timeout), so if everyone is happy with that I'll rebase/squash.

I'd especially like opinions on the QgsAction modifications.

Contributor

vmora commented Sep 14, 2017

@nyalldawson the test should be ok now (loop+timeout), so if everyone is happy with that I'll rebase/squash.

I'd especially like opinions on the QgsAction modifications.

void QgsActionManager::onNotifyRunActions( const QString &message )
{
for ( const QgsAction &act : qgsAsConst( mActions ) )

This comment has been minimized.

@m-kuhn

m-kuhn Sep 14, 2017

Member

Better iterate on a copy of mActions, while it's totally unlikely we never know if the container is not modified inside runAction and we don't want to risk a crash.

@m-kuhn

m-kuhn Sep 14, 2017

Member

Better iterate on a copy of mActions, while it's totally unlikely we never know if the container is not modified inside runAction and we don't want to risk a crash.

This comment has been minimized.

@vmora

vmora Sep 14, 2017

Contributor

we pass a copy of the action to runAction, so we are safe, or I'm missing something.

@vmora

vmora Sep 14, 2017

Contributor

we pass a copy of the action to runAction, so we are safe, or I'm missing something.

QgsExpressionContext context = createExpressionContext();
Q_ASSERT( mLayer ); // if there is no layer, then where is the notification coming from ?
context << QgsExpressionContextUtils::layerScope( mLayer );

This comment has been minimized.

@m-kuhn

m-kuhn Sep 14, 2017

Member

Wouldn't it be very handy to also add make the notification command available in the expression context?

@m-kuhn

m-kuhn Sep 14, 2017

Member

Wouldn't it be very handy to also add make the notification command available in the expression context?

This comment has been minimized.

@vmora

vmora Sep 14, 2017

Contributor

clearly, this would allow to use the message in the function, I was looking for a way to to that, thanks.

How do you do that ?

I was considering regex matching instead of '==' for filtering, but @mhugo mentioned matching the beginning of the message only (like a header), that would allow to send complex messages (like a list of modified features) and, with the message in the context, this would be really powerful.

@vmora

vmora Sep 14, 2017

Contributor

clearly, this would allow to use the message in the function, I was looking for a way to to that, thanks.

How do you do that ?

I was considering regex matching instead of '==' for filtering, but @mhugo mentioned matching the beginning of the message only (like a header), that would allow to send complex messages (like a list of modified features) and, with the message in the context, this would be really powerful.

This comment has been minimized.

@3nids

3nids Sep 14, 2017

Member

jumping in... I have been secretly hoping for regex support here, but was afraid to get banned ;)

@3nids

3nids Sep 14, 2017

Member

jumping in... I have been secretly hoping for regex support here, but was afraid to get banned ;)

This comment has been minimized.

@luipir

luipir Sep 14, 2017

Contributor

@3nids +1 for regex

@luipir

luipir Sep 14, 2017

Contributor

@3nids +1 for regex

This comment has been minimized.

@vmora

vmora Sep 14, 2017

Contributor

@luipir, @3nids sorry but which kind of regex and which regex engine are we talking about ? ;)

With the last commit, the notification message is filtered on the beginning of the message, so it's nice because the user just has to specified my_header for the action, and anything beginning with my_header will trigger the action. I think I prefer that to my_header.* or my_header%.

Note, you have a limitation of the message length (8000 bytes by default) so nothing too fancy in there.

@vmora

vmora Sep 14, 2017

Contributor

@luipir, @3nids sorry but which kind of regex and which regex engine are we talking about ? ;)

With the last commit, the notification message is filtered on the beginning of the message, so it's nice because the user just has to specified my_header for the action, and anything beginning with my_header will trigger the action. I think I prefer that to my_header.* or my_header%.

Note, you have a limitation of the message length (8000 bytes by default) so nothing too fancy in there.

This comment has been minimized.

@luipir

luipir Sep 14, 2017

Contributor

I suppose @3nids was talking to add also a regexp for topic action activation => not only need to match exactly but match some regexp => from the db side can be se sent namespaced notifications.
About size limitation it is referred to paylod (e.g. the value sent with the event) not related with the topic name.

@luipir

luipir Sep 14, 2017

Contributor

I suppose @3nids was talking to add also a regexp for topic action activation => not only need to match exactly but match some regexp => from the db side can be se sent namespaced notifications.
About size limitation it is referred to paylod (e.g. the value sent with the event) not related with the topic name.

This comment has been minimized.

@haubourg

haubourg Sep 15, 2017

Contributor

Not sure I understand that clearly;
@luipir what is what you call "topic action activation" ? Is that the Channel name of the notify you are talking about? Currently it's fixed to qgis, and we thought that the payload offered lot's of flexibility.
Current header style can be matched and then pass any free text up to 8000 bytes. Any JSON, XML or whatever free text can be send and then filtered in the action code.
Maybe some basic examples could help.

@haubourg

haubourg Sep 15, 2017

Contributor

Not sure I understand that clearly;
@luipir what is what you call "topic action activation" ? Is that the Channel name of the notify you are talking about? Currently it's fixed to qgis, and we thought that the payload offered lot's of flexibility.
Current header style can be matched and then pass any free text up to 8000 bytes. Any JSON, XML or whatever free text can be send and then filtered in the action code.
Maybe some basic examples could help.

This comment has been minimized.

@luipir

luipir Sep 15, 2017

Contributor

sorry I used the ROS/MQTT terminology... topic = channel name

@luipir

luipir Sep 15, 2017

Contributor

sorry I used the ROS/MQTT terminology... topic = channel name

}
void QgsActionManager::removeAction( const QUuid &actionId )
{
int i = 0;
Q_FOREACH ( const QgsAction &action, mActions )
for ( const QgsAction &action : qgsAsConst( mActions ) )
{
if ( action.id() == actionId )
{
mActions.removeAt( i );

This comment has been minimized.

@m-kuhn

m-kuhn Sep 14, 2017

Member

You definitely have to take a copy of mActions here or there be dragons.

@m-kuhn

m-kuhn Sep 14, 2017

Member

You definitely have to take a copy of mActions here or there be dragons.

This comment has been minimized.

@vmora

vmora Sep 14, 2017

Contributor

so, no qgsAsConst( mActions ) if mActions is not const hugh ?

@vmora

vmora Sep 14, 2017

Contributor

so, no qgsAsConst( mActions ) if mActions is not const hugh ?

This comment has been minimized.

@m-kuhn

m-kuhn Sep 14, 2017

Member

Not if inside the loop mActions.removeAt( i ) is called

@m-kuhn

m-kuhn Sep 14, 2017

Member

Not if inside the loop mActions.removeAt( i ) is called

This comment has been minimized.

@m-kuhn

m-kuhn Sep 14, 2017

Member

As far as I know qgsAsConst doesn't take a copy, does it @nyalldawson ?

@m-kuhn

m-kuhn Sep 14, 2017

Member

As far as I know qgsAsConst doesn't take a copy, does it @nyalldawson ?

This comment has been minimized.

@vmora

vmora Sep 14, 2017

Contributor

next line is a break, so I'm not sure it matters, or am I missing something ?

@vmora

vmora Sep 14, 2017

Contributor

next line is a break, so I'm not sure it matters, or am I missing something ?

This comment has been minimized.

@vmora

vmora Sep 14, 2017

Contributor

but I would expect an undefined behavior if I were to take the next value (typical usecase for a reverse loop or something)

@vmora

vmora Sep 14, 2017

Contributor

but I would expect an undefined behavior if I were to take the next value (typical usecase for a reverse loop or something)

This comment has been minimized.

@m-kuhn

m-kuhn Sep 14, 2017

Member

Nope, you are right, sorry.

@m-kuhn

m-kuhn Sep 14, 2017

Member

Nope, you are right, sorry.

This comment has been minimized.

@vmora

vmora Sep 14, 2017

Contributor

pfffff, you had me frightened there ! But you also had me thinking, which is a good thing. Thanks.

@vmora

vmora Sep 14, 2017

Contributor

pfffff, you had me frightened there ! But you also had me thinking, which is a good thing. Thanks.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 14, 2017

Contributor

@m-kuhn now the only thing that I have difficulty with is to add the notification_message to the expression generator. I'll continue searching, but pointers would be really appreciated.

Contributor

vmora commented Sep 14, 2017

@m-kuhn now the only thing that I have difficulty with is to add the notification_message to the expression generator. I'll continue searching, but pointers would be really appreciated.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 14, 2017

Contributor

@m-kuhn what I just did works, but I can't help but think it's a hack. What do you think ?

Contributor

vmora commented Sep 14, 2017

@m-kuhn what I just did works, but I can't help but think it's a hack. What do you think ?

@m-kuhn

This comment has been minimized.

Show comment
Hide comment
@m-kuhn

m-kuhn Sep 14, 2017

Member

You could make a QgsExpressionContextUtils::notificationScope() but I think you got it quite right.

And there's also some help text you can add over at void QgsExpression::initVariableHelp()

Member

m-kuhn commented Sep 14, 2017

You could make a QgsExpressionContextUtils::notificationScope() but I think you got it quite right.

And there's also some help text you can add over at void QgsExpression::initVariableHelp()

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 14, 2017

Contributor

@m-kuhn thanks a lot for the pointers, feels a lot less hacky now.

With that we should be all set. Who is going to +1 for merge ?

Contributor

vmora commented Sep 14, 2017

@m-kuhn thanks a lot for the pointers, feels a lot less hacky now.

With that we should be all set. Who is going to +1 for merge ?

@m-kuhn

This comment has been minimized.

Show comment
Hide comment
@m-kuhn

m-kuhn Sep 15, 2017

Member

@vmora I really like this functionality (of course I do, I think it originates from a discussion I had with @mhugo ;) ), it's great to have dynamic updates and to be able to trigger custom business logic by server-side commands, this will allow for much more integrated systems.

What I'm still not completely sure is if the system can be adjusted (if required later on) to give more precise hints on which - qgis internal - caches need to be invalidated and which ones not. I see no need to implement this now, but I think it's a must to keep the possibility open.

Member

m-kuhn commented Sep 15, 2017

@vmora I really like this functionality (of course I do, I think it originates from a discussion I had with @mhugo ;) ), it's great to have dynamic updates and to be able to trigger custom business logic by server-side commands, this will allow for much more integrated systems.

What I'm still not completely sure is if the system can be adjusted (if required later on) to give more precise hints on which - qgis internal - caches need to be invalidated and which ones not. I see no need to implement this now, but I think it's a must to keep the possibility open.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 15, 2017

Contributor

@luipir with the latest commits the "not only need to match exactly but match some regexp" is not satisfied, but I think the current activation with "not only need to match exactly but begins with" is easier to use and sufficient to filter notifications.

@3nids do you agree ?

@m-kuhn if @luipir and @3nids reeeeeeeealy want regex, I'd love an answer to the "which regex engine" question. If they are ok with the current "begins with", then all is well.

about size limitation: that was just a remark related to sending something else after the "header", the something_else must satisfy sizeof(something_else) + sizeof(header) < 8000

Contributor

vmora commented Sep 15, 2017

@luipir with the latest commits the "not only need to match exactly but match some regexp" is not satisfied, but I think the current activation with "not only need to match exactly but begins with" is easier to use and sufficient to filter notifications.

@3nids do you agree ?

@m-kuhn if @luipir and @3nids reeeeeeeealy want regex, I'd love an answer to the "which regex engine" question. If they are ok with the current "begins with", then all is well.

about size limitation: that was just a remark related to sending something else after the "header", the something_else must satisfy sizeof(something_else) + sizeof(header) < 8000

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 15, 2017

Contributor

@m-kuhn, as mentioned earlier, the refresh layer on notification (triggerRepaint) is now here to "advertise" the "trigger action on notification" feature. With the later, you can choose whatever cache you want invalidated as long as the invalidation as a python API.

Contributor

vmora commented Sep 15, 2017

@m-kuhn, as mentioned earlier, the refresh layer on notification (triggerRepaint) is now here to "advertise" the "trigger action on notification" feature. With the later, you can choose whatever cache you want invalidated as long as the invalidation as a python API.

@m-kuhn

This comment has been minimized.

Show comment
Hide comment
@m-kuhn

m-kuhn Sep 15, 2017

Member

@m-kuhn, as mentioned earlier, the refresh layer on notification (triggerRepaint) is now here to "advertise" the "trigger action on notification" feature. With the later, you can choose whatever cache you want invalidated as long as the invalidation as a python API.

For business logic, yes.
For an attribute table reload or other internal changes, no, there should not be any requirement to write custom python code.

Member

m-kuhn commented Sep 15, 2017

@m-kuhn, as mentioned earlier, the refresh layer on notification (triggerRepaint) is now here to "advertise" the "trigger action on notification" feature. With the later, you can choose whatever cache you want invalidated as long as the invalidation as a python API.

For business logic, yes.
For an attribute table reload or other internal changes, no, there should not be any requirement to write custom python code.

@nyalldawson

This comment has been minimized.

Show comment
Hide comment
@nyalldawson

nyalldawson Sep 15, 2017

Contributor

If any, this one: http://doc.qt.io/qt-5/qregexp.html

Nooooo this one: http://doc.qt.io/qt-5/qregularexpression.html

Let's save ourselves SOME work when Qt upstream drops the next round of classes they dislike!

Contributor

nyalldawson commented Sep 15, 2017

If any, this one: http://doc.qt.io/qt-5/qregexp.html

Nooooo this one: http://doc.qt.io/qt-5/qregularexpression.html

Let's save ourselves SOME work when Qt upstream drops the next round of classes they dislike!

@m-kuhn

This comment has been minimized.

Show comment
Hide comment
@m-kuhn

m-kuhn Sep 15, 2017

Member

Of course, who proposed the other one? 😜

Member

m-kuhn commented Sep 15, 2017

Of course, who proposed the other one? 😜

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 15, 2017

Contributor

if the system can be adjusted (if required later on) to give more precise hints on which - qgis internal - caches need to be invalidated

and also

For an attribute table reload or other internal changes, no, there should not be any requirement to write custom python code.

@m-kuhn Sorry, I don't get what you want me to do here on that topic. I do feel stupid for that. Please elaborate.

Nooooo this one: http://doc.qt.io/qt-5/qregularexpression.html
Let's save ourselves SOME work when Qt upstream drops the next round of classes they dislike!

OK, thanks @nyalldawson.

Waiting for @3nids and @luipir to confirm that they reeeeeeally want regex there. Should be a piece of cake, but I don't feels it's necessary here.

Contributor

vmora commented Sep 15, 2017

if the system can be adjusted (if required later on) to give more precise hints on which - qgis internal - caches need to be invalidated

and also

For an attribute table reload or other internal changes, no, there should not be any requirement to write custom python code.

@m-kuhn Sorry, I don't get what you want me to do here on that topic. I do feel stupid for that. Please elaborate.

Nooooo this one: http://doc.qt.io/qt-5/qregularexpression.html
Let's save ourselves SOME work when Qt upstream drops the next round of classes they dislike!

OK, thanks @nyalldawson.

Waiting for @3nids and @luipir to confirm that they reeeeeeally want regex there. Should be a piece of cake, but I don't feels it's necessary here.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 15, 2017

Contributor

@luipir

from the db side can be se sent namespaced notifications

and

topic = channel name

I just want to be clear on what the notification mechanism does: the postgres NOTIFY command syntax is
NOTIFY channel[, payload] the channel is hard-coded and named qgis, what I call the "header" is the beginning of the payload.

I you wonder "Why not specifying/filtering on the channel too?": it's to keep the provider interface "minimal". A file-based provider can issue a notification message "file changed" but there is no channel involved there, and the concept is not necessary IMO.

If someone out there has a database running that already "NOTIFY" on another channel and wants those signals listened to, it's doable, but that's a lot of changes and should really be worth the effort.

Contributor

vmora commented Sep 15, 2017

@luipir

from the db side can be se sent namespaced notifications

and

topic = channel name

I just want to be clear on what the notification mechanism does: the postgres NOTIFY command syntax is
NOTIFY channel[, payload] the channel is hard-coded and named qgis, what I call the "header" is the beginning of the payload.

I you wonder "Why not specifying/filtering on the channel too?": it's to keep the provider interface "minimal". A file-based provider can issue a notification message "file changed" but there is no channel involved there, and the concept is not necessary IMO.

If someone out there has a database running that already "NOTIFY" on another channel and wants those signals listened to, it's doable, but that's a lot of changes and should really be worth the effort.

@luipir

This comment has been minimized.

Show comment
Hide comment
@luipir

luipir Sep 15, 2017

Contributor

@vmora yes I suppose it's necessary only to change here:
https://github.com/vmora/QGIS/blob/3f240e780f676cd303825c9b05fb4d3f2034bb3f/src/core/qgsactionmanager.cpp#L78
instead to use the simple startWith. startWith can be expressed with regular expression.
A future improvement can be some regexp preparation before (or cached in the user interface) to avoid to compile it on the fly each notification.

Contributor

luipir commented Sep 15, 2017

@vmora yes I suppose it's necessary only to change here:
https://github.com/vmora/QGIS/blob/3f240e780f676cd303825c9b05fb4d3f2034bb3f/src/core/qgsactionmanager.cpp#L78
instead to use the simple startWith. startWith can be expressed with regular expression.
A future improvement can be some regexp preparation before (or cached in the user interface) to avoid to compile it on the fly each notification.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 15, 2017

Contributor

@luipir as opposed to the comment I just made about "channel" selection, I know it's a trivial change to take regex into account. My only concern was the clarity of the interface and the actual need for for this flexibility.

Contributor

vmora commented Sep 15, 2017

@luipir as opposed to the comment I just made about "channel" selection, I know it's a trivial change to take regex into account. My only concern was the clarity of the interface and the actual need for for this flexibility.

@luipir

This comment has been minimized.

Show comment
Hide comment
@luipir

luipir Sep 15, 2017

Contributor

@vmora ok, no problem to leave it simple... btw I don't know how much is more complex expressing:
topic% instead of ^topic.*$

Contributor

luipir commented Sep 15, 2017

@vmora ok, no problem to leave it simple... btw I don't know how much is more complex expressing:
topic% instead of ^topic.*$

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 15, 2017

Contributor

Beware ! Awesome regex included. But watch out for regex behavior since whatever will match whatever anywhere in the message. The previous starts-with whatever behavior now requires ^whatever (which is simple enough).

Contributor

vmora commented Sep 15, 2017

Beware ! Awesome regex included. But watch out for regex behavior since whatever will match whatever anywhere in the message. The previous starts-with whatever behavior now requires ^whatever (which is simple enough).

@haubourg

This comment has been minimized.

Show comment
Hide comment
@haubourg

haubourg Sep 15, 2017

Contributor

hum, @vmora We need to add the tag [needs-doc]to the PR. Can you write one or two examples of possible payloads more explicit to illustrate ŵhatever start-with possibilities ?

Contributor

haubourg commented Sep 15, 2017

hum, @vmora We need to add the tag [needs-doc]to the PR. Can you write one or two examples of possible payloads more explicit to illustrate ŵhatever start-with possibilities ?

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 15, 2017

Contributor

@haubourg Ctrl+F needs-docs

I will rebase/squash and improve the commit message.

Contributor

vmora commented Sep 15, 2017

@haubourg Ctrl+F needs-docs

I will rebase/squash and improve the commit message.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 15, 2017

Contributor

all done, I think the commit message is explicit enough for documentation now

Contributor

vmora commented Sep 15, 2017

all done, I think the commit message is explicit enough for documentation now

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 18, 2017

Contributor

@nyalldawson, @m-kuhn there is one failing test (PyQgsLocator timeout) that doesn't fail on my machine.

Contributor

vmora commented Sep 18, 2017

@nyalldawson, @m-kuhn there is one failing test (PyQgsLocator timeout) that doesn't fail on my machine.

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 18, 2017

Contributor

Last commit is a useless rebase on master to trigger rebuild.

Contributor

vmora commented Sep 18, 2017

Last commit is a useless rebase on master to trigger rebuild.

@m-kuhn

This comment has been minimized.

Show comment
Hide comment
@m-kuhn

m-kuhn Sep 18, 2017

Member

@m-kuhn Sorry, I don't get what you want me to do here on that topic. I do feel stupid for that. Please elaborate.

I wonder if we just need to send (an initially empty) struct/class parameter that could contain some flags for internal usage containing more details about what changed to determine if some caches should be invalidated. E.g. a geometryChanged to determine if the snapping cache needs to be invalidated or fidChanged flag (thinking of things like repack of ogr shapefiles here) to determine if everything should be dismissed.
It partially relates to this pull request because it's the first time we have the chance to give QGIS more information than "something changed" which was/is the case for repack and "reload every 5 seconds".

Does that make any sense to you? I'm not completely convinced myself, but I think it's an important question to consider before the 3.0 API is frozen. (CC @wonder-sk @nyalldawson).

Member

m-kuhn commented Sep 18, 2017

@m-kuhn Sorry, I don't get what you want me to do here on that topic. I do feel stupid for that. Please elaborate.

I wonder if we just need to send (an initially empty) struct/class parameter that could contain some flags for internal usage containing more details about what changed to determine if some caches should be invalidated. E.g. a geometryChanged to determine if the snapping cache needs to be invalidated or fidChanged flag (thinking of things like repack of ogr shapefiles here) to determine if everything should be dismissed.
It partially relates to this pull request because it's the first time we have the chance to give QGIS more information than "something changed" which was/is the case for repack and "reload every 5 seconds".

Does that make any sense to you? I'm not completely convinced myself, but I think it's an important question to consider before the 3.0 API is frozen. (CC @wonder-sk @nyalldawson).

@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 18, 2017

Contributor

Does that make any sense to you?

Not sure yet, sorry.

send (an initially empty) struct/class parameter

To who, from who ?

Right know, you can NOTIFY qgis, 'geometry_changed for feature id in (13, 14, 51)', or actually SELECT pg_notify('qgis', 'geometry_changed for feature id in ('||string_agg(..., ','))||')' from ... if you don't have too many features (msg<8ko).

There is no class/struct involved there, nor is the message standardized in any way.

Now, once you receive the notification in qgis, you could have a special handler (could be the provider "listener" itself) that understand a "standard" syntax and takes "standard" actions accordingly. Then we may require some additional slots to invalidate some cache an not some other.

So if I understand you: this PR opens the possibility for a datasource to notify qgis with only one kind of notifications (message containing anything) and you'd like more granularity, like (e.g. attributeValuesChanged or geometryChanged) ?

Contributor

vmora commented Sep 18, 2017

Does that make any sense to you?

Not sure yet, sorry.

send (an initially empty) struct/class parameter

To who, from who ?

Right know, you can NOTIFY qgis, 'geometry_changed for feature id in (13, 14, 51)', or actually SELECT pg_notify('qgis', 'geometry_changed for feature id in ('||string_agg(..., ','))||')' from ... if you don't have too many features (msg<8ko).

There is no class/struct involved there, nor is the message standardized in any way.

Now, once you receive the notification in qgis, you could have a special handler (could be the provider "listener" itself) that understand a "standard" syntax and takes "standard" actions accordingly. Then we may require some additional slots to invalidate some cache an not some other.

So if I understand you: this PR opens the possibility for a datasource to notify qgis with only one kind of notifications (message containing anything) and you'd like more granularity, like (e.g. attributeValuesChanged or geometryChanged) ?

@m-kuhn

This comment has been minimized.

Show comment
Hide comment
@m-kuhn

m-kuhn Sep 18, 2017

Member

I was thinking about the internal signalling (something like dataChanged(const QgsDataChangeHints &changeHints))

How the information is put into this structure is yet a different question.

Member

m-kuhn commented Sep 18, 2017

I was thinking about the internal signalling (something like dataChanged(const QgsDataChangeHints &changeHints))

How the information is put into this structure is yet a different question.

[FEATURE] layer refresh and trigger actions on provider notification
[needs-docs]

In vector layer properties (only usefull for postgres datasources)

**in the rendering tab**

A "Refresh layer on notification" checkbox has been added to refresh layer
on provider notification.

For a postgres datasource, if a `NOTIFY qgis;` command is issued by one of the database clients,
a refresh of the layer will occur.

If the "Only if message is" checkbox is checked, the notification will trigger the refresh only
if the message contend is the one specified, e.g. if the user enters
"refresh" in the box right next to the "Only if message is" checkbox,
then a `NOTIFY qgis, 'refresh';` command in the datatabase will trigger
a layer refresh, but `NOTIFY qgis;` or `NOTIFY qgis, 'something else';`
won't.

**in the actions tab**

A column "On notification" has been added, the action editor widget
a has text field "Execute if notification message matches" to
specify a filter for notification from the provider. The filter is a
Perl-type regex.

Note that, as opposed to the "layer refresh" that

Exemple:
  - QGIS side "Execute if notification message matches" `^trigger my action`
  - Postgres side: `NOTIFY qgis, 'trigger my action'` will trigger the action
  - Postgres side: `NOTIFY qgis, 'trigger my action some additional data'` will trigger the action
  - Postgres side: `NOTIFY qgis, 'do not trigger my action some additional data'` will NOT trigger the action

Please note that if the `^`, which means "starts with",  in `^trigger my action` had been ommited,
the last notification would have triggered the action because the
notification message contains the `trigger my action`

A new qgis variable `notification_message` is available for use in
actions, it holds the contend of the notification message. To continue
with the previous exemple, if the action is of python type with the code:

```python
print('[% @notification_message %]')
```

The three notifictions above will result in two printed lines
```
trigger my action
trigger my action some additional data
```

User Warning:

For postgres providers, if the "Refresh layer on notification" is checked, or if one layer action has
"On notification" specified, a new connection to the database is made to
listen to postgres notifications. This olds even if transaction groups
are enabled at the project level.

Note that once the notification mechanism is started in a QGIS
session, it will not stop, even if there is no more need for it (Refresh
layer on notification" unchecked and no "On notification" in any
action). Consequently the connection listening to notification will
remain open.

IMPLEMENTATION DETAILS:

A notify signal has been added to the abstract QgsVectorDataProvider
along with a setListening function that enables/disble the notification
mechanism.

For the moment only the postgres provider implements the notification.

QgsAction has a notificationMessage member function that holds the regex
to match to trigger action

QgsActionManager becomes a QObject and is doing the filtering and execute actions on
notifications.

The notification notion extends beyond SRGBD servers (postgres and oracle at
least have the notify) and the "watch file" in the delimitedtext
provider could also benefit from this interface.

For the postgres provider a thread is created with a second connection
to the database. This thread is responsible for listening postgres
notifications.

It would be nice to avoid the creation of one listening chanel per
provider in the case transaction groups are enabled.

Please note that when listening starts (a thread and connection is
created in the postgres provider) it cannot be stopped by removing the
connected actions or unchecking the refresh check box. Indeed, since we
don't know who needs the signals, we dont't want to stop the service.

The service will not restart in the next qgis session though.

If this behavior is not deemed appropriate, we could use
```
int QObject::receivers ( const char * signal ) const
```
and have QgsDataProvider::setListening return a bool to tell the caller
if the signal has actually been closed.
@vmora

This comment has been minimized.

Show comment
Hide comment
@vmora

vmora Sep 22, 2017

Contributor

@m-kuhn can the PR be accepted or are you waiting for something ? If the former, can you be specific ? If the later, can @nyalldawson or yourself merge that please ? Resolving .ui conflicts is not my cup of tea.

Contributor

vmora commented Sep 22, 2017

@m-kuhn can the PR be accepted or are you waiting for something ? If the former, can you be specific ? If the later, can @nyalldawson or yourself merge that please ? Resolving .ui conflicts is not my cup of tea.

@m-kuhn

This comment has been minimized.

Show comment
Hide comment
@m-kuhn

m-kuhn Sep 22, 2017

Member

Let's just merge this for testing soon. It's an awesome feature which I really like to see land!
Let's see how this turns out and we can still add a second signal that is emitted next to dataChanged() once we get into the situation where we actually want to have more precise controls.

Member

m-kuhn commented Sep 22, 2017

Let's just merge this for testing soon. It's an awesome feature which I really like to see land!
Let's see how this turns out and we can still add a second signal that is emitted next to dataChanged() once we get into the situation where we actually want to have more precise controls.

@m-kuhn m-kuhn merged commit 6dfe44f into qgis:master Sep 22, 2017

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@vmora vmora deleted the vmora:listen_notify branch Sep 25, 2017

@mayeulk

This comment has been minimized.

Show comment
Hide comment
@mayeulk

mayeulk Oct 18, 2017

Hi, looks awesome. I guess we could have applications such as live monitoring of movements (bus; trains; trucks;safety vehicles...) such as in https://traintimes.org.uk/map/london-buses/#73
Correct?

mayeulk commented Oct 18, 2017

Hi, looks awesome. I guess we could have applications such as live monitoring of movements (bus; trains; trucks;safety vehicles...) such as in https://traintimes.org.uk/map/london-buses/#73
Correct?

@haubourg

This comment has been minimized.

Show comment
Hide comment
@haubourg

haubourg Oct 18, 2017

Contributor

@mayeulk yes exactly! You could already use a timer to refresh the canvas, but now, you can trigger the refresh only when needed, from a database perspective.

Contributor

haubourg commented Oct 18, 2017

@mayeulk yes exactly! You could already use a timer to refresh the canvas, but now, you can trigger the refresh only when needed, from a database perspective.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment