Skip to content

Failing push notification #770

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ageru opened this issue May 14, 2017 · 153 comments
Closed

Failing push notification #770

ageru opened this issue May 14, 2017 · 153 comments

Comments

@ageru
Copy link

ageru commented May 14, 2017

Hi,

Sorry to create yet another bug about push...

Setup

  • Server: prosody .9.12 with CA-signed TLS certificate
  • Relevant modules: cloud_notify, smacks, pinger, carbons, mam, csi, filter_chatstates, throttle_presence.
  • IPhone SE with latest iOS. Low battery mode off, all is ON in General Settings > ChatSecure.
  • Chatsecure 4.0.8

Steps to reproduce

  1. Install Chatsecure, log to server
  2. Send a couple of messages to user on iPhone, they reach him, OMEMO works... Everything looks clear.
  3. Confirmed push is active in the app's interface
  4. Leave the app.

Actual result

  • Quickly, the iPhone user shows as away on other devices, then as offline.
  • Messages don't reach the device unless the iPhone's user starts the app
  • The iPhone's user has to specifically open the app to get the sent messages
  • Relevant portions of the server log:
    • Successfully initiating push (it seems):
      May 14 HH:MM:SS [alphanum characters] info Push notifications enabled for [ID]@[chatdomain.tld]/chatsecure (pubsub.chatsecure.org<A3486BB3-6442-4945-84E2-[more alphanum characters])

    • When trying to send a message to offline device (the last number increments with every try of the server):
      May 14 HH:MM:SS [chatdomain.tld]:cloud_notify info Got error of type 'cancel' (not-allowed) for identifier 'pubsub.chatsecure.org<A3486BB3-6442-4945-84E2-[alphanum characters]': error count for this identifier is now at 1

Expected Result
If iPhone is connected to the Internet, the iPhone user gets a notification straight away.

Bonus questions

  • What does mod_pinger do? I can make an educated guess looking at the name and bits of its code, but I'm not 100% certain.
  • Would it be possible to get a wiki of FAQ detailing exactly what modules in prosody are needed to use all the features of ChatSecure? @chrisballinger gives a few times his config in threads here and there, but that's hardly discoverable and probably not comprehensive or up to date.

Many thanks.

@chrisballinger
Copy link
Member

Thanks for the thorough investigation. I've never seen that error message before from the push module. It looks like the code has been updated recently: https://hg.prosody.im/prosody-modules/file/tip/mod_cloud_notify/mod_cloud_notify.lua#l73

Perhaps try rolling back mod_cloud_notify to a (much older) commit?

@KingKili
Copy link

I have the same issue with https://trashserver.net/

@ageru
Copy link
Author

ageru commented May 15, 2017

All the modules are automatically updated weekly on my server (and I didn't have any ChatSecure user before the past week-end) so that would be consistent with your theory. As a temporary fix, rolling back to an earlier commit should be possible, I just hope it won't break things with the users running Conversations.

Would you be able to let me know what commit of this module you are using, or alternatively to date it?

By the way, when I noticed the issue, I installed mod_pinger to try to fix it, since you seemed to be using it. Is it in any way necessary to have it installed for ChatSecure and Push to work properly?

@chrisballinger
Copy link
Member

mod_pinger shouldn't be required. Not sure the exact mod_cloud_notify commit, but I'll figure that out and test against the latest commit. Push is still working fine as of a few minutes ago on my personal server configuration.

@paskalito
Copy link

could this be avoided using this push mechanism ?
#650 (comment)

@beeloo42
Copy link

I'm also encountering this issue
I tried the 5 latest mod_cloud_notify versions without success...

@afriedmanGlacier
Copy link
Contributor

We are using eJabber and have changed a bit of the underlying functionality of push, but just thought I'd chime in and say push still works ok for us. However...

@chrisballinger the one problem I have seen lately, I thought I'd add here just in case it is relevant to these issues is that our RubDub has died unexpectedly a few times. Haven't figured out why yet (and we also haven't updated to the RubDub with Rollbar yet). When this happened, eJabber could not connect to it and push stopped working. Have you seen this at all Chris? I'm not sure if you guys use the same RubDub and ChatSecure-Push-Server setup as is available on github?

If you have not had this problem, we'll just monitor it on our end, but just thought I'd add this here in case it is relevant.

@chrisballinger
Copy link
Member

Yeah it does sometimes, but I re-provisioned our server don't see this anymore. I have an external port monitor that emails me when it goes down, and it hasn't had an issue since then. (Ubuntu 16.04 and latest Node PPA.)

@therob84
Copy link

Exactly the same problem I encounter with two of my contacts who use ChatSecure 2.0.8 (while I am using Conversations 1.18.4). All of us are using jabber.de (prosody >0.9.10)

  • At iOS devices Push is active and running (as ChatSecure me confirmed).
  • For about 1-2 days delivering messages to the iOS-devices works perfectly:
    • I compose a message (Conversations) while iOS-users are displayed as offline
    • triggered by the typing-notifier iOS-users are shown as "online" for some seconds
    • i send the message some time later, iOS again return to "online", receives the message, send an delivery-notifier - perfect!
  • But after 2-3 days this does not work anymore and the iOS user has to reopen ChatSecure to get Push activated again (which is not very userful for long-term conversations....), otherwise the message never will be delivered.

I think it is the same problem than stated above, but I could not provide any logfile but are eager to help solving this problem.

Is the reason now already undrestood or is it still a mystery, why/when/where this happens?

I hope the best for a fix in the next release....cheers, Robert

@chrisballinger
Copy link
Member

chrisballinger commented May 23, 2017 via email

@therob84
Copy link

therob84 commented May 23, 2017 via email

@chrisballinger
Copy link
Member

chrisballinger commented May 23, 2017 via email

@therob84
Copy link

therob84 commented May 24, 2017

@chrisballinger
Anwser from jabber.de-guys are:

  • "Server is restarted about once a month" ....so I don't think this is the explanation for me (but can't say for sure)
  • they "can't find the ability for storing a session" (I guess in mod_cloud_notify)
  • but they updated today the module Cloud_Notify (don't know between which versions) and I will check if it is improved (should the ChatSecure-Apps be restarted or should they use the new modul version automatically in background?)

Any more suggestion about the problem from your side?
For my information: is anywhere written how the ChatSecure-Push-Servers are working together with the iOS devices and with the jabber-servers? I can't understand how it works in principle until now...

Edit1:
5 days after the Cloud_Notify module was updated at jabber.de it seems the situation is improved (I can reach 3 contacts reliable within the last 5 days) But it is too early to say anything for long-term-stability.

Edit2:
2 weeks later I can NOT confirm any success in reaching my iOS-contacts. Only one contact is reachable anymore. But I think this is connected to the problem that ChatSecure/push-service is not restarted automatically after "restartingOS/swiping CS away/kill CS"... Which is a very deadly drawback and makes CS nearly unusable by iOS-contacts which do not care that much about "always checking if CS is launched and running" ......... :-(

@therob84
Copy link

Two more questions I want to raise:

  • how does ChatSecures push-functionality behaves, after CS is activley "swiped out" in taskmanager or even forced killed?
  • Or if iOS is freshly restarted an ChatSecure is not yet launched manually by user interaction?

--> Will CS (re-)establish a Push-Session in this scenarios or is there first an user activity needed (which would not be that relieable and practicable)?

@chrisballinger
Copy link
Member

chrisballinger commented May 30, 2017 via email

@mimi89999
Copy link
Contributor

Zom-iOS has the same issues, right?

@chrisballinger
Copy link
Member

chrisballinger commented May 30, 2017 via email

@mimi89999
Copy link
Contributor

But how does Facebook, Twitter, Snapchat, etc. work then? They don't do VoIP and users are still able to get notifications after swiping the app and after a [re]boot.

@chrisballinger
Copy link
Member

chrisballinger commented May 31, 2017 via email

@mimi89999
Copy link
Contributor

Can't we send you have unread messages. Please open the app to see them or something similar then?

@chrisballinger
Copy link
Member

chrisballinger commented May 31, 2017 via email

@mimi89999
Copy link
Contributor

@chrisballinger It would be a huge UX improvement. Users won't need to remember not to swipe the app from recent or to launch it after [re]boot.

As for presence, status, etc. smacks does that very well. We could also check for the <body> tag in the stanza in the cloud_notify module and send a push message only if there is one.

@mimi89999
Copy link
Contributor

@chrisballinger Will it require a lot of work to implement those high priority push notifications in the app and on the node?

@therob84
Copy link

therob84 commented Jun 7, 2017

I just edited my comment above, about only updating the Cloud_Notify module on the prosody-server (of jabber.de) can NOT solve the problem that ChatSecure/push-service is not restarted automatically after "restartingOS/swiping CS away/kill CS"... And thus is a very deadly drawback and makes CS nearly unusable by iOS-contacts which do not care that much about "always checking if CS is launched and running" ......... :-(

@chrisballinger
Copy link
Member

@therob84 I have plans to improve this in an upcoming release. Going to take the approach of sending generic foreground "New Message" notifications and possibly "Someone is typing..." as well.

@paskalito
Copy link

@chrisballinger
Hi that's sounds so great to me!! thank you!
i wanted to post a bounty on this, but i somehow could not find the issue there.. - did you stop supporting bountysource? ( looks like the last push bounty didn't end well..)
btw. is the project on gratipay (couldn't find it) or liberpay?

@chrisballinger
Copy link
Member

@paskalito You can now set up a monthly donation via in-app purchase within the app. I also have PayPal and Bitcoin options if you'd like to give a little extra:

Thanks for your support!

@therob84
Copy link

@chrisballinger: I saw your announcment for beta-testing 4.1.0 at twitter (with http-upload and inline images) - great !!!!!!
Could you also include already some improvments about behaviour of push messages (after restart,kill-app etc.) in this release?
Great work, chris!

@chrisballinger
Copy link
Member

chrisballinger commented Jun 18, 2017 via email

@tmolitor-stud-tu
Copy link

FYI: I just pushed my proposed changes to modules.prosody.im over here: https://hg.prosody.im/prosody-modules/rev/df86ce6bb0b4

(the one I talked about over here: #770 (comment))

@ageru
Copy link
Author

ageru commented Apr 9, 2018

Hi @tmolitor-stud-tu and everyone,

So just to be sure I get everything right...

Currently, as a Prosody server admin, if I want my Chatsecure users to get notifications of new messages, I need to:

  • Check for cipher suite compatibility, as per Can' connect to pubsub #822, and adapt cipher settings accordingly
  • Install the latest version of mod_cloud_notify
  • Insert a value for "push_notification_important_body", or is it not mandatory?
  • Have MAM activated?
  • ... (anything else I'm forgetting?)
  • PROFIT?

Thanks again for all your contributions on this issue.

@tmolitor-stud-tu
Copy link

@ageru Yes, push_notification_important_body has to be set to some text like "New Message".
But to be honest it can be anything because ChatSecure's AppServer strips the text off when sending the notification via APNS.
It's just needed for the AppServer to know the push is a high priority one.

@chrisballinger
Copy link
Member

@ageru @tmolitor-stud-tu Please don't send us message content, make sure to replace it with something static.

@ldvc
Copy link

ldvc commented Apr 10, 2018

In #770 (comment):

I recently talked to @weiss (ejabberd developer) and we agreed to add a new config option to include a dummy message in last-message-body (something like "New message").

@weiss is this "dummy message" still planned for ejabberd?

@weiss
Copy link

weiss commented Apr 10, 2018

@weiss is this "dummy message" still planned for ejabberd?

Yes, I'll add this soon™.

@Lyokovic
Copy link

Lyokovic commented Apr 11, 2018

Hi,

Just a comment to say that after updating my prosody modules and adding the push_notification_important_body option, push notifications now work for me.

Thanks for your work!

@danielreuther
Copy link

@weiss Sounds great. In case you have a development branch that contains it, I'd be happy to try it out.

@ageru
Copy link
Author

ageru commented Apr 12, 2018

I was despairing a bit after Lyokovic's message, since I was still getting the same "cancel" message as before.

So I tested a few things this evening, and it seems what did the trick for me was to activate S2S, which had been off until then. Does it means that S2S is required for the connection to pubsub.chatsecure.org to happen? Is that expected behaviour? This is a completely candid question, as I was expecting S2S to be for pure messaging only, not in this context.

Here is the config that was successful:

  • ChatSecure 4.3
  • Conversation 2.0.0 on the other side of the conversation
  • Prosody 0.10.0
  • All that cipher jazz set properly
  • main modules enabled: mam (internal), smacks, smacks_offline, pep_vcard_avatar, csi, filter_chatstates, throttle_presence, cloud_notify...
  • ...and S2S enabled as of a few minutes ago.

I'll keep testing in the next few day. Please do chip in with your insights and your own experience.

Fingers crossed I'm closing this one soon...

@weiss
Copy link

weiss commented Apr 12, 2018

Does it means that S2S is required for the connection to pubsub.chatsecure.org to happen? Is that expected behaviour?

Yes.

@weiss
Copy link

weiss commented Apr 17, 2018

@danielreuther

@weiss is this "dummy message" still planned for ejabberd?

Yes, I'll add this soon™.

Sounds great. In case you have a development branch that contains it, I'd be happy to try it out.

It's in the master branch, now. In order to enable it:

    modules:
      mod_push:
        include_body: "New message"

@antonyablonsky
Copy link

Hi everyone! Is there way to make s2s connection between ejabberd server and pubsub.chatsecure.org if server was not configured with DNS records and domain name like "example.com"? it just lives on my static ip and I connect straightforward, like user@ip_adress. Thanks

@chrisballinger
Copy link
Member

chrisballinger commented May 12, 2018 via email

mwlczk pushed a commit to mwlczk/docker-ejabberd that referenced this issue Jun 12, 2018
@Echolon
Copy link

Echolon commented Jul 28, 2018

Hi,

just started an EtherCalc Pad to collect information about your and others experience about push. Feel free to add your data or additional columns (suggest only to add columns where the information can be given from a normal user)

https://ethercalc.org/29iyo1nunqdp

Cheers

@ageru
Copy link
Author

ageru commented Aug 8, 2018

For the past few months, I have been running Prosody 0.10.2, and my users have been running ChatSecure 4.3 and latest. Right now:

  • I don't get anything out of the ordinary in the logs
  • iOS users mostly get push messages.

I says "mostly" because there are still 2 instances when the ChatSecure users don't get messages

  • One is with group chat, but that's an issue with MAM on Prosody, and it impacts Android also.
  • due to iOS closing the app after some time, which is kind of an expected issue at this stage of ChatSecure's development.

But those are outside the scope of my initial issue, wich I consider solved, so I'm closing tis bug. Thanks everyone for participating!

Here is how my server is set up to allow for push, in case this is useful to anyone:

  • Running prosody 0.10.2, but should mostly work on the 0.9 branch
  • S2S enabled ("Server-to-server" refers to any communication between servers, not just the transmission of messages between users on different servers. So it needs to be on for push to work, as push involves server-to-server communication.
  • dialback doesn't need to be enabled
  • TLS Ciphers set-up in a very specific way, but hopefully this will be solved at some point.
  • main modules enabled: mam (internal), smacks, smacks_offline, pep_vcard_avatar, csi, filter_chatstates, throttle_presence, cloud_notify
  • users running Conversations, Xabber, ChatSecure.

@ageru ageru closed this as completed Aug 8, 2018
@Echolon
Copy link

Echolon commented Aug 8, 2018

Better leave this issue open

@STPKITT
Copy link

STPKITT commented Aug 9, 2018

iOS users mostly get push messages.
I says "mostly" because there are still 2 instances when the ChatSecure users don't get messages

due to iOS closing the app after some time, which is kind of an expected issue at this stage of ChatSecure's development.

But isn't iOS closing the app the issue what push was actually implemented for in the first place? That's the main tripping hazard the ordinary end user faces & ultimately the deal breaker which keeps them on proprietary instant messaging services.
So if this issue stays closed, which issue # is the relevant one for this remaining problem?

@ergofroggy
Copy link

Trawling through the various issues about push notifications it's not clear what is up to date and what is not. What's the current requirement for a working prosody setup with push notifications?

  • latest version of mod_cloud_notify is clearly needed
  • do we still need the very specific cipher setup
  • what about needing to set a random value for push_notification_important_body = "New Message!"
  • other required modules? pubsub, pinger ?

@licaon-kter
Copy link
Contributor

latest version of mod_cloud_notify is clearly needed

yes

what about needing to set a random value for push_notification_important_body = "New Message!"

Is already included: https://hg.prosody.im/prosody-modules/file/tip/mod_cloud_notify/mod_cloud_notify.lua#l25

mwlczk pushed a commit to mwlczk/docker-ejabberd that referenced this issue Oct 18, 2018
mwlczk pushed a commit to mwlczk/docker-ejabberd that referenced this issue Mar 6, 2021
mwlczk pushed a commit to mwlczk/docker-ejabberd that referenced this issue Mar 6, 2021
double domain and a/v

finalized 20.01

preparation for 19.08

19.08

remove warnings about deprecated listeners

19.05

don't be to strict

ninja fixes

use 19.02 plus only a few fixes

bug-fix

move http_upload to nginx

hardening

remove not not used code

TLS: server prefered ciphers
avatar: compatibilty with old and new

bump to 18.04,
public OMEMO namespace,
preparation for ldap-roster

Push: test of pre-release mster for ChatSecure/ChatSecure-iOS#770 (comment)

19.02

temporary include post 19.02 notification fix.

max_fsm_queue of 1000 might be too small for production use. the default was 5000-10000 (since 2017.11)

Add missing erlang-inets package

- switch to proper 18.12.1 release
- install latest packages if base-image is not updated frequently
- install and cleanup package 'git' properly
- i don't want inband registration

some packages are being auto-installed and bloat the image even though they're not needed.

buster does not know erlang-ic erlang -corba erlang-percept erlang-typer
buster otp breaks on elixir.

buster is following stretch.

remove inband registration.

pre-official-release 18.12.1-branch

Add missing configuration options

EJABBERD_REGISTER_TRUSTED_NETWORK_ONLY is true by default

fixed missing default value quotes for env.get calls in ejabberd.yml.tpl

The lookup call for the environment variables for
EJABBERD_MOD_ADMIN_EXTRA and EJABBERD_MOD_VERSION are missing quotes
for the default value. They are evaluated to false, while they look like
they would evaluate to true.
mwlczk pushed a commit to mwlczk/docker-ejabberd that referenced this issue Mar 6, 2021
21.01

double domain and a/v

finalized 20.01

preparation for 19.08

19.08

remove warnings about deprecated listeners

19.05

don't be to strict

ninja fixes

use 19.02 plus only a few fixes

bug-fix

move http_upload to nginx

hardening

remove not not used code

TLS: server prefered ciphers
avatar: compatibilty with old and new

bump to 18.04,
public OMEMO namespace,
preparation for ldap-roster

Push: test of pre-release mster for ChatSecure/ChatSecure-iOS#770 (comment)

19.02

temporary include post 19.02 notification fix.

max_fsm_queue of 1000 might be too small for production use. the default was 5000-10000 (since 2017.11)

Add missing erlang-inets package

- switch to proper 18.12.1 release
- install latest packages if base-image is not updated frequently
- install and cleanup package 'git' properly
- i don't want inband registration

some packages are being auto-installed and bloat the image even though they're not needed.

buster does not know erlang-ic erlang -corba erlang-percept erlang-typer
buster otp breaks on elixir.

buster is following stretch.

remove inband registration.

pre-official-release 18.12.1-branch
mwlczk pushed a commit to mwlczk/docker-ejabberd that referenced this issue Mar 6, 2021
README with new VARIABLES is gonna follow.

dockerfile cleanup
clean up private changes

21.01

double domain and a/v

finalized 20.01

preparation for 19.08

19.08

remove warnings about deprecated listeners

19.05

don't be to strict

ninja fixes

use 19.02 plus only a few fixes

bug-fix

move http_upload to nginx

hardening

remove not not used code

TLS: server prefered ciphers
avatar: compatibilty with old and new

bump to 18.04,
public OMEMO namespace,
preparation for ldap-roster

Push: test of pre-release mster for ChatSecure/ChatSecure-iOS#770 (comment)

19.02

temporary include post 19.02 notification fix.

max_fsm_queue of 1000 might be too small for production use. the default was 5000-10000 (since 2017.11)

Add missing erlang-inets package

- switch to proper 18.12.1 release
- install latest packages if base-image is not updated frequently
- install and cleanup package 'git' properly
- i don't want inband registration

some packages are being auto-installed and bloat the image even though they're not needed.

buster does not know erlang-ic erlang -corba erlang-percept erlang-typer
buster otp breaks on elixir.

buster is following stretch.

remove inband registration.

pre-official-release 18.12.1-branch
mwlczk pushed a commit to mwlczk/docker-ejabberd that referenced this issue Mar 6, 2021
README with new VARIABLES is gonna follow.

dockerfile cleanup
clean up private changes

21.01

double domain and a/v

finalized 20.01

preparation for 19.08

19.08

remove warnings about deprecated listeners

19.05

don't be to strict

ninja fixes

use 19.02 plus only a few fixes

bug-fix

move http_upload to nginx

hardening

remove not not used code

TLS: server prefered ciphers
avatar: compatibilty with old and new

bump to 18.04,
public OMEMO namespace,
preparation for ldap-roster

Push: test of pre-release mster for ChatSecure/ChatSecure-iOS#770 (comment)

19.02

temporary include post 19.02 notification fix.

max_fsm_queue of 1000 might be too small for production use. the default was 5000-10000 (since 2017.11)

Add missing erlang-inets package

- switch to proper 18.12.1 release
- install latest packages if base-image is not updated frequently
- install and cleanup package 'git' properly
- i don't want inband registration

some packages are being auto-installed and bloat the image even though they're not needed.

buster does not know erlang-ic erlang -corba erlang-percept erlang-typer
buster otp breaks on elixir.

buster is following stretch.

remove inband registration.

pre-official-release 18.12.1-branch
mwlczk pushed a commit to mwlczk/docker-ejabberd that referenced this issue Mar 9, 2021
mwlczk pushed a commit to mwlczk/docker-ejabberd that referenced this issue Mar 9, 2021
README with new VARIABLES is gonna follow.

dockerfile cleanup
clean up private changes

21.01

double domain and a/v

finalized 20.01

preparation for 19.08

19.08

remove warnings about deprecated listeners

19.05

don't be to strict

ninja fixes

use 19.02 plus only a few fixes

bug-fix

move http_upload to nginx

hardening

remove not not used code

TLS: server prefered ciphers
avatar: compatibilty with old and new

bump to 18.04,
public OMEMO namespace,
preparation for ldap-roster

Push: test of pre-release mster for ChatSecure/ChatSecure-iOS#770 (comment)

19.02

temporary include post 19.02 notification fix.

max_fsm_queue of 1000 might be too small for production use. the default was 5000-10000 (since 2017.11)

Add missing erlang-inets package

- switch to proper 18.12.1 release
- install latest packages if base-image is not updated frequently
- install and cleanup package 'git' properly
- i don't want inband registration

some packages are being auto-installed and bloat the image even though they're not needed.

buster does not know erlang-ic erlang -corba erlang-percept erlang-typer
buster otp breaks on elixir.

buster is following stretch.

remove inband registration.

pre-official-release 18.12.1-branch
mwlczk pushed a commit to mwlczk/docker-ejabberd that referenced this issue Mar 10, 2021
Added AV-STUN/TURN Service Settings to README
21.01
remove warnings about deprecated listeners
remove not not used code
bump to 18.04,
public OMEMO namespace,
preparation for ldap-roster

Push: test of pre-release master for ChatSecure/ChatSecure-iOS#770 (comment)

- switch to proper 18.12.1 release
- install latest packages if base-image is not updated frequently
- install and cleanup package 'git' properly
- i don't want inband registration

some packages are being auto-installed and bloat the image even though they're not needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests