Push notifications #577
Comments
Strongly against! On the contrary, I would add a line to manifest.xml to request to ignore battery optimization "android.permission.REQUEST_IGNORE_BATTERY_OPTIMIZATIONS" so that all sorts of Chinese shells do not kill the background process. |
|
Well, you can always self-host your notification server. My opinion is there should be at least an option for using the unified push system because as I said some people host their own instances and I don't see any problem there. |
On your part, there is an unreasoned exchange of opinion, a wish. |
|
@2743ab2 not having a framework for push notifications in the FOSS android ecosystem is definitely a long standing pain point. Sure if you only use a single messenger, and it implement its background sync correctly, you probably don't see a problem. But as you add more apps it start to crumble because all these background processes keep constantly waking your phone and all together they do consume quiet some power. Now if you take a look at what unifiedphush is about you will see that it is basically an API, it doesn't mandate how the push system is to be implemented. So it neither lock you with a vendor, nor with a specific push transport technology. I don't think anybody is asking to drop the current background sync implementation and switch to UP only, just to have optional support. |
|
I'll bump this request - I'm trying to de-google and having a bunch of messengers syncing in the background is neither pleasant nor good for battery. I just set up my own ntfy server to handle unifiedpush notifications (as well as other push notifications from my own alerts), and it was very simple to do. |
|
Here is some information on how the matrix client element does it: https://github.com/vector-im/element-android/blob/16ca26569f99df230888f78a8e2529c5e385ddda/docs/unifiedpush.md |
|
Any news? |
|
I am all for this, however, I believe in order to do this properly, you would need something on the server side to actually publish notifications to the client(s). There is a zero percent chance that Telegram themselves will do it, so that leaves someone to write a middle-layer service that would run on a server somewhere, logged into your Telegram account, that will handle pushing notifications to your other client(s) when you get a new message. It's not as easy as setting this up on something like Matrix, where they control the server implementation and have a system in place for setting up arbitrary notification systems already. So the tldr; this would require a server component to be written in addition to client changes, which is probably out-of-scope for this project. |
|
I agree. UnifiedPush has no chance to work on Telegram.
Yet FOSS implementation of FCM would help a lot. There are some in the hood.
|
|
Telegram allows sending push notifications to arbitrary HTTPS endpoints using the web push protocol: https://core.telegram.org/api/push-updates ( I might give a go at writing a web-push=>unifiedpush proxy this weekend, which should solve this issue. |
|
Turns out no proxy is needed, Telegram's implementation is lax enough and doesn't check the return code of the push endpoint, for example pushing to ntfy works OK even if the return code is 200 and no Link header is present. |
|
@2743ab2 Sorry if I sound harsh, but it seems you have not spend a dime looking into https://unifiedpush.org/, have you? This is not a "service", it's an open-source library that can be implemented in any app and it allows the user to decide themselves what push server to pick (self-hosted like ntfy or NextCloud or FCM etc.) by installing the proper push client on his/her device for it. The user could even select an open-source FCM push client (which could also be embedded, since it's not official proprietary stuff), but this makes no sense here, because people could just get the official Telegram app for that, if they really want Google to be involved. Take a look:
The green parts are installed by the user, the app registers itself through his/her push client with the push server (own infrastructure) and passes the received endpoint to the Push Server, afaik. When the App Server wants to push the message, it just sends it to the endpoint, which points to your Push Server (or a proprietary Push Server like FCM, if the user has chosen to). There is just a thing to be mind of: The "App-Server" would be the Telegram server in this case, which won't happen. So, there has to be something in between. Since we don't want to introduce another to-be-trusted dependency that our messages have to go through, the "App Server" needs to be a self-hosted gateway as well, so it really only makes sense to consider for self-hosting. The benefit is huge though. For all apps that implement Unified Push you only have to have one Push Client and its notifications run through your infrastructure, your battery will thank you without relying on any other dependency. |
|
The only things Telegram-FOSS has to do to use unified push are:
|
|
@danog I wasn't talking to you, I was talking to @2743ab2, but it's nice to get some more insight about what actually need to be done on Telegram-FOSS by someone who is even deeper into the specs, I scratched the surface so far. I actually missed your last comment. Wow this is good news, but kinda unsupported. A proxy would make this stable and prevent things from breaking for long if they decide to change something. But it's great to know that this would work without a proxy. |
|
Any update? |
|
Not sure if this is in the Todo list. |
hi, i want to send notification to my own telegram clone with fcm, but i can not recognize what should be the json that i send, can you help me? i can send remote message with data payload from fcm api but telegram need its own data payload |
|
unified push is quite nice, element (f-droid) has the option to use unified push as notification system. they use https://ntfy.sh/ as default server but ntfy can easily be self hosted. |
|
is anyone working on implementing this? or am i free to fork and make a PR? |
|
no doubt, yes, you can
|
I have done some work to push to Telegram (https://github.com/quqkuk/Telegram-webpush) (that could be merged here too), currently it's not the best code (it uses Java's security API instead of using BoringSSL and there's not a lot of error handling), but most of the RFCs are implemented |
|
hi, |
Thank you so much for this, I'm trying it right now! EDIT: I installed the UnifiedPush FCM Distributor (since I'm on microG but here no TG client is delivering notifications except Telegram X) and was able to register TG FOSS, let's see how it goes! EDIT 2: |
I'm testing it and everything is working right now. Do you want to merge it upstream or keep a separate fork? |
Moreover, are you willing to maintain both this fork and Mercurygram? Thanks! |
|
hi all, In the future, I'll only push stuff inside Mercurygram, that will include the features I use, and I'll open PR here with bug fixes for UnifiedPush. |
That's great! Thank you for all this: given my long-standing problems with Telegram notifications, it's exactly what I've been helplessly looking for since ages! |
Hey, great work, you're going to try to have it added to Telegram itself? I didn't see a pull request there yet |
|
I've been monitoring this issue in the last few days, thank you for the attention, but there are bad news I'll get to in a bit.
Yes, that was my goal - Instead of making a patch based upon Telegram-FOSS I wanted to base it on Telegram, hoping that even if it wouldn't get merged, applying it to this codebase would still be feasible without too much hassle.
Are you sure? I checked @drizzt's repo (thank you for the work btw) and it still has the Push Service Notification to keep the connection to Telegram's servers open, which means that notifications might be coming from that connection instead of the UnifiedPush distributor I'm saying this because my patches never worked even for me - At some point I doubted my own RFC8188 message parser and since I had accepted that I would've had to rewrite the whole thing to follow the "guidelines" I had guessed while writing and debugging, I pulled Google's Tink which implements Message Encryption for Web Push (it's the same library that the UnifiedPush example uses) and it too threw an error about the record size field bearing an invalid value, which means that Telegram doesn't seem to return a valid encrypted WebPush payload (I'm ruling out it being a WebPush-through-UnifiedPush problem, as the UnifiedPush example treats the message received from the distributor as an encrypted WebPush payload that follows the RFC8291 spec). |
|
@quqkuk In my tests I disabled push notification in options and I'm using a self-hosted ntfy. |
Well, that's good news! I'll improve the code if I can, as it's clearly something that should not go in production and was there just to test things, but that will let me rest more easily |
|
@quqkuk ugh, you were right, it seems it's not parsing the message correctly (wrong size as you told me) and I'm investigating about that. Hopefully it'll possible to fix your Web Push implementation some days, but in the meanwhile the simple one works |
JFYI Before this change it was perfectly working here, not anymore now. EDIT: it seems that re-enabling the keepalive did the trick but I'll be testing it throughout the day... |
|
I think I know what's going on |
|
@quqkuk I think it'll require a proxy to be put BEFORE nfty (a proxy that directly receive Telegram messages) and that converts it to the real RFC 8291, since headers are stripped by nfty server and not forwarded |
Yes that's what I wrote, or at least what I meant |
Ok, this morning's build was working reliably with the keepalive service enabled; this afternoon's build is working with all disabled, don't know whether it's coincidental or you changed something! 👍 |
|
@quqkuk ok, I created a simple proxy that appends the headers to the payload (using the same method as https://github.com/UnifiedPush/common-proxies/blob/1567aa9ea7fc6aef80162180f4e54208cd8784f9/gateway/webpush.go#L33-L42), so now we only need to change your java function in order to get the keys from the base64url headers and use it by using the old method in draft-ietf-httpbis-encryption-encoding-03. Hint: if you want to install the proxy on your setup, be sure to put the hostname inside /etc/hosts (in my case I have 116.203.117.243 ntfy.belloworld.it) so you don't need to do lots of dns queries |
|
From an user perspective: since right now the simple implementation is working like a charm, won't this new reworked one overcomplicate things? |
|
@gcvl yes, I changed something in the UnifiedPush handler (threads, postInitApplication, etc). if you can confirm it works also after 30+ minutes from the previous notification I'll remove the draft from the PR so they could merge it |
Yes, I currently have it working, I tested with variable timespans, from some minutes to over 30 (screen off) and it got notifications near instantly every single time. I'll tell you my setup though: I'm on Lineage + microG, Doze entirely disabled, Mercurygram unrestricted from battery optimizations with keepalive and TG background connection both disabled, ntfy connected to ntfy.sh and registered to FCM through microG (surprisingly I never ever managed to make the official TG receive notifications through microG, on all my degoogled phones it's the only app that just wouldn't reliably alert me no matter what I tried to do and all its other forks were no better). I could do some tests with Doze fully enabled and battery optimization ON for MG or if you wish to remain on the safe side you can toggle ON the keepalive service as default so that it places just the persistent notification to stay active and running. |
|
Doze ON and battery optimization ON: the app fell asleep and didn't notify. |
https://github.com/drizzt/Mercurygram#notes Thank you for reporting, I hope it's clear enough and it's what you said |
Yes, they're needed and very clear hints! |
if you can try the last apk I pushed some minutes ago. it should not change anything, but who knows 😵💫 |
Sure! Updated, I'm doing some tests and letting you know. |
It's great! It works just like the earlier one (with battery optimizations OFF) 😊🫡 |
|
Ok, I happily switched to Mercurygram from Telegram X, I'm sold 😊 |
|
I'm waiting for a reply on UnifiedPush/common-proxies#64. |
|
I'll preface this by saying that I love Mercurygram and @quqkuk I love your PR. But on the UP side we have previously discussed the merits of sync on push vs a webpush proxy: UnifiedPush/wishlist#15 (comment) We're looking at this from the perspective of simple self hosters, not just UP enthusiasts, i.e. people that might be using their own NextPush or Prosody UP plugins. Sync on Push: they don't need to host common proxies (or a similar gateway doesn't have to be built into all the distributors), and they don't need to use a public gateway (privacy). With a WebPush proxy: Maybe one or two fewer requests to the server? The phone still has to wake up and talk to the server. Additionally, we endorse and support an outdated specification which is (imho) a bad thing. @quqkuk I'm not totally familiar with the Telegram API, are there any advantages here in particular of decryption instead of sync on push? @p1gp1g what do you think? |
|
We had this discussion mainly for Mastodon, which is also self-hostable. In this case this is particularly inefficient to push from one mastodon server to the centralized push-gateway (common-proxy to convert old-webpush->webpush) then to the push provider, which may be on the same server than Mastodon server. But, in the case of Telegram, this is a bit different: its server is centralized. So, if the notif goes throw a centralized server then to the push provider, it is nearly the same than from the centralized server (telegram) to the push provider. You still have to check if this is more efficient than a sync-on-push and you have to consider the accessibility of the gateway : if it goes down no one will receive their notif. Finally, in my opinion: you should check with sync-on-push and the push gateway. If you don't see any difference, then the sync-on-push is probably preferable |
The phone does not have to wake up, per Android's Documentation on Broadcasts, and the application does not have to ping the server (even though Telegram resumes the connection, it does so only after processing the notification).
I agree, that's why I filed their implementation as a bug on the Telegram bug tracker.
That's why I sent in a PR that includes a transparent proxy - the gateway doesn't have to be public and we get a somewhat standard format for WebPush Draft 4 messages that is compatible with UnifiedPush (my PR isn't related to telegram - it just moves headers into the body) and keeps encryption (which would be kept even if the gateway were public). A Transparent Proxy put in front of the UP Provider also means that there's no need to find a way to make the gateway's endpoint discoverable - the URL given by the provider is enough.
I'd argue that sync on push is a good way to notify, but it's not the best - it adds an empty message to the whole update fetching flow and doesn't allow for applications to notify people without first going online (which often requires waiting for the TCP handshake and maybe other handshakes as the protocol stack reaches the application level). Also, sync-on-push is not mutually exclusive to Draft 4 over UP - both can be achieved in multiple ways, even by just using aesgcm format's body to signal updates |
|
Yeah that makes sense. Just looked at Telegram's API and it looks like there is sufficient information to notify in the message itself, so it's possible we don't even need to sync with the server? In that context, decryption seems like a good idea to me as long as the app falls back to sync on push. i.e. If it receives a push message, and its not in the right format, it should sync with the server. That way, if someone doesn't have the gateway, their Telegram UP will still work. And as you said, messages can be redirected to the gateway based on the aesgcm header, so we don't need the complexity of discovery. Overall seems like this gateway is a good idea then. |
|
@quqkuk First, push gateways are something common in mobile development, they are host by the app developer to push to the device push provider. For instance for mastodon: toot-relay, PushToFcm and metatext-apns, for matrix: sygnal and fluffygate. As long as the service server OR push provider is a centralized server, this has no real impact on the architecture for sending notifications with a centralized push gateway. Most of the time this is the case because the push provider is centralized. With telegram, we are in this case too: the service server is centralized. Nevertheless, you should test with sync-on-push and and the gateway, if the results are comparable (latency, battery drain), then the effort of a push gateway is not worth it. (And the time required for a TCP handshake is invisible for a user) When both are decentralized, sync on push is way more efficient. The UnifiedPush discovery response must not include anything related to push gateway. The provider must be application agnostic. Else it will start fragmenting everything. |
and others
What about use unified notification ?https://unifiedpush.org/
The text was updated successfully, but these errors were encountered: