Publish to F-droid

[lachmanfrantisek]

Hello,

thank all of you for such a nice app. Would it be possible to publish talk also to the f-droid repository like other nextcloud apps for android?

Thanks in advance!

[mario]

Hopefully in the future, yes - but the issue is you won't be able to receive push notifications unless you have MicroG, and I hear it's relatively hard to install.

[lachmanfrantisek]

Thank you for a quick answer.

I am not sure about the MicroG -- it can be probably installed directly via apk? (Or use play only for this.)

Would be nice to install at least talk via f-droid.

[mario]

MicroG is a set of services that are open source replacements for the Google Play services on "regular" devices.

https://microg.org/

You can download an APK of the Talk app directly until I figure out a proper way to support F-droid.
https://github.com/nextcloud/talk-android/releases

[lachmanfrantisek]

OK, sorry that I messed the MicroG and GlooglePlayServices...

I have a google-play, but would like to use it as little as possible...;-)

Thank you!

[mario]

Understood. Feel free to install the APK directly for now and let me know how you like the app!

[lachmanfrantisek]

Still waiting for the server update and preparing on the phone side..;-)

[ghost]

To publish to F-Droid, there would need to be a seperate FOSS flavor without GCM... And I don't think it's acceptable to be dependent on Google services for a project such as Nextcloud in the long run. Anyway, really cool app, I would love it to be completely independent from proprietary alternatives like Telegram :)

[mario]

@matttter I've done the separation for the Nextcloud Android app, so I know. There's no way to do push notification without Google services, at least not on the server side. On the client side I hear it's possible to use MicroG but I've never tried it.

Maybe I will in the future.

[hex-m]

We had the discussions of GCM-dependency for years until it was fixed in Signal. It's true that you don't have push notifications without gapps/microg but that just means the device has to contact the server regularly and thus use more battery - right?

[mario]

This is not a philosophical, but a technical issue. This is a talk application - once I get a call, I want to get a notification right away, not in 5 or 10 minutes. Now, technically:

• the only way to make sure I can check the server in an interval is alarm manager

• how often would you query the server? 0.5 seconds? 1 second? 5 seconds? 10?

• to make sure such frequent polling works, you would:

  • have to disable battery saving
    • and still, some phones/vendors would stop the process after you do this for a certain number of times
    • the battery would be depleted in half an hour to an hour max

• Signal implements a service which is more than likely to die on quite a few phones, even when you ignore battery optimizations and other things. Sure, you can restart automatically, unless the watchdog services dies too :)

So if you ask me is this possible: definitely. Is it possible in a way where you wouldn't yell at me after a while for killing your phone? Absolutely not.

That being said, there will be an implementation for F-Droid eventually. I just don't know when I'll get to it, but am more than available to help you contribute if you decide to do so! :)

[kertase]

hasn’t push been around since way longer than google play? So it should be possible to use for example an http push protocol instead of google play?

[tobiasKaminsky]

Until you find the time @mario to get push implemented on f-droid in a proper way, why not do it the same as in the nextcloud app:
If detected that push is not supported/working show a snackbar saying that the user will not receive a notification on an incoming call.
But the user can still use the app to call people, to join a conference call.

I think that people that are using a complete google free device are willing to accept some drawbacks.
(At least I am doing ;-))

[ghost]

Yes, if it's not possible or practical to implement a notification service, then just having the app available on F-Droid, even without the notifications, would be great. Like @tobiasKaminsky proposed, a snackbar or a dialog on first launch or something of the sort that tells users what's up would then make sure users who do really need the notifications could resort to the regular app

[ghost]

@mario

There's no way to do push notification without Google services

why is that? what is so exceptional about GCM that it can't be replicated by another party? is there any documentation where i can read up on this? how does it work, shouldn't it just be a server sending notifications to a client and that's it? how does Google prevent the enormous battery drain and what prevents anyone else from doing the same?

[mario]

It's a long story. Like I said - you can use MicroG to receive notifications sent by Google WITHOUT having Google services installed, but it's a bit involved. Alternate ways include things what signals does, or what we'll eventually do - service, IGNORE BATTERY OPTIMIZATIONS, and do http get every now and then (quite often, 10 seconds?).

I'd say GCM is magical in a way it can interact with the system due to having the privilege of being a Google product, so system will never kill, as opposed to even foregrounded services (or even apps that ignore battery optiomizations) which are less likely to get killed, but still do get killed.

You can download an APKs from here if you don't have Play Store:
https://github.com/nextcloud/talk-android/releases

Sure, they have Google Play compile code baked in, but I think it shouldn't crash - the only thing is you won't be receiving notifications.

Like I said before, there will be an F-Droid version, I just don't want to make promises when since I want to get it right, as much is possible in this situation.

[kertase]

Instead of get could you not use use long pooling or maybe webpush api could be implemented in nextcloud and could be used instead ?

[mario]

Since Nextcloud has to be accessible on lots of installations, websockets or such is not an option (and you'd still need a service!). And long polling still requires a service that would do it.

[ghost]

@mario

so the only actual advantages that GCM has over other notification services is that it's very optimized to not drain much battery, and that its service doesn't have to restart because the system will never kill it?

then maybe the best way to do it without requiring multiple different app versions would be:

if GCM/microG is installed on the system then the app uses it

if GCM/microG is not installed but the system is rooted the app asks the user for permission to run the service as root so it won't be killed

if GCM/microG is not installed and the system is not rooted, the app tells the user that the notification experience may not be perfect

to make sure that not every nextcloud suite app needs root maybe the main app could implement this and handle the notification service of the talk app and any other app that needs it, if GCM/microG isn't present.

also the user should be able to set a custom refresh rate for the notifications, so everyone can decide for themselves how much speed vs battery they want.

btw since google plans to switch from GCM to FCM, will any of this change or will it basically stay the same?

[mario]

GCM == FCM mostly. It's the same thing.

[testbird]

An elegant way for nextcloud to avoid depending on google push servers (or whatever current cover name) could be to leverage on the IMAP push. On phones without GCM it is proven to work (https://delta.chat) and it is using even less battery then having GCM installed.

All the NC server would have to do is to send a special email (so they can also be easily filtered), and the NC client could watch the IMAP server for such messages.

(For XMPP there might also exist a working implementation, but almost all users may already have an email account.)

[testbird]

Actually, there is a delta.chat issue open to support initiating video chat.
So maybe get in contact to work out an "email signaling format" that could let delta chat or other MUAs open a NC video chat.
Then you don't even have to implement and run another IMAP watcher on the phone.
deltachat/deltachat-core#101

[testbird]

@mario is there maybe already some manual way to invoke the nextcloud app from another app? For example, can it answer a call if the caller sends a link in an email message, and the user clicks on the link.

[testbird]

Some "email push" message standard brain storming (possibly based on the autocrypt, and email chat standard) at deltachat/deltachat-core#101 (comment)

[testbird]

The paging / notification works very fast with deltachat. So if initiating a call with Nextcloud talk could send an email message with a link, then answering/receiving a call may be made as easy as clicking the link in the message when it arrives in the mobile email messenger (deltachat, conforming to autocrypt and other stanards https://delta.chat/en/standards), and have the messenger app open the link (call) with the nextcloud talk app.

[schemar]

I am assuming you do not want to look at push implementations where you need to set up a different server as everything should be part of the Nextcloud server, right?
Maybe it is time for Nextcloud to create an open PHP project for push notifications ¯_(ツ)_/¯

[mario]

Well ... afaik, you cannot do a websockets as part of PHP, so it would need to be a separate server, sure. And this would obviously increase the barrier to entry as not everyone will be able to host it. It would also need an app to nextcloud to make it work, or a patch to an existing one. So not that easy :)

[testbird]

That's why I proposed email. By now it has wide-spread push functionality, and virtually everybody has email, can get or set up email (even through self-hosting projects).

Establishing an Email-push-notification standard that defines push messages in a way that allows to auto-sort/filter them into a separate folder by sieve server filters and/or the IMAP push subscribed client (i.e. an Email-notify standard similar to the Email-chat standard https://delta.chat/en/spec) would realize an alternative that does not require to set up additional new servers. Push capable client libs that allow easy encryption through back-and-forth handshake are available (e.g. https://github.com/deltachat/deltachat-core), and seem easily expandable to support notifications in addition to chat messages.

[mario]

Once polling mechanism is in place, chances are that you will get call notifications while actively using the app, and not in other cases. On the other hand, chances of getting a notification while in background are rather slim I'm afraid.

This all comes with the additional drawbacks:

• the other end might hang up in 10-15 seconds window on how long it takes to poll so you will miss the call
• the battery will discharge at super fast rates :(

[testbird]

OK, thanks, now I understand the current state better. (Maybe inform about this limitation in to the F-Droid description?)

So, in lack of implementing a free push messaging client https://gitlab.com/foss-push/planning/ , piggybacking setup links into asynchronous email / massaging communication (#96) might be an option that could avoid polling in nextcloud talk, while allowing to receive calls in foreground and background.

[DaCryptor]

@mario
Original article in German: https://www.computerbase.de/2018-03/google-play-store-apps-sperre/
Google-translation: https://translate.google.de/translate?sl=de&tl=en&js=y&prev=_t&hl=de&ie=UTF-8&u=https%3A%2F%2Fwww.computerbase.de%2F2018-03%2Fgoogle-play-store-apps-sperre%2F&edit-text=

Hopefully Google does not cut off the custom-rom-users one day...

[Gaeldrin]

Hello, I would have a question about this as I am using F-Droid without any Google anything (Android 7, LineageOS 14), but with a few extra APKs like WhatsApp and Facebook Messenger. The things is - Messenger seemingly polls the Facebook servers only when opened which is in accordance with what @mario said.

WhatsApp, however, receives all kinds of notifications basically in real time without me opening the app or turning off any battery optimization. Incoming calls received immediately. Battery usage is basically always under 5% on full cycle. After some search for any background info, I found this, maybe will help with ideas for the Talk service:
https://android.stackexchange.com/questions/43970/how-is-whatsapp-able-to-receive-messages-when-not-in-use

PS: I would second testbird's suggestion to mention this limitation in F-Droid's version if possible (unless it gets fixed soon) :)

[mario]

@Gaeldrin there are some apps that are automatically whitelisted by the phone vendors so they can do that, and that includes Whatsup. Unfortunately, we're nowhere near as "famous" as they are yet to be able to request this.

[hex-m]

@mario do you have a reference for this? @Gaeldrin mentioned he is using LineageOS so I don't see how the phone vendor could do that.

[mario]

@hex-m you're right, this is not related to Lineage. This is related to Oppo, Xiaomi, Asus, Huawei and a few others.

[violoncelloCH]

would it be possible to publish pre-releases to f-droid too? so they can be installed manually? I think there are many people using f-droid willing to test pre-releases...

[mario]

@Bubu could you add 1.2.0beta1 to F-Droid as a manual install?

[Bubu]

@mario I can add it tomorrow if that's not too late already?

[mario]

@Bubu it's not. There'll be a beta2 this week though :)

[Bubu]

Added beta1 now. I noticed the version name doesn't reflect the betaX status, it just shows 1.2.0 not sure if that's intentional? (It works this way as well, it just might be confusing for some people).

Just to clarify, you are planning to bump the VersionCode for beta2 and again for release I then?

[mario]

Yes. The version name lacking beta1 was just an omission on my part. Beta2 will have it.

…

On Mon, 14 May 2018 at 09:37, Marcus Hoffmann ***@***.***> wrote: Added beta1 now. I noticed the version name doesn't reflect the betaX status, it just shows 1.2.0 not sure if that's intentional? (It works this way as well, it just might be confusing for some people). Just to clarify, you are planning to bump the VersionCode for beta2 and again for release I then? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#58 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAWsq5tcgMx5pqfQAfaHlKQJUF9greEks5tyTRJgaJpZM4RZfK7> .

[ChildLearningClub]

I’m not a programmer so I don’t know that something like this would even be possible, but would there be a way for the polling frequency to be changed automatically based on if a user on the server opens their app then the polling frequency of each of the other users apps would also increase? Once all users apps go into ideal or are closed the polling frequency can drop back down to conserve battery power? I would guess that the current polling when the app is closed would have to be adjusted to the average time that it would take a user to open their app hit the contacts tab, scroll to the user they wish to call, and then actually call them, for this to actually even work.

[ChildLearningClub]

Or for larger groups it could even be dynamic where the system learns who you frequently call and when you open your app it increases the poll rate of those individuals apps on their devices.

[testbird]

Is someone interested to try exploring a proof-of-concept for push signaling in F-droid?

What kind of intent would a push notification have to emit for incoming nc-talk calls?

The idea would be to patch deltachat-android (IMAP-push based) to emit an intent (e.g. to nc-talk) on messages coming from a specific email contact.
https://github.com/deltachat/deltachat-android/blob/c5701745ebbd031c2aaa536ad47c194cc595c9f3/MessengerProj/src/main/java/com/b44t/messenger/NotificationsController.java#L377

[testbird]

(i.e. for allowing the nextcloud server to wake up the called client device, and signal the call, by sending an email to the user)

[bam80]

@testbird Would IMAP-push based notification require yet another persistent connection to IMAP server?
The idea is fine otherwise

[testbird]

If someone could provide a patch to the NotificationsController of deltachat-android (as in #58 (comment)) to emit an android "intent" to nextcloud-talk, this could be a proof of concept. It would work based on the same IMAP connection (email account) that is configured in deltachat. The nexcloud-talk server would just have to send a recognizable email, and deltachat would wake up the device and pass a message to nextcloud-talk.

The further idea is to convert the message reception and passing into a separate service (app) later (to be used by apps like deltachat, nextcloud-talk etc.). https://gitlab.com/foss-push/planning/wikis/home Even then it should be possible for it to maintain only a single IMAP connection, and passing received email-notifications, email-chat, and classic emails on to the corresponding apps.

[bam80]

@testbird what it's intended to do with these "push" emails - should they be deleted immediately upon receive? AFAIK if they are grouped to separate IMAP folder they would require additional connection anyway..

[testbird]

Deltachat currently only watches the INBOX, moves chat messages into the separate DeltaChat folder, and internally groups incoming emails from "approved contacts" into separate "chats" according to the sender and recipient group, there is no proper auto archive or delete yet (deltachat/deltachat-core#120), however, it should be possible to delete the whole "nextcloud-talk notifications chat" from time to time if desired, as it would only contain wake-up notifications.

For a proof-of-concept and testing how IMAP-push could work, a patch to deltachat NotificationsController would seem like the easiest path.

(An idea could be to allow configuring a combination of sender address AND a subject, to trigger sending an intent containig the message to a configurable app. This would allow using the same account in nextcloud to send such messages to oneself, and even selecting a specific device or similar.)

[testbird]

@bam80 Some servers do allow to watch multiple folders with one connection (imap-notify) https://github.com/deltachat/deltachat-core/wiki/IMAP-strategy

(also added more details to my answer #58 (comment))

[github-k8n]

I think there seems to be some notion that "Push" messages and specifically Google Cloud Messaging are something "special".

If I understood it correctly it should be like this:
Basically it is just the google app keeping a connection open with one of the nearby GCM servers.
That is basically the same thing that IMAP idle does.
Through this open connection, any notification can be "pushed" to the device.
I don't know of any side channel that would notify/wakeup the device, however if anyone has any details on that I'd be eager to know more...
The advantage that the GCM servers might have is that mobile phone proivders will not drop idle connections to those servers (which avoids having to reconnect if the connection is cut after a while..)
The second "advantage" is that you are bundling the notifications, i.e. only a single connection has to be maintained instead of multiple ones.

=> as testbird and others mentioned, IMAP idle gives me push notifications without battery drain.
The only battery drain comes if you do polling instead (i.e. opening new connections every x seconds...)

The ideal open source way would be to:

• Create a notification server that can be deployed anywhere.
• Create a library in android that handles the communication with that server (IP configurable on the phone)
• Create libraries that can be used by servers to use the notification server

(Basically recreating GCM but with a configurable server)

The disadvantage is that this requires support by the server and the apps so probably will not be popular...
-- the other solution would be to just do long-lived open connections in the app itself, but as others mentioned Google is actively trying to break things there by killing apps in the background... so the only way "GCM" is special is because google says so (and does not kill it running in the background)

[arendtio]

@mario I think waking the device when a call comes in is an essential feature for a 'talk' app and Nextcloud as a challenger of the large cloud corporations might want to use some alternative to the Google push services (people who use F-Droid are probably the same who use Nextcloud, and they do it for a reason). So this issue might be a bit more critical than it was qualified.

Nevertheless, I understand that writing a push alternative isn't simple and takes some time. A few years back I read this essay from the Conversations (XMPP chat app) author about push notifications and battery usage which I found very insightful, and as a regular Conversations user (installed through F-Droid) I can tell you that it has a superb energy footprint: https://gultsch.de/xmpp_2016.html

So maybe looking at the Conversations source code could prove to be useful.

[bes1002t]

As Huawei smartphones are shipped without google services, this issue might become important again. I know it is difficult to implement a push service, but I think for the future this will be very important.

Maybe there is already a open source push service alternative? The big disadvantage would be a battery that will be empty very fast if multiple apps are using play services alternatives.

[DJCrashdummy]

@bes1002t there are some projects, but nothing really ready for prime time (yet)... e.g. the 2 i "know" a little bit better are:

• FOSS-Push, which never gained track beside planing resp. brainstorming...
• OpenPush, by @Bubu

[danialbehzadi]

Any plan to support UnifiedPush?

[AndyScherzinger]

Currently no plans but as always contributions are welcome so if anybody makes an effort we are here to help