Doze and App Standby whitelist

[daradib]

We may want trigger a dialog to be added to the exemption whitelist for battery optimization on Android 6+ (ACTION_REQUEST_IGNORE_BATTERY_OPTIMIZATIONS). This is an acceptable use case because we can not use GCM. See Optimizing for Doze and App Standby (aka deep sleep).

I did some brief testing with adb. During Doze (device idle), Conversations goes quiet (no logcat messages, no notifications), until the device is unlocked (moving or turning on the screen should work, but didn't seem to be enough for me). Not sure if the maintenance window works. During App Standby (app inactive), Conversations goes quiet until it is brought to the foreground. Presumably the server will eventually notice and destroy the session.

I'm running Android 6.0.1 on a Nexus 6P. I've manually disabled battery optimization of Conversations under Settings -> Battery -> Battery optimization. However, if you don't mind Doze, I think you can enable the foreground service setting to allow Doze but not App Standby.

Thoughts? Do others have similar problems with marshmallow?

[andersruneson]

I'm having problems as well but I'm still not convinced doze is my problem.

The session established time is often set to "just now" when I check that
after opening the app, this would happen after doze but I often have my
phone in the pocket or on charge so it should not be dozing. Oom could also
be my problem.

Is there any logging of when phone was dozing?
On 31 Dec 2015 00:56, "Dara Adib" notifications@github.com wrote:

We may want trigger a dialog to be added to the exemption whitelist for
battery optimization on Android 6+
(ACTION_REQUEST_IGNORE_BATTERY_OPTIMIZATIONS). This is an acceptable use
case because we can not use GCM. See Optimizing for Doze and App Standby
https://developer.android.com/training/monitoring-device-state/doze-standby.html
(aka deep sleep).

I did some brief testing with adb. During Doze (device idle),
Conversations goes quiet (no logcat messages, no notifications), until the
device is unlocked (moving or turning on the screen should work, but didn't
seem to be enough for me). Not sure if the maintenance window works. During
App Standby (app inactive), Conversations goes quiet until it is brought to
the foreground. Presumably the server will eventually notice and destroy
the session.

I'm running Android 6.0.1 on a Nexus 6P. I've manually disabled battery
optimization of Conversations under Settings -> Battery -> Battery
optimization. However, if you don't mind Doze, I think you can enable the
foreground service setting to allow Doze but not App Standby.

Thoughts? Do others have similar problems with marshmallow?

—
Reply to this email directly or view it on GitHub
#1624.

[daradib]

The session established time is often set to "just now" when I check that after opening the app, this would happen after doze but I often have my phone in the pocket or on charge so it should not be dozing.

In my case this was caused by App Standby. Shouldn't happen while charging, but unlike Doze the device can be moving/in use. You can check by looking under Developer options -> Inactive apps (last entry on my phone) before you bring the app to the foreground by opening it. If it says "Inactive", then App Standby is in effect. I disabled battery optimization of Conversations (enabling the foreground service should work as well) and so it is now always "Active".

Is there any logging of when phone was dozing?

In adb logcat I see "JobSchedulerService: Receieved: android.os.action.DEVICE_IDLE_MODE_CHANGED" when presumably entering/exiting Doze states.

Unfortunately, even on the whitelist, it looks like Doze eventually breaks the connection after ~7 hours (depending on ping interval). From adb -d logcat -v time:

01-01 05:36:59.466 D/conversations(18866): <myjid> send ping
01-01 05:44:30.344 D/conversations(18866): <myjid> send ping
01-01 05:52:37.040 D/conversations(18866): <myjid> send ping
01-01 05:59:32.069 D/conversations(18866): <myjid> send ping
01-01 06:07:43.044 D/conversations(18866): <myjid> send ping
01-01 06:14:33.030 D/conversations(18866): <myjid> send ping
01-01 06:22:49.086 D/conversations(18866): <myjid> send ping
01-01 06:29:34.068 D/conversations(18866): <myjid> send ping
01-01 06:37:55.064 D/conversations(18866): <myjid> send ping
01-01 06:44:35.063 D/conversations(18866): <myjid> send ping
01-01 06:53:02.054 D/conversations(18866): <myjid> send ping
01-01 06:53:16.525 D/JobSchedulerService( 5045): Receieved: android.os.action.DEVICE_IDLE_MODE_CHANGED
01-01 07:42:04.101 D/conversations(18866): <myjid> send ping
01-01 07:42:04.188 D/JobSchedulerService( 5045): Receieved: android.os.action.DEVICE_IDLE_MODE_CHANGED
01-01 07:47:06.072 D/conversations(18866): <myjid> send ping
01-01 07:47:06.240 D/JobSchedulerService( 5045): Receieved: android.os.action.DEVICE_IDLE_MODE_CHANGED
01-01 09:46:32.040 D/conversations(18866): <myjid> send ping
01-01 09:46:32.557 D/JobSchedulerService( 5045): Receieved: android.os.action.DEVICE_IDLE_MODE_CHANGED
01-01 09:51:34.005 D/conversations(18866): <myjid> send ping
01-01 09:56:36.041 D/conversations(18866): <myjid> send ping
01-01 09:56:36.203 D/JobSchedulerService( 5045): Receieved: android.os.action.DEVICE_IDLE_MODE_CHANGED
# 12:07:54 Prosody server logs "Client disconnected: closed"
01-01 13:21:59.989 D/conversations(18866): <myjid> send ping
01-01 13:22:00.036 D/JobSchedulerService( 5045): Receieved: android.os.action.DEVICE_IDLE_MODE_CHANGED
01-01 13:22:00.187 D/conversations(18866): <myjid>: Read error: ssl=0xef548ac0: I/O error during system call, Connection reset by peer
01-01 13:22:26.238 D/conversations(18866): <myjid>: connecting
# and we reconnect with a new session...

According to Doze whitelist documentation:

An app that is whitelisted can use the network and hold partial wake locks during Doze and App Standby. However, other restrictions still apply to the whitelisted app, just as they do to other apps. For example, the whitelisted app’s jobs and syncs are deferred, and its regular AlarmManager alarms do not fire.

So it looks like Doze breaks scheduleWakeUpCall in XmppConnectionService because it runs alarmManager.set(AlarmManager.ELAPSED_REALTIME_WAKEUP, timeToWake, alarmIntent), which only fires during the increasingly spaced apart maintenance windows.

We can instead use setAndAllowWhileIdle, but that cannot fire more than once per 15 minutes per app during Doze.

[github-k8n]

Same issue on CM12.1, after turning off the screen, messages are not received and sender may get an error. It seems somehow the issue only occurs on WiFi (or at least it does not occur so quick when on phone network)

At the same time, K-9 Mail does not seem to have any connectivity issues with IMAP Idle messages at all.

Is there a way to make Conversations to behave more like K-9 mail (i.e. somehow ensure that it stays connected but don't show the icon in the notifications?)

[github-k8n]

False alarm :-) it seems there is a connectivity issue as soon as I use IPv6 and K9-Mail seems to always use the IPv4 address.
Disabling IPv6 RA so that the device only gets an IPv4 seems to work so far.

[github-k8n]

In case someone has similar issues to the ones I saw (sorry to paste so much here that does not have to do with the original issue):
http://developer.samsung.com/forum/board/thread/view.do?boardName=General&messageId=239890

Samsung phones seem to block ALL IPv6 traffic as soon as the screen is turned off..

Maybe we could have an option to disable IPv6 connections in conversations? (unfortunately Android does not have an option to disable IPv6 in WiFi)

[andersruneson]

@github-k8n Please open a separate issue or discuss this in the Conversations chat room. It's not appropriate to hijack this issue.
@daradib You are correct about app standby. It's even vorse than Doze in some aspects (it can make Conversations in standby for the whole day).

It's also unfortunate that Conversations does not notice that it's disconnected after returning from app standby, after more half an hour in active Conversations still disconnected so I assume it will won't reconnect until Conversations is opened.

Solutions?

Force foreground service for devices on Android 6

This does not help for doze mode and always keeps an ugly notification visible in the notification list.

Ask for permission disable battery optimisation for devices on Android 6

This works for both doze and app standby.
When to ask for this? Will the user accept this?

Timer wakeup?

From app standby there must be some other way to wake up, like the calendar notifies of new appointments. This won't help for doze mode however.

Implement push XEP-0357

I suppose it's a lot of work, but this might be the final solution anyway?

[iNPUTmice]

Ask for permission disable battery optimisation for devices on Android 6

I'm working on that. Might even make it to 1.9.0

Implement push XEP-0357

yes that is the solution. However in case push is not available (not play store version or old server) we will still have to ask to disable battery optimizations

When to ask for this? Will the user accept this?

Thats the big question. For now I'm displaying a warning in the account details. But users might not go there very often. So we should probably just trigger the dialog once! from the conversations overview.

[quantum1423]

Chiming in here to note that Chinese users cannot use GCM because Google servers are blocked in China. Most phones sold in China don't even have Google Play Services. Using XEP-0357 exclusively would mean that Chinese users cannot receive push notifications; that's especially bad since I've been recommending this app to my Chinese friends due to the strong encryption to protect against the draconian Internet surveillance.

[licaon-kter]

XEP-0357 does not depend on GCM, the server does the push-ing.

[quantum1423]

It still refers to a "push service", which I assume means GCM/APNS etc?

[licaon-kter]

Umm, no, the server does the pushing.

[quantum1423]

Well, through GCM etc...

[licaon-kter]

GCM is an option, but that depends on the server I guess.
A server could in theory include a push service too, so it does not depend on Google/Apple.

[quantum1423]

Of course, but the point of XEP-0357 is to integrate XMPP with things like GCM. Without GCM, App Standby would still break Conversations regardless of what other push provider is cooked up to use with XEP-0357. So I think it's irrelevant if we decide that GCM isn't going to be used anyway.

[strb]

@quantum1423 Well what do you propose then? If people in China (or wherever) can't use "things like GCM" (which I take to mean general-purpose push services), there's literally nothing you can do about that. There is no alternative technical solution. Those users will still always be able to white-list Conversations in their Android Doze settings. They may lose out on some of the battery savings, but otherwise won't be affected, so as far as I'm concerned, this is not really an issue.

If you wish to discuss this further, feel free to join our MUC at conversations@conference.siacs.eu. This is an issue tracker, not a forum for discussion, and it's hard to keep track of progress when technical comments are interspersed with general chatter. Thank you for understanding. :)

[iNPUTmice]

Both push and whitelisting are available now