Collect battery statistics comparing websocket usage to vanilla

[relyt29]

See my comment in #21.

If we have some people with spare phones that can dedicate their phones to the cause, and can really really strictly run the phone under exact same conditions, that would be ideal for testing.

Something like (on a testing phone that you don't do ANYTHING else on during the test):

1. Install Vanilla TextSecure
2. Charge to full overnight, take phone off charger
3. Every hour on the hour alternate between sending a message, recieving a message, send picture, recieve picture
4. at 10 hours, take note of battery drain.
5. Install Websockets
6. repeat steps 2-4

Then get that shit into a spreadsheet and make some pretty graphs and that should be useful. It's important to do this right, because give Moxie a statement like "I haven't noticed bad battery drain" - and that's not helpful because its anecdotal. But if we can present aggregated properly taken stats showing the battery drain is on an order of X% worse, we might have a better chance of being merged.

At some point in a few days I will attach a spreadsheet on google docs for people to add their stats so that we can collect aggregate data across multiple devices, although the data is self-reported which is not optimal because people can lie, but please don't do that

[patcon]

This is a great idea, but let's use dumpsys batterystats org.thoughtcrime.securesms.dev :)

If we really want to be thorough, Facebook has a tool called Augmented Traffic Control for simulating mobile network conditions, which might drastically differ from ideal conditions of your home wifi. I believe they created this tool to test their app's battery performance themselves.

The OWS devs might even appreciate any info sent upstream on setting up the ATC toolchain for this sort of testing, although they may be doing it internally already

[JavaJens]

@patcon Really like the idea of ATC, This way we can even further create an identical scenario for everybody.
I think however that for now regular battery stats like @f41c0r proposed (with your addition of batterystats) should suffice, the rest is extra bonus ;)

[relyt29]

heads up to anyone planning on doing this - on API 19 and less, the command is adb shell dumpsys batteryinfo not adb shell dumpsys batterystats.

ATC looks interesting as a tool for testing the network conditions. Probably I will start by collecting battery stats normally for now (Two sample runs, one connected to wifi, and one only on carrier network) and then maybe later use ATC, maybe if Moxie says its necessary for merge? It doesn't look hard to set up, if you already have something like openWRT sitting on your network

[relyt29]

the battery historian application that makes pretty html graphs of the battery data for you only has a minimum supported SDK of lollipop:

minSupportedSDK = 21 // We only support Lollipop bug reports and above

So I get this error.

I haven't yet gotten a contiguous chunk of time to perform the test under the conditions I want to perform it under, still working on this.

[ddast]

I have tried to do some testing of the battery drain the last couple of days. I used an old LG Optimus 2x with Android 4.4.4 (SlimRom) installed. The battery is old and does not perform well anymore. I guess these are pretty bad preconditions, so I do not claim accuracy for any of the results following.

As you can see I did some tests twice because I did not trust the results. The runtime of each test is only approximately the same but assuming a linear decay we can compare them using the ratio. In each test I sent about 10 messages to the device, some with pictures. All test were done with a 3G connection. For the last test in the list (yes the one with the smallest drain) I forgot to turn off WiFi, but the phone was not connected to a WiFi network.

Websocket (no GApps installed):

1. Drain 10.2%/h (runtime 6h)
2. Drain 11.9%/h (runtime 6.5h, batterystats)

Websocket (GApps installed, using websocket flavor):
3. Drain 8.9%/h (runtime 7h, batterystats)

GCM (GApps installed (obviously), play store version):
4. Drain 2.0%/h (runtime 7.5h)
5. Drain 2.1%/h (runtime 36.3h)

GCM (GApps installed, same apk as for the websocket test 1 and 2):
6. Drain 1.8%/h (runtime 8h, batterystats)

TL;DR: It seems that the total battery drain with websockets on this device is about 10%/h and with GCM about 2%/h. This shows that there is an increased amount of battery drain but without further data I would not use these results to quantify the increased drain.

However, if one would take the values serious the conclusion would be an increased drain of at least a factor 5 which seems quite drastic. It's at least 5 because if p is the amount of drain without TextSecure and we assume that p is the same with and without GApps then the actual factor is (10-p)/(2-p).

[JavaJens]

Thanks for the detailed analysis! I'm sure battery life could be heavily improved if we could lower the timeout for the connection. I think GCM has a timeout of ~15min, whereas we have to ping every 30s.
I asked Moxie if we could increase the timeout, but he said they couldn't support that, referring to the required server resources.

Do you have any idea as to where the big difference between 4 and 5 comes?

[ddast]

The battery drain in 4 and 5 is almost the same. Are you referring to the different runtimes? This is just the runtime of one test run and does not mean that the battery emptied in this time span or something like that.

[JavaJens]

Ah ok. I do not know why, but I was under the impression this was the runtime of the phone. Like from full battery to dead empty.

[geileszeuch]

@JavaJens We cannot have the same timeout as GCM because Google has contracts with providers, which make sure that the connection is not killed after a short amount of time. We obviously don't have those perks.

[JavaJens]

@geileszeuch While this may be true, I'm sure longer connections would be possible if there where unlimited resources on the server side. For example, a dynamic timeout could be used, based on the current connection limitations. But for now we have to live with the drawback of increased battery usage.

[geileszeuch]

@JavaJens I totally agree that there is a lot of improvement possible. Others, like Telegram for example, manage to work without GCM and without any noticeable battery drain.

[relyt29]

I don't have a phone that I can test this with right now or in the forseeable future cause I'm too broke right now to buy a second phone even pay as you go and my primary won't work for this, but it would be nice to get one or two more different test conditions to compare against so we have more data than just one device.

[paride]

I'm trying it out and I'm surprised by how well it works. At the moment I can't do a strict comparison like the one that has been suggested by @f41c0r (and I'm not sure if it is really possible to do a meaningful one), but it is apparent that the websocket implementation does not drain the battery. TextSecure does not even appear in the battery stats.

[zebra-ok]

I am planning to help out with more standardized battery reviews in the next days. But as far is I can tell until now, TextSecure with WS is draining the battery quite a lot. On my Nexus 5 with OmniRom and no GAPPS, the latest build takes around the same power as the display (!) as listed in Settings -> Battery. I had a medium phone usage, mostly checking / replying TS messages and checking news on the browser a few times. Here is a screenshot of the battery statistics: https://imgur.com/iqmkX7u

Btw I think the battery test scenario proposed by @f41c0r is in my opinion quite ambitious concerning required testing time and not very close to realistic usage. I'd rather reduce the testing interval to something like 4-5 hours and send/receive messages every 20-30 min.