Feature Request: XEP-0313 (Message Archive Management)

[jfrederickson]

It would be nice to have server-side history support. It would of course not work for OTR chats, but for plaintext it would be pretty useful, and maybe PGP (haven't tried it, personally).

[ildar]

Whenever this implemented, it might be further improved with GTalk flavor of server-side archive.
I guess some code may be borrowed from Xabber project.

[fisle]

+1, would be great addition.

[devurandom]

Would be a great addition, as not all clients support XEP-0198, yet, and my phone is not always online, either. So with XEP-0313 support, at least I could see on my phone, that I received some messages while offline.

[pulser]

I saw mention in #574 about MAM potentially being in 0.9. I had been considering this for a while, and a potential way that this could be used to "save power" when using Conversations.

Currently, if you have an ongoing and "busy" conversation on your PC, your phone ends up staying awake a lot of the time (receiving carbons of messages etc). This is obviously by-design and working as intended, but my anecdotal tests are finding that the phone is using a measurable amount of power as a result of the regular packets arriving at the device, due to the conversation that is ongoing.

I set about trying to find a solution, and wonder what your thoughts on this would be - it's likely some kind of "low power usage" mode, rather than a default way, but I am no expert on that. Obviously this wouldn't play nice with OTR - I'm sure this could be triggered to only occur when OTR was not in use, or carbon messages were what was being received.

1. Conversations would support XEP-0313 (MAM)
2. When device receives a message, a notification is fired as normal. At this point, Conversations would stop listening for further messages, and would allow the device to sleep fully at this point - the user now knows there's a message waiting, and a notification was triggered.
3. When the user wakes the device, XEP-0313 is used to "update" the current chat transcript against the server-side copy. This means the user receives any and all "missed" messages while the device was fully asleep and not connected. When there is no current notification of an unread message, the device would listen over XMPP as normal (which should be fine, given the radio should be in a low power state etc at this point).

As normal, if connection was lost for longer than stream management would be able to recover, MAM would retrieve any missed messages.

I'm just wondering if a clever implementation of MAM could help make Conversations a really power-efficient way of working, given the notification LED is typically fairly "boolean" - you either have some messages needing your attention, or you don't. By not listening for further messages, when one has already been received, the radio could likely spend a much greater time "asleep" without regular messages bouncing back and forth - right now I find the biggest single drain on my device in a day is the "extra" network awake time from heavy chat sessions being carried out on a PC, and if I haven't missed something super-obvious here, this might help it a fair bit.

Thoughts appreciated.

[jkufner]

The tricky part is to recognize messages read on another device.

When notified message is read on another device, the notification should not be displayed anymore. But in deep sleep Conversations cannot receive that 'read' information. So it will display notification for something that should not be notified, but when a new message arrives later, it will not notify user about it.

I think this will lead to semi-permanent notification with low power consumption.

[devurandom]

I think @jkufner hints at the right thing. We'll have to think of a few scenarios:

• Message from Alice triggers the behaviour you describe, but a message from Charly arrives afterwards
• Some message triggers the behaviour you describe, I take a quick look at the display, don't bother looking at it, but an hour later another message arrives

Maybe CSI can be used to implement a rate limit for incomming messages instead? However, I really like the idea of using MAM (or any XEP in general) for other than the obvious purposes. :)

[iNPUTmice]

mam is definitely not the right tool to improve on battery consumption. that's what csi is for. csi is able to at least filter unimportant messages.
furthermore I'm not entirely convinced that receiving messages actually consumes that much battery. I would like to see a study with a sleeping phone over night with conversations enabled and receiving messages let's say every minute or so and a sleeping phone with conversations enabled and not receiving messages.

[jkufner]

Huge difference is in whether the phone is in deep sleep or not. Everything else is minor. If phone is not allowed to enter deep sleep, it will consume about 30-60% of battery over night (by doing nothing; depends on model).

[maufl]

I'm willing to pay money for this. Is there a way to crowd found a feature?

[jkufner]

It is also important to have server support. As far as I know, not many servers (if any) support XEP-0313 out of the box. I'm affraid a much more work is required to reach working state.

[ildar]

... and FYI GMail XMPP has its own protocol for this functionality.

[iNPUTmice]

@maufl donations are always welcome but currently there is no special crowd founding program in place.

Either way the work on MAM has begun in special feature branch.

[SebastianS90]

I thought about @pulser's idea and think MAM can be used to save some battery power, but in a slightly different way. CSI strongly depends on the server's behavior and will take some time to be implemented by all the servers out there, therefore an approach with MAM could still be useful.
If I understood things right, a nightmare for an on battery phone is the user chatting a lot on a different device with tons of carbons arriving at our phone. To prevent this, my idea is as follows:

• When the screen is on, we stay connected and operate as we do now. The phone is in use anyway and cannot enter deep sleep.
• When the screen turns off, we still stay connected but start counting messages. If 3 or more carbon messages sent by another device (own jid) arrive within 5 minutes, we assume the user has an active conversation on that device. We wait for the next message from another jid (within the 5min timeframe), insert the notification for this message and cut the connection. This notification will stay visible and we will not receive further messages.
  • If the screen is turned on, we will re-establish the connection and fetch archived messages. If the last message in a conversation is from our own jid, we delete the notification. If it is from the partner's jid, we update the notification to show all unread messages (= messages after the last archived message sent by our jid). In either situation the connection will stay active even if the screen turns off - but that will start counting messages again to detect a conversation held from another device.
  • After a reasonable amount of time (30 minutes?) where the screen was off we wake the phone, establish the connection and fetch the messages. After this we leave the connection active and are again counting own carbon messages to see if the conversation still continues.

To summarize the main pros of this approach:

• This approach will wake the phone only twice a hour even if we write thousands of messages from another device.
• We do not lose notifications: The LED is flashing since the connection is cut only after receiving a message
• If we check the phone display it synchronizes with the server
• If we do not use another device we will not cut the connection and are notified (vibrate) for each message that we receive
• It works without serverside CSI support.

The only downside I've found:

• Use your PC to chat there for quite some time, so the phone cuts the connection. Go away from the PC (maybe log off there). Somebody sends us a message. Unfortunately, the phone will not vibrate since it is not connected at this time. The LED is flashing, since there is an older notification.
  • If the user turns the screen on he will notice the new notification
  • If the user waits for the 30 minutes to end, the phone will connect and notify him

I think CSI is great for filtering presence stanzas. For filtering out the carbons we should have a server-independent solution as the XEP for CSI clearly states that the client must not make any assumptions about how the server will use the active/inactive state information. Implementing MAM in Conversations would be a good point to think about such a solution.

[jkufner]

@SebastianS90 This will make status icon blink in other clients. The check for messages should be done without sending presence stansa.

Another way is to cut the connection always instantly, and check every 20 minutes (or so) only via MAM and using only partial initialization -- no presence sent, no roster fetched, only short look into message archive and then back to sleep.

One way or another, user should have option to turn off automatic waking completely. Then Conversations would check messages only when phone is awaken by user.

Another problem is, that when phone is awaken a lots of other intents are fired and lots of other apps will do some useless stuff. So it will kill the battery quickly anyway.

[iNPUTmice]

before we do any kind of crazy battery optimizations we need some test to actually quantify the problem. I'm thinking a tool that can send messages to conversations in some configurable way. for example send x messages every y seconds. rest for a longer period. repeat. and compare that to an idle conversations, and an idle phone without conversations. also this test should be performed for wifi and 3g.
without such a benchmark there won't be any further optimizations because right now I'm kinda doubting that there even is a problem.

[tallero]

Is there a way to try mam branch? If someone need a server with 313 support, I strongly advise http://Jappix.com, use metronome and jappix itself. The Web client is really nice and supports mam.

[SamWhited]

Mam has been merged into the development branch now.