New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Websocket support #1000

Closed
JavaJens opened this Issue Mar 4, 2014 · 232 comments

Comments

Projects
None yet
@JavaJens

JavaJens commented Mar 4, 2014

As #127 is a little overloaded and goes beyond the technical, I thought to open up this issue to gather implementation details and coordinate efforts for this.

Support for WebSockets is already present on the server, so the idea is to use WebSockets on Android as well (for now instead of a possible more suited protocol like MQTT)

@geileszeuch mentioned a guide on how to use websockets in native apps: http://www.elabs.se/blog/66-using-websockets-in-native-ios-and-android-apps

However I don't think this would be a good guide as it doesn't talk about Wake/Sleep and connection loss.
This article list several issues one has to deal with in a mobile enviroment: http://dalelane.co.uk/blog/?p=1599
I believe the approach mentioned here http://stackoverflow.com/a/16602970 to be reasonable, if I find the time I will give it a try.

I guess for developing this one has to run an own server, as WS is disabled per default.

So please, if you have a question, concern or are already working on this: comment!

@moxie0 moxie0 self-assigned this Mar 4, 2014

@nameless-

This comment has been minimized.

nameless- commented Mar 4, 2014

Is there any technical reason to use web technology ? why not any XMPP like ?

@JavaJens

This comment has been minimized.

JavaJens commented Mar 4, 2014

There is already support in the Server :)

@nameless-

This comment has been minimized.

nameless- commented Mar 4, 2014

As you said, a mobil environment is likely to break connections and thus we will be force to reimplement what already exists in session protocol like XMPP or SIP that is registration, session management, etc.

I think it would be easier to develop something on top of a session protocol even though it will require some slight modification on the server side.

@JavaJens

This comment has been minimized.

JavaJens commented Mar 4, 2014

Do you have anything particular in mind? I haven't checked the code, but
maybe the GCM server part is using XMPP already which could be reused?
I know @v-0-d mentioned MQTT in the other thread, which sounds pretty good
as well.

Nevertheless, I think the current server can handle session mobility to
some extend, but I haven't looked very thoroughly.
From what I see the client simply has to connect (and reconnect if needed)
to the server and listen to incoming messages which are queued until
delivered. And as a client as a clear identification number the mapping to
a WebSocket connection shouldn't be such a big deal.

@JavaJens

This comment has been minimized.

JavaJens commented Mar 5, 2014

I've created a local branch with very rudimentary support, more a PoC: https://github.com/JavaJens/TextSecure/tree/feature-1000
Maybe @thors you can take this as a starting point?
@TheBlueMatt is working on the desktop client afaik. Maybe he can provide some insight to the registration for WebSocket devices?

@geileszeuch

This comment has been minimized.

geileszeuch commented Mar 12, 2014

He recently open sourced bis early version of the textsecure browser extension. Hopefully this will provide some insight: https://github.com/TheBlueMatt/textsecure-chrome

@JavaJens

This comment has been minimized.

JavaJens commented Mar 12, 2014

Yes, I looked into that. right now multiple devices can be registered on one number.

For the Android version i've got to make little changes therefore.

Right now the device connects and all seems well, I only have to test sending/receiving messages which doesn't work right now.

Probably because I only have one device and try to work with the emulator and a modified server version.

If you want you can checkout my feature branch and test yourself :)

cheers

@mank319

This comment has been minimized.

mank319 commented Mar 18, 2014

Thank you for your efforts! I will checkout your branch as soon as my current exams are over and maybe even contribute something.

I hope that we will soon be able to (optionally) use TextSecure without GCM (and PlayStore in general!).

@SecUpwN

This comment has been minimized.

SecUpwN commented Apr 1, 2014

@JavaJens, thank you for opening up this issue, I can't wait for this to be implemented! ;-)

@JavaJens

This comment has been minimized.

JavaJens commented Apr 1, 2014

Feel free to test by branch and contribute. Sadly I'm pretty busy right
now, but rudimentary support exists :)

@SecUpwN

This comment has been minimized.

SecUpwN commented Apr 1, 2014

@JavaJens, is there a compiled WIP-version somewhere or do I have to compile it by myself?

@JavaJens

This comment has been minimized.

JavaJens commented Apr 1, 2014

Sadly, no!

@deutrino

This comment has been minimized.

deutrino commented Apr 29, 2014

Just wanted to comment after reading https://missingm.co/2014/02/fighting-dishfire-the-state-of-mobile-cross-platform-encrypted-messaging/ that XMPP and WebSockets will probably work, sort of, but will be painful on mobile devices due to frequent IP switching, sleep/wake, etc.

Unfortunately this is not (yet) a library that anyone can use, or tunnel over like ssh, but have a look at http://mosh.mit.edu/ - it's a partial ssh replacement (terminal emulator only, no tunneling or forwarding yet) which has solved ssh's problems with IP switching and sleep/wake, while managing to get around many many firewalls and NATs. Their code itself probably can't be used, but the ideas may be worth reading over.

@seroma

This comment has been minimized.

seroma commented May 13, 2014

@JavaJens Great Work you are doing!!! Me and quiet a couple of friends are longing desperately to get their hands on a GCM free TextSecure version. I am so glad to see that you are making it happen. This means really the world to us, since installing Google Play is absolutely no option for us. We are all really excited and following your efforts day to day. Is there anything we can do to help and speed up development other than coding (since we don't have any coding experience)? Since the day TextSecure with data came out (WhisperPush for CM) we all are dreaming of the day we can enjoy this wonderful app with all our friends. Thank you already for all you have done. You are a hero :).

👍 for Websocket support for TextSecure.

@SecUpwN

This comment has been minimized.

SecUpwN commented May 13, 2014

@seroma, I second that. Awesome work, @JavaJens! Keep rockin'! 😸

@Wikinaut

This comment has been minimized.

Contributor

Wikinaut commented May 13, 2014

Just in addition https://devcenter.heroku.com/articles/websocket-security

You should strongly prefer the secure wss:// protocol over the insecure ws:// transport. Like HTTPS, WSS (WebSockets over SSL/TLS) is encrypted, thus protecting against Man-in-the-Middle attacks. A variety of attacks against WebSockets become impossible if the transport is secured.

@JavaJens

This comment has been minimized.

JavaJens commented Jun 12, 2014

I've updated my branch. The most recent updates feature a different package name for parallel installation, increased debug output for the Wakelocking and fixed an error for a minor issue with a Wakelock not being released.

However one of the main issues, severe battery drain, I could not find a reason for; my phone does not go into deep-sleep mode when WiFi is enabled. This has nothing to do with TextSecure though, Wakelocks are held for about 10s in 2h.
As I don't have 3G I can't test the battery consumption without Wifi.
I would suspect this being an issue with my CM11.

The last issue for the high battery drain is the low timeout frequency of @TheBlueMatt's server of about 50s; that is we have to send a ping every ~50s, which is far to often. In real life, this could be decreased to 30min or so.

One last thing I want to address, @Wikinaut: this build is purely for testing and uses the non secure WS for developing. I've updated the Readme to point this out more clearly. In the end certificate pinning will of course be used!

Please report any issues back here or as an issue in my fork.
I don't know if you can use the "Report a Bug" feature or have to use ADB....

@moxie0

This comment has been minimized.

Member

moxie0 commented Jun 12, 2014

The last issue for the high battery drain is the low timeout frequency of @TheBlueMatt's server of about 50s; that is we have to send a ping every ~50s, which is far to often. In real life, this could be decreased to 30min or so.

With RedPhone, we've found that mobile data networks often require you to send keepalives as little as every 15 seconds. So 50s is potentially too long.

@JavaJens

This comment has been minimized.

JavaJens commented Jun 12, 2014

Ok...that comes as a surprise....
Current GCM timeout is apparently ~15min on Wifi, which is stated here as being to high: https://productforums.google.com/forum/#!msg/nexus/fslYqYrULto/lU2D3Qe1mugJ

That would support your findings!
How about the Battery life with RedPhone? As I said my phone doesn't deep sleep with Wifi on...

@moxie0

This comment has been minimized.

Member

moxie0 commented Jun 12, 2014

WiFi is unpredictable, since it depends on the NAT. So you'll have to deal with that the same way GCM does. The advantage GCM has is that Google has made agreements with mobile carriers not to timeout connections on those networks. Without those, websocket connection idle timeouts on mobile data connections will be unpredictable, and they tend to be more trigger happy (and also often just silently close without sending an RST). I think some kind of adaptive solution is probably going to be necessary.

@geileszeuch

This comment has been minimized.

geileszeuch commented Jun 12, 2014

As I don't have 3G I can't test the battery consumption without Wifi.
I would suspect this being an issue with my CM11.

I am building your branch right now. I will test on WiFi and 3G and report as soon as possible (probably in around 10 hours). I am on CM10.2 so let's hope the battery drain really is just a problem of CM11.

@Selaron

This comment has been minimized.

Selaron commented Dec 4, 2014

@JavaJens (changed my github Name)
No its different: As soon I activate push message in settings and fill in my mobile number, then hit [register] I see the message reading that Google play is missing.
My only choice is "Get Google Play" but nothing happens when choosing that. I can then only skip registration. Should I paste logcat or submit debug log?

@marciomr

This comment has been minimized.

marciomr commented Dec 6, 2014

I have got the same message here.

@jensMF

This comment has been minimized.

jensMF commented Jan 2, 2015

Is there any progress in websocket implementation since last post / last activity at javajens fork (27 days ago)?
Is it possible to build TextSecure with websocket and communicate with people that have the official gcm build?
If not, is it possible to unregister from TextSecure push with websocket? I tried to unregister, but my friends with official gcm build automatically send me push messages and I can't receive thus...

@JavaJens

This comment has been minimized.

JavaJens commented Jan 3, 2015

Sorry, there has been so much silence. I didn't have the time to further work on this.
@Selaron and @marciomr Please open up an issue on my fork, if possible with logcat.

@jensMF As far as I know, it is possible to communicate with the official GCM client, but you are not supposed to run this code against the production server, staging only. At least that is what I read from moxies comment some while back.
Regarding unregistering...make sure that you are running on the same server (production vs. staging) and if that still not works, try to get a debug log and open up an issue on my fork.
To me it wasn't exactly clear how unregistering works, as the Wiki Page only specifies unregistering of GCM or APN codes.

@patcon

This comment has been minimized.

patcon commented Jan 3, 2015

@jensMF I'd say just install the official app, re-register, and then unregister for push notifications using the standard advice:
#845 (comment)

@jensMF

This comment has been minimized.

jensMF commented Jan 5, 2015

@JavaJens Thanks for your answer, but how can I check the server where I'm running textsecure on? And how can I change it? (I tried to unregister with websocket version 2.2.0 because I registered with that one, that should run on the same server cause I used the same apk) For the last try i used the latest version of your fork (I checked out the repository, installed everything needed via android-studio and did ./gradlew build). I can send masseges, but don't receive messages…

@patcon I don't have gapps and I don't want to install them…

@ddast

This comment has been minimized.

ddast commented Jan 5, 2015

@jensMF You can unregister from TextSecure on this website: https://whispersystems.org/textsecure/unregister/

@JavaJens

This comment has been minimized.

JavaJens commented Jan 5, 2015

@jensMF If you haven't changed anything then you are using the Staging (aka correct) server.
However I never touched the version number, hence I would look at the commit checksums for verification.

You can check the server in the Release.java file. If you need more help, open a ticket on my fork or send me a mail.

@jensMF

This comment has been minimized.

jensMF commented Jan 6, 2015

Thanks @ddast, that worked well.

@JavaJens is it possible communicate whith playstore-version when being on the staging server?

@JavaJens

This comment has been minimized.

JavaJens commented Jan 6, 2015

@jensMF As fas as I know, the two servers are not federated. That is if you
register against the staging server, people on the other server don't know
that you use TS and can't send you messages.

In order to communicate with them, you would need to change the
Release.java file, but shouldn't be using development code against a
production system.

@ThomasWaldmann

This comment has been minimized.

ThomasWaldmann commented Feb 2, 2015

+1 for textsecure without google services. can't use it currently due to that.

@jensMF

This comment has been minimized.

jensMF commented Feb 14, 2015

Thanks for all the information, it helped me out very well. Are there any news about websocket or plans getting away from gcm? Maybe @moxie0 could say something about it? Or does @JavaJens knows something?
Thanks in advance.

@h-2

This comment has been minimized.

h-2 commented Feb 16, 2015

I recently tried the last version in @grote 's repository. It still drains a lot of battery. If there is any testing and reporting that I could do to help, please let me know. All Jabber-Clients I know, Kontalk, Telegram ... all have some mechanism that has only a slight influence on the battery life...

@JavaJens

This comment has been minimized.

JavaJens commented Feb 16, 2015

I don't know which version you were using, but I do know that the server has a connection timeout of 15s. This, in my opinion, is the only reason why there is such an enormous battery drain. Compare it to GCM which is said to have a timeout of 15m [1](however I don't know if this is outdated info). The server side of things will most likely not be updated, due to limited server resources: https://github.com/WhisperSystems/TextSecure-Server/issues/27.

About the progress of integrating WebSocket in TextSecure; I know that my initial PR was not very good and required lots of refactoring, furthermore there were some changes planned that would break the communication again. Now I've lost most interest in it as I don't know if/when it will break again and if it would ever be accepted. I don't know how the crew at Signal deals with battery issues, maybe they have a completely different way (like only establish the connection on APN or if the app is focus), but it might be worth to check that out.

So again, feel free to pick up my work, I'd be more that happy to assist :)

[1] https://productforums.google.com/forum/#!msg/nexus/fslYqYrULto/lU2D3Qe1mugJ

@JavaJens

This comment has been minimized.

JavaJens commented Feb 18, 2015

If I see it right there is support for WebSocket on the roadmap for milestone 2.6 https://github.com/WhisperSystems/TextSecure/milestones/v2.6.0

See PR #2423

@schiessle

This comment has been minimized.

schiessle commented Feb 18, 2015

This sounds awesome! Is there any estimation when 2.6 will be released?

@jeremymasters

This comment has been minimized.

jeremymasters commented Feb 18, 2015

Sometime after 2.5.3, sometime before 2.6.1. :)

@moxie0

This comment has been minimized.

Member

moxie0 commented Feb 18, 2015

That is not "websocket support" in the sense you want, which is still a ways off.

@devurandom

This comment has been minimized.

devurandom commented Feb 19, 2015

@moxie: What is the purpose of #2423? I am unable to make it out from the diffs.

@JavaJens

This comment has been minimized.

JavaJens commented Feb 19, 2015

I believe it is to receive messages but triggered by gcm. This would prevent messages from being lost.

Haven't checked thoroughly though.

@h-2

This comment has been minimized.

h-2 commented Feb 19, 2015

So again, feel free to pick up my work, I'd be more that happy to assist :)

Thanks for your work thus far, JavaJens. But to be honest, judging from this thread and other conversations, I feel like the overall climate here is not very friendy towards suggestions or even pull requests, so for now I will spend what free time I have on other Free Software projects like Kontalk [no offence to anyone here, its of course their choice where to put priorities].

@jensschulz

This comment has been minimized.

jensschulz commented Feb 25, 2015

Would it be a possible alternative to implement polling (e.g. every 15 minutes) until websocket support is available? I use this on Threema and it's almost as good as push and there is no heavy battery drainage. The load on the server shouldn't be too heavy as only few people are not using GCM. I really would like to switch to TextSecure, but as many others won't install Google Services for it.

edit: Issue has just been closed (#2530)

@ThomasWaldmann

This comment has been minimized.

ThomasWaldmann commented Feb 25, 2015

@jensschulz maybe open a separate ticket for that and link to it from here? would also like such a feature!

@signalapp signalapp locked and limited conversation to collaborators Mar 4, 2015

nrizzio referenced this issue Feb 20, 2017

Support for using Signal without Play Services
This is now possible with beta calling, so non-GCM users are a
part of beta calling by default.

// FREEBIE
@moxie0

This comment has been minimized.

Member

moxie0 commented Feb 20, 2017

in 3.30.0

@moxie0 moxie0 closed this Feb 20, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.