Skip to content
This repository has been archived by the owner on Jun 12, 2018. It is now read-only.

Websocket handshake error #170

Closed
nicofrand opened this issue Sep 2, 2015 · 19 comments
Closed

Websocket handshake error #170

nicofrand opened this issue Sep 2, 2015 · 19 comments
Labels

Comments

@nicofrand
Copy link

I regularly meet this error in my browser console after refreshing the page:

WebSocket connection to 'wss://***/public/files/socket.io/?EIO=3&transport=websocket&sid=DGvqewDGc_GvZX-3AAAB' failed: Error during WebSocket handshake: Unexpected response code: 400

This does not seem to impact the applications though.

Here are my proxy logs after refreshing the page for the photos app:

[Wed, 02 Sep 2015 08:43:04 GMT] - GET / - 200 - 10370.167 ms -  13924
[Wed, 02 Sep 2015 08:43:05 GMT] - GET /app-icons/sprite-svg-data.css - 200 - 10.526 ms -  61300
[Wed, 02 Sep 2015 08:43:05 GMT] - GET /stylesheets/app.css - 200 - 12.869 ms -  70168
[Wed, 02 Sep 2015 08:43:05 GMT] - GET /javascripts/vendor.js - 200 - 26.633 ms -  1173617
[Wed, 02 Sep 2015 08:43:05 GMT] - GET /javascripts/app.js - 200 - 13.611 ms -  377741
[Wed, 02 Sep 2015 08:43:05 GMT] - GET /stylesheets/app.css.map - 200 - 8.901 ms -  90231
[Wed, 02 Sep 2015 08:43:05 GMT] - GET /socket.io/?EIO=3&transport=polling&t=1441183381194-0 - 200 - 6.624 ms -  101
[Wed, 02 Sep 2015 08:43:05 GMT] - GET /api/notifications - 200 - 31.676 ms -  2
[Wed, 02 Sep 2015 08:43:05 GMT] - GET /api/applications/stack - 200 - 40.202 ms -  890
[Wed, 02 Sep 2015 08:43:05 GMT] - GET /api/backgrounds - 200 - 52.809 ms -  2
[Wed, 02 Sep 2015 08:43:05 GMT] - GET /img/spinner.svg - 200 - 16.604 ms -  410
[Wed, 02 Sep 2015 08:43:05 GMT] - GET /api/sys-data - 200 - 164.330 ms -  162
[Wed, 02 Sep 2015 08:43:05 GMT] - GET /socket.io/?EIO=3&transport=polling&t=1441183381435-1&sid=XkCDEhqwAFi6XI0mAAA_ - 200 - 7.316 ms -  5
[Wed, 02 Sep 2015 08:43:06 GMT] - GET /javascripts/vendor.js.map - 200 - 68.510 ms -  1427297
[Wed, 02 Sep 2015 08:43:06 GMT] - GET /img/backgrounds/background_05.jpg - 200 - 84.928 ms -  281774
[Wed, 02 Sep 2015 08:43:06 GMT] - GET /socket.io/?EIO=3&transport=websocket&sid=XkCDEhqwAFi6XI0mAAA_ - 400 - 19.077 ms -  -
[Wed, 02 Sep 2015 08:43:06 GMT] - POST /api/instance - 200 - 140.436 ms -  42
[Wed, 02 Sep 2015 08:43:06 GMT] - GET /img/happycloud.png - 200 - 8.362 ms -  4208
[Wed, 02 Sep 2015 08:43:06 GMT] - GET /img/notification-orange.png - 200 - 10.870 ms -  642
[Wed, 02 Sep 2015 08:43:06 GMT] - GET /img/apps/store.svg - 200 - 8.899 ms -  886
[Wed, 02 Sep 2015 08:43:06 GMT] - GET /img/apps/state.svg - 200 - 8.225 ms -  3379
[Wed, 02 Sep 2015 08:43:06 GMT] - GET /img/apps/settings.svg - 200 - 8.337 ms -  6460
[Wed, 02 Sep 2015 08:43:06 GMT] - GET /img/apps/help.svg - 200 - 8.195 ms -  3246
[Wed, 02 Sep 2015 08:43:06 GMT] - GET /api/applications/calendar.svg - 200 - 95.957 ms -  -
[Wed, 02 Sep 2015 08:43:06 GMT] - GET /api/applications/contacts.svg - 200 - 165.714 ms -  -
[Wed, 02 Sep 2015 08:43:06 GMT] - GET /api/applications/databrowser.svg - 200 - 256.930 ms -  -
[Wed, 02 Sep 2015 08:43:06 GMT] - GET /api/applications/import-from-google.svg - 200 - 125.533 ms -  -
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /api/applications/konnectors.svg - 200 - 124.689 ms -  -
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /api/applications/files.svg - 200 - 177.119 ms -  -
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /api/applications/kresus.svg - 200 - 232.457 ms -  -
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /api/applications/photos.svg - 200 - 238.089 ms -  -
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /api/applications/quickmarks.svg - 200 - 277.672 ms -  -
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /api/applications/sync.svg - 200 - 217.297 ms -  -
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /api/applications/tasky.svg - 200 - 212.535 ms -  -
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /api/applications/term.svg - 200 - 276.376 ms -  -
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /img/hard-drive.png - 200 - 246.101 ms -  964
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /img/ram.png - 200 - 202.088 ms -  1335
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /img/contribute.jpg - 200 - 117.314 ms -  366143
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /img/logo-brand.png - 200 - 108.462 ms -  15528
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /img/git.png - 200 - 70.173 ms -  891
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /img/link.png - 200 - 70.542 ms -  982
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /img/backgrounds/background_none_th.png - 200 - 53.548 ms -  320
[Wed, 02 Sep 2015 08:43:07 GMT] - GET /img/backgrounds/background_01_th.png - 200 - 54.978 ms -  32936
[Wed, 02 Sep 2015 08:43:08 GMT] - GET /img/backgrounds/background_02_th.png - 200 - 95.707 ms -  55632
[Wed, 02 Sep 2015 08:43:08 GMT] - GET /img/backgrounds/background_03_th.png - 200 - 104.191 ms -  88211
[Wed, 02 Sep 2015 08:43:08 GMT] - GET /img/backgrounds/background_04_th.png - 200 - 106.338 ms -  30670
[Wed, 02 Sep 2015 08:43:08 GMT] - GET /img/backgrounds/background_05_th.png - 200 - 139.809 ms -  43401
[Wed, 02 Sep 2015 08:43:08 GMT] - GET /img/backgrounds/background_06_th.png - 200 - 60.563 ms -  23720
[Wed, 02 Sep 2015 08:43:08 GMT] - GET /img/backgrounds/background_07_th.png - 200 - 89.507 ms -  44542
[Wed, 02 Sep 2015 08:43:08 GMT] - GET /img/backgrounds/background_08_th.png - 200 - 78.169 ms -  124337
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/ - 200 - 5149.159 ms -  2692
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/stylesheets/vendor.css - 200 - 44.707 ms -  107231
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/stylesheets/app.css - 200 - 50.919 ms -  39930
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/javascripts/vendor.js - 200 - 60.441 ms -  752418
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/javascripts/app.js - 200 - 48.447 ms -  103360
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/stylesheets/app.css.map - 200 - 19.437 ms -  58997
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/stylesheets/vendor.css.map - 200 - 26.107 ms -  110980
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/socket.io/?EIO=3&transport=polling&t=1441183387036-0 - 200 - 24.803 ms -  101
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/img/nophotos.gif - 200 - 18.316 ms -  506
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/socket.io/?EIO=3&transport=polling&t=1441183387093-1&sid=3jaWnmCZ6-JtaULrAAAB - 200 - 19.781 ms -  5
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/photos/thumbs/e589541f64243439cb2ee2d782315075.jpg - 200 - 94.651 ms -  -
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/javascripts/vendor.js.map - 200 - 39.912 ms -  934375
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/javascripts/app.js.map - 200 - 16.958 ms -  149629
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/img/loading.gif - 200 - 10.200 ms -  1492
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/socket.io/?EIO=3&transport=websocket&sid=3jaWnmCZ6-JtaULrAAAB - 400 - 8.779 ms -  -
[Wed, 02 Sep 2015 08:43:11 GMT] - GET /apps/photos/clearance/contacts - 200 - 496.590 ms -  70432
@clochix
Copy link
Contributor

clochix commented Sep 3, 2015

Thanks for reporting @nicofrand
This seems to be a known problem with socket.io (cf socketio/socket.io#1942 for example). So it's probably not related to the proxy, but to the version of socket.io we use in every app.
Anyway, we need to investigate.

@nicofrand
Copy link
Author

I am not sure it is socket.io related… Maybe that comes from Apache ?
Shouldn't something like http://httpd.apache.org/docs/2.4/mod/mod_proxy_wstunnel.html be configured ?

Is there anyone using nginx that can reproduce the error ? Does cozy configure nginx like this socketio/socket.io#1942 (comment) ?

@nicofrand
Copy link
Author

@thierrymarianne you posted a solution here, did you remove it ? Why ?

The solution was to enable a2enmod proxy_wstunnel module:

a2enmod proxy_wstunnel

And then add this into the apache VirtualHost configuration before the ProxyPass line:

RewriteEngine           On  
RewriteCond             %{REQUEST_URI}  ^/socket.io            [NC]
RewriteCond             %{QUERY_STRING} transport=websocket    [NC]
RewriteRule             /(.*)           ws://localhost:9104/$1 [P,L]

It seems to work for me. However there are 2 things:

  • Shouldn't it use wss instead of ws ?
  • This solves the issue for cozy but some apps (like term) use socket.io too. The URL for term is then wss:domain.tld/apps/term/socket.io and is not detected by the RewriteCond

If we wanted to make it more global maybe we could use a RewriteCond like this:

RewriteCond             ^(apps\/\w*\/)?socket.io(.*)

Once those questions answered, that might be a good thing to add to cozy-debian apache-conf ? (ping @nledez)

@thierrymarianne
Copy link

@nicofrand It didn't seem to solve the issue entirely. I keep seeing the transport parameter switching back to polling.

@nicofrand
Copy link
Author

I think I solved it, the following rules should cover most of the cases (with socket.io I mean):

RewriteEngine           On
RewriteCond             %{REQUEST_URI} ^/.*socket\.io [NC]
RewriteCond             %{THE_REQUEST} websocket [NC]
RewriteRule             /(.*)           ws://127.0.0.1:9104/$1 [P,L]

You will also need to enable proxy_wstunnel and mod_rewrite modules:

a2enmod proxy_wstunnel mod_rewrite

@nicofrand
Copy link
Author

Hi @thierrymarianne, would you mind testing the solution above please, if you have time ? If that works for you, maybe I could create a ticket on cozy-debian (or try a PR) and update the documentation…

@thierrymarianne
Copy link

@nicofrand Sure thing, I'll let you know ASAP :)

@thierrymarianne
Copy link

@nicofrand Unfortunately, I can not confirm the configuration above works for me. First, I thought it did before noticing Bad request responses which took quite some time to be delivered.

@nicofrand
Copy link
Author

Cool.
I don't get why it would not work for 127.0.0.1 though..

Le mer. 18 nov. 2015 20:52, Thierry Marianne notifications@github.com a
écrit :

@nicofrand https://github.com/nicofrand I confirm the configuration
above works for me after replacing 127.0.0.1 with localhost. Thank you1


Reply to this email directly or view it on GitHub
#170 (comment).

@thierrymarianne
Copy link

@nicofrand

Keeping any of the following sets of rules seems to prevent the Bad request responses from ever occurring for the "main" websocket connection request:
wss://domain/socket.io/?EIO=3&transport=websocket&sid=[...]

RewriteEngine           On  
RewriteCond             %{REQUEST_URI}          ^(.*)/socket.io         [NC]
RewriteCond             %{QUERY_STRING}         transport=websocket     [NC]
RewriteRule             /.*(socket.io/.*)       ws://localhost:9104/$1  [P,L]

or

RewriteCond             %{REQUEST_URI}          ^/.*socket\.io          [NC]
RewriteCond             %{THE_REQUEST}          websocket               [NC]
RewriteRule             (.*)                    ws://localhost:9104/$1  [P,L]

However, it fails with WebSocket opening handshake timed out error for apps like emails:
wss://domain/apps/emails/socket.io/?EIO=3&transport=websocket&sid=[...]

The host resolution was a mistake on my end.

@thierrymarianne
Copy link

@nicofrand I have the socket transport mode working with the following rules for both basic apps requests (calendar, contacts, emails, files and photos) and the root path request:

RewriteCond             %{REQUEST_URI}          ^(.*)/socket\.io                [NC]
RewriteCond             %{QUERY_STRING}         transport=websocket             [NC]
RewriteRule             /(.*socket.io/.*)       ws://127.0.0.1:9104/$1          [P,L]

It might be considered as a an alternative to match requests using %{THE_REQUEST} server variable with apache (which use didn't work out for me with Apache/2.4.10).

P.S.

# helpful directives to debug rewriting and proxying
LogLevel alert rewrite:trace8
LogLevel debug

@nicofrand
Copy link
Author

I used %{THE_REQUEST} because some apps are not using the query string to specify the websocket transport. For example with Term :

wss://domain/apps/term/socket.io/1/websocket/_WMhXnFmnIsSeHwvAHzv

I do not see why that would not work for you :(.

@thierrymarianne
Copy link

@nicofrand I might have been too caffeinated yesterday (or the exact opposite might be true) as your original set of rules perfectly works as is for me today though I could not get Term installed to double-check with it. Sorry for the confusion.

@thierrymarianne
Copy link

@nicofrand The devil was in the details but I nailed it eventually:

RewriteEngine           On
RewriteCond             %{REQUEST_URI} ^/.*socket\.io [NC]
RewriteCond             %{THE_REQUEST} websocket [NC]
-RewriteRule            (.*)           ws://127.0.0.1:9104/$1 [P,L]
+RewriteRule            /(.*)          ws://127.0.0.1:9104/$1 [P,L]

@nicofrand
Copy link
Author

Oops, that 's what I have too on my server, I guess I forgot it here, I'll edit my post…
That would be cool if someone else could test with Term though.

@thierrymarianne
Copy link

@nicofrand I figured Term requires manual installation of coffee-script (speaking of being caffeinated... it was not enough in this regard). I can now confirm the Term app works as well with this set of rewriting rules :) Thank you!

@nicofrand
Copy link
Author

Cool, I'll see for the documentation and the cozy-debian package then.

@clochix
Copy link
Contributor

clochix commented Nov 20, 2015

I would like to thank all of you for the awesome job you did in fixing and documenting this !!! Cozy community rox !

@thierrymarianne
Copy link

@clochix Thank you for having provided us with great foundations for applications self-hosting :)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

3 participants