Communication on ARMv5TE fails #264

Closed
pettno opened this Issue Jun 10, 2013 · 8 comments

Projects

None yet

3 participants

@pettno
pettno commented Jun 10, 2013

Connecting from x86-64 linux to an ARMv5TE target and sending data fails. Usually this library gets a "consume error: Invalid UTF8 encoding" before closing down the connection. On occation we can get some data transfer, but the data sent from the web browser to our application containing websocketpp gets garbled (compiling and using our application for 64 bit x86 and 32 bit powerpc targets works fine).

TCP stream from Wireshark when data is transferred to an ARMv5TE target:

00000000  47 45 54 20 2f 6a 62 70  20 48 54 54 50 2f 31 2e GET /jbp  HTTP/1.
00000010  31 0d 0a 55 70 67 72 61  64 65 3a 20 77 65 62 73 1..Upgra de: webs
00000020  6f 63 6b 65 74 0d 0a 43  6f 6e 6e 65 63 74 69 6f ocket..C onnectio
00000030  6e 3a 20 55 70 67 72 61  64 65 0d 0a 48 6f 73 74 n: Upgra de..Host
00000040  3a 20 31 39 32 2e 31 36  38 2e 31 30 2e 36 33 3a : 192.16 8.10.63:
00000050  38 30 38 30 0d 0a 4f 72  69 67 69 6e 3a 20 6e 75 8080..Or igin: nu
00000060  6c 6c 0d 0a 50 72 61 67  6d 61 3a 20 6e 6f 2d 63 ll..Prag ma: no-c
00000070  61 63 68 65 0d 0a 43 61  63 68 65 2d 43 6f 6e 74 ache..Ca che-Cont
00000080  72 6f 6c 3a 20 6e 6f 2d  63 61 63 68 65 0d 0a 53 rol: no- cache..S
00000090  65 63 2d 57 65 62 53 6f  63 6b 65 74 2d 4b 65 79 ec-WebSo cket-Key
000000A0  3a 20 65 74 2f 52 34 42  52 58 78 78 71 2f 52 4a : et/R4B RXxxq/RJ
000000B0  2f 2f 6a 46 55 4c 6a 41  3d 3d 0d 0a 53 65 63 2d //jFULjA ==..Sec-
000000C0  57 65 62 53 6f 63 6b 65  74 2d 56 65 72 73 69 6f WebSocke t-Versio
000000D0  6e 3a 20 31 33 0d 0a 53  65 63 2d 57 65 62 53 6f n: 13..S ec-WebSo
000000E0  63 6b 65 74 2d 45 78 74  65 6e 73 69 6f 6e 73 3a cket-Ext ensions:
000000F0  20 78 2d 77 65 62 6b 69  74 2d 64 65 66 6c 61 74  x-webki t-deflat
00000100  65 2d 66 72 61 6d 65 0d  0a 55 73 65 72 2d 41 67 e-frame. .User-Ag
00000110  65 6e 74 3a 20 4d 6f 7a  69 6c 6c 61 2f 35 2e 30 ent: Moz illa/5.0
00000120  20 28 58 31 31 3b 20 4c  69 6e 75 78 20 78 38 36  (X11; L inux x86
00000130  5f 36 34 29 20 41 70 70  6c 65 57 65 62 4b 69 74 _64) App leWebKit
00000140  2f 35 33 37 2e 33 36 20  28 4b 48 54 4d 4c 2c 20 /537.36  (KHTML, 
00000150  6c 69 6b 65 20 47 65 63  6b 6f 29 20 43 68 72 6f like Gec ko) Chro
00000160  6d 65 2f 32 37 2e 30 2e  31 34 35 33 2e 31 31 30 me/27.0. 1453.110
00000170  20 53 61 66 61 72 69 2f  35 33 37 2e 33 36 0d 0a  Safari/ 537.36..
00000180  0d 0a                                            ..

    00000000  48 54 54 50 2f 31 2e 31  20 31 30 31 20 53 77 69 HTTP/1.1  101 Swi
    00000010  74 63 68 69 6e 67 20 50  72 6f 74 6f 63 6f 6c 73 tching P rotocols
    00000020  0d 0a 43 6f 6e 6e 65 63  74 69 6f 6e 3a 20 75 70 ..Connec tion: up
    00000030  67 72 61 64 65 0d 0a 53  65 63 2d 57 65 62 53 6f grade..S ec-WebSo
    00000040  63 6b 65 74 2d 41 63 63  65 70 74 3a 20 52 79 32 cket-Acc ept: Ry2
    00000050  4e 37 74 39 56 4a 6e 30  6f 57 61 38 39 55 4c 41 N7t9VJn0 oWa89ULA
    00000060  6d 45 62 77 4c 78 67 45  3d 0d 0a 53 65 72 76 65 mEbwLxgE =..Serve
    00000070  72 3a 20 57 65 62 53 6f  63 6b 65 74 2b 2b 2f 30 r: WebSo cket++/0
    00000080  2e 33 2e 30 2d 61 6c 70  68 61 32 0d 0a 55 70 67 .3.0-alp ha2..Upg
    00000090  72 61 64 65 3a 20 77 65  62 73 6f 63 6b 65 74 0d rade: we bsocket.
    000000A0  0a 0d 0a                                         ...

00000182  81 aa e9 a5 a2 c7 92 af  80 b3 90 d5 c7 e5 d3 85 ........ ........
00000192  80 a0 8c d1 fd a3 8c d3  cb a4 8c d6 80 eb e3 87 ........ ........
000001A2  d0 a2 8f d7 c7 b4 81 87  98 e7 9d d7 d7 a2 e3 d8 ........ ........

This results in the following data in our "on_message" method:

00 00 79 70 22 74 3A 20 65 22 65 74 22 67 65 76 
5F 64 65 73 69 63 0A 22 22 2C 66 72 72 65 68 22 
65 73 74 72 3A 20 D7 A2 0A 7D

Wireshark seems to correctly unmask the payload (92 af 80 b3, 90, ...) to:

7b 0a 22 74 79 70 65 22 3a 20 22 67 65 74 5f 64
65 76 69 63 65 73 22 2c 0a 22 72 65 66 72 65 73
68 22 3a 20 74 72 75 65 0a 7d

Data in the other direction after this (which is our error response due to a garbled message) looks fine. Our platform is a custom PXA255 (32-bit ARMv5TE) system generated by Buildroot.

Issue #145 might be related to this issue. The websocketpp library does currently not compile when defining the WEBSOCKETPP_STRICT_MASKING macro.

@zaphoyd zaphoyd added a commit that referenced this issue Jun 13, 2013
@zaphoyd update byte mask to use separate input & output types references #264
In particular this allows const iterators to be use for the input types.
cae30ac
@zaphoyd
Owner
zaphoyd commented Jun 13, 2013

I've pushed an update that should correct the compile issue with WEBSOCKETPP_STRICT_MASKING. That may or may not help with the core issue here. Knowing whether or not it does will help narrow this down further though.

I've identified a few other areas where non-strict aliasing is being used and will work on cleaning that up to see if that helps at all.

@pettno
pettno commented Jun 13, 2013

The update works for compiling with WEBSOCKETPP_STRICT_MASKING. When doing so and testing, the core issue is still the same.

@zaphoyd
Owner
zaphoyd commented Jun 15, 2013

I've put up another build that enables strict masking for incoming messages. Previously it only applied to outgoing ones. Does this change anything?

If not, is your platform capable of running the unit tests? Ideally scons test in the build directory or, if you don't have scons available, g++ test/utility/frame.cpp -I. -static -lboost_system -lboost_unit_test_framework -o test_frame would build be the most useful individual test.

@pettno
pettno commented Jun 17, 2013

The last change results in correctly unmasked data in both directions when compiling with WEBSOCKETPP_STRICT_MASKING.

When compiling the test:

pettno ~/wsd/websocketpp $ arm-linux-g++ -c test/utility/frame.cpp -I . -static -lboost_system -lboost_unit_test_framework -o test_frame
test/utility/frame.cpp:242: error: integer constant is too large for 'long' type
test/utility/frame.cpp:257: error: integer constant is too large for 'long' type
pettno ~/wsd/websocketpp $ 
@pettno pettno closed this Jun 17, 2013
@pettno pettno reopened this Jun 17, 2013
@pettno
pettno commented Jun 26, 2013

Ouch. The boost unit test framework is not available in our ARM cross-toolchain for this target. This makes us currently unable to link an executable of the test for it. We have decided to use the STRICT_MASKING for this target, at least for now, and move on to work on other matters. Do you want me to close this issue?

@zaphoyd
Owner
zaphoyd commented Jun 26, 2013

Using STRICT_MASKING will slow down client->server messages slightly but otherwise shouldn't have any problematic effects. Lets go ahead and close this issue. I'll open another one to track a lower priority "support non-standard masking optimizations on ARM".

Non-strict masking deliberately uses unsafe casting & aliasing to improve performance on the most commonly used hardware. I would still like to explore whether or not this optimization can be used on ARM, but I think I will need to get my hands on some ARM hardware to experiment with. Do you have any suggestions on a good reference ARM platform to test with?

Also, is there anything more you can say about the limitations with respect to the boost unit test framework on this target?

@pettno
pettno commented Jun 27, 2013

The Raspberry Pi is cheap and handy. It contains an ARM11 and could be used as a reference platform. Scons should not be a problem on it. I actually compiled and tested our application on it yesterday evening. No communication issues. My Pi use Raspbian, gcc 4.6.3 and boost 1.49.

I guess our ARMv5TE issue can be related to versions of boost (1.52) and/or gcc (4.2.4). The cross-toolchain we use for this platform is a bit complex to upgrade/build, and we hope it won't be necessary. The old gcc is related to usage of an old linux kernel (2.6.27).

In an attempt to understand more, I tried compiling our application on a FOX G20 board without success. This board has 64MB RAM, which is not enough to compile all the boost stuff. I have (currently) not built a cross-toolchain for this platform.

I'll close this issue now. Thanks for all your efforts on making this great library. It's much appreciated. Don't hesitate to PM me if you think I can be of assistance.

@pettno pettno closed this Jun 27, 2013
@kometen
kometen commented Oct 26, 2015

I received a similar error on FreeBSD 10.2 stable on raspberry pi and websocketpp 0.6.0. Compiled using

clang++ -Wall -std=c++11 -I/usr/local/include -L/usr/local/lib -lpthread -lboost_system -o main main.cpp

Works on OS X (El Capitan) and FreeBSD 10.2 stable on amd64 (vmware-guest).

Modified the 'extended connections storage' example on http://www.zaphoyd.com/websocketpp/manual/common-patterns/storing-connection-specificsession-information.

Shall I provide more information?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment