Need help for Multi-threading

[EhsanVahab]

Hi friend @bsergean,
I know my question is not completely related to your great library, but I need some advice to take full advantage of IXWebsocket and I will be thankful if you help me.
I use the IXWebsocket' client (only client not server) in a multi-threading application(one instance in 10 threads) which transmits data with len=500000 (~500kb) to the server.
1.Is there any configuration that I should disable? because of not using the onMessage event on the client side?
2. Should I use mutex.lock when I use sendText function?
3.everything is going well when the app and server are both on the same machine but when I test it on the real word or Internet, the client takes high processing cost. is there any recommendation to handle this situation?

System Specification:
Server : Ubuntu with Tomcat for socket server
Client : Raspbian Model B+ with Processing unit 1.2 GHz

[bsergean]

So it looks like your client only 'sends' data to the server right ? Is your server reply back with the same data ? If that's the case stop doing that as it will take time to 'download and process that data on the client'.

Also are you compiling the library in Release mode ? Here is what you should do to do that if you are using the top level makefile.

diff --git a/makefile b/makefile
index 30b3a29..01a3ba5 100644
--- a/makefile
+++ b/makefile
@@ -9,10 +9,10 @@ install: brew
 # on osx it is good practice to make /usr/local user writable
 # sudo chown -R `whoami`/staff /usr/local
 brew:
-       mkdir -p build && (cd build ; cmake -DUSE_TLS=1 -DUSE_WS=1 .. ; make -j install)
+       mkdir -p build && (cd build ; cmake -DCMAKE_BUILD_TYPE=Release -DUSE_TLS=1 -DUSE_WS=1 .. ; make -j install)
 
 ws:
-       mkdir -p build && (cd build ; cmake -DUSE_TLS=1 -DUSE_WS=1 -DUSE_MBED_TLS=1 -DUSE_VENDORED_THIRD_PARTY=1 .. ; make -j)
+       mkdir -p build && (cd build ; cmake -DCMAKE_BUILD_TYPE=Release -DUSE_TLS=1 -DUSE_WS=1 -DUSE_MBED_TLS=1 -DUSE_VENDORED_THIRD_PARTY=1 .. ; make -j)
 
 uninstall:
        xargs rm -fv < build/install_manifest.txt

[EhsanVahab]

My server doesn't reply any message to the client in this part of app.
No I haven't compiled IXW in Release mode. I've just used:

make

sudo make install
`
```

[bsergean]

If your messages are 500K large (which isn't nothing), maybe you are running into bandwidth problems. Can you try to *enable* per message deflate compression. That should help with that. If you are CPU bound, you can try to disable it.

…

On Jun 18, 2019, at 10:35 AM, EhsanVahab ***@***.***> wrote: My server doesn't reply any message to the client in this part of app. No I haven't compiled IXW in Release mode. I've just used: make sudo make install ` ``` — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#93?email_source=notifications&email_token=AC2O6ULST3APKERDDFVQBALP3EMFVA5CNFSM4HZAOQ5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX7MNGQ#issuecomment-503236250>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC2O6UM3DO6FFJTQPHYUPKLP3EMFVANCNFSM4HZAOQ5A>.

[EhsanVahab]

The BW is not the problem.
I will build IXW in ReleaseMode then enable per message comperession. Good advices
I let you know the result.
Thans for your help.
You always come to my aid

[bsergean]

Sounds good, tell me how it goes. The next step will be to run a proper profiler for your code, you can look at 'perf' on Linux. http://www.brendangregg.com/perf.html

…

On Jun 18, 2019, at 10:54 AM, EhsanVahab ***@***.***> wrote: The BW is not the problem. I will build IXW in ReleaseMode then enable per message comperession. Good advices I let you know the result. Thans for your help. You always come to my aid — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#93?email_source=notifications&email_token=AC2O6UP37MRNUL7VYSCTXWLP3EOORA5CNFSM4HZAOQ5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX7OIDI#issuecomment-503243789>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC2O6UNZWQKZ2KCWV754J7LP3EOORANCNFSM4HZAOQ5A>.

[EhsanVahab]

Dear @bsergean ,
I've compiled IXW in ReleaseMode and tested with disabling and enabling perMessageDeflate.
sadly non of them has been highly effective.
the another parameter that crosses my mind is fragmentation size.
One the server side, I get the data fragmented into 42 to 45 parts(the length of each one is exactly 8192 except the last part).
So I took a look at the IXWebsocketTransport.cpp line:796 and realized that seemingly the large data is broken into some parts.
Is there any parameter to increase the len of each fragmented part or decrease the number of part?
I guess by decreasing the step size, we will have lower processing cost

[EhsanVahab]

I got new thing that I think it's very important.
when there is no packet to be sent, the cpu is still busy and around 20% of it's performance is taken.
I think the IXW tries to have a negotiation with the server to keep its connection alive.

[bsergean]

At what frequency are you pushing messages ? 500k messages each seconds ? There is a heartbeat mechanism that will send periodically ping messages to keep the connection alive. Is it active ?

…

On Jun 19, 2019, at 3:54 AM, EhsanVahab ***@***.***> wrote: I got new thing that I think it's very important. when there is no packet to be sent, the cpu is still busy and around 20% of it's performance is taken. I think the IXW tries to have a negotiation with the server to keep its connection alive. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#93?email_source=notifications&email_token=AC2O6UMGXPCLBG5WHNCVQRDP3IF6RA5CNFSM4HZAOQ5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYBPP7Y#issuecomment-503511039>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC2O6UNEOUSK6QHL4RS7MATP3IF6RANCNFSM4HZAOQ5A>.

[EhsanVahab]

Dear @bsergean ,
seemingly my last message hasn't been sent on the github.
doesn't matter. I find the problem.
there is an endless loop in IXWebsocket.cpp line: 291 to 351
I don't know why and don't have enough OS information about that, but I'm pretty sure, based my experiences , Raspbian is too sensitive to endless loop. it takes high processing cost even there is no command inside.
I put sleep(5) inside the loop and it's solved. I know if I have a message from the server, it's triggered every 5 seconds but my server's message is not so critical.

[bsergean]

I tried running ws connect on macOS, and watched CPU usage which was at 0%.

If you have a standalone program that I can run and that display the problem please attach it to this issue and I can try to troubleshoot this further. There shouldn't be any need for that sleep in that wait loop. There should be no sleep call in that wait loop, the loop block waiting for input when it call poll.

I'm happy if you found a workaround, but I don't think that the whole mystery is solved.

[EhsanVahab]

I tested on the Ubuntu too and the CPU usage was around 1%
As I mentioned I don't why the Raspbian gets stuck into trouble when there's an endless loop,even an empty loop. it's a little bit strange to me.
I'm working on it. I'll send you a standalone instance of the app to be watched

[bsergean]

Try to run: strace <the progrem> It will list all the system calls being run, very helpful.

…

On Jun 24, 2019, at 12:02 AM, EhsanVahab ***@***.***> wrote: I tested on the Ubuntu too and the CPU usage was around 1% As I mentioned I don't why the Raspbian gets stuck into trouble when there's an endless loop,even an empty loop. it's a little bit strange to me. I'm working on it. I'll send you a standalone instance of the app to be watched — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#93?email_source=notifications&email_token=AC2O6UMDYG3VIDDLDLM5M4TP4BWO3A5CNFSM4HZAOQ5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYL65JI#issuecomment-504884901>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC2O6UN3UHNIV6WFYN3XRCDP4BWO3ANCNFSM4HZAOQ5A>.