Alternative queue rarely works incorrect

[andrew2311]

I've used 1.10 civetweb release version, and my internal tests rarely stucks at web socket connection creation.
Also issue was reproduced using "clean" civetweb code.

Steps to reproduce:
0. My environment - Ubuntu 16.04 (uname -a output: Linux work 4.13.0-38-generic #43~16.04.1-Ubuntu SMP Wed Mar 14 17:48:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux)

1. Use 1.10 civetweb release (https://github.com/civetweb/civetweb/archive/v1.10.tar.gz)
2. Compile with next options: make slib WITH_WEBSOCKET=1 WITH_CPP=1 WITH_IPV6=1 WITH_DEBUG=1 (with debug issue reproduces faster)
3. Use simple c++ test with server, make 10 connections and clean-up resources - run it forever (using bash script). Source file and script attached.
4. Compile test.cpp with next options: g++ test.cpp -L. -lcivetweb -lpthread -ldl -o TestEmptyServer
   (g++ (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609)
5. Wait until test stucks on any connection creation (in my case it was 178th run (always different), may be much more if use release version)

Note: when I built civetweb with COPT='-DNO_ALTERNATIVE_QUEUE' option, debug or release version, I have no such issue (stuck at connection creation).

P.S. Could you please double check that I've correctly created server and clients, and correctly clean-up resources after test? Also this code is just an example to reproduce an issue.

test.cpp.txt
run_test.sh.txt
cut_test_log.txt

[xtne6f]

I have opened a pull request #661 for this issue.
I think the stuck is roughly caused by following steps.

<produce_socket() in the master thread>
  L16016: client_socks[0] = *sp; // socket for 1st connection
  L16017: client_socks[0].in_use = 1;
  L16018: event_signal();
<consume_socket() in 0th worker thread>
  L16032: client_socks[0].in_use = 0;
  L16033: event_wait();
  L16034: *sp = client_socks[0]; // socket for 1st connection
  .. busy until 1st connection is closed.
<produce_socket() in the master thread>
  L16016: client_socks[0] = *sp; // socket for 2nd connection
  L16017: client_socks[0].in_use = 1;
  L16018: event_signal();
  .. 2nd connection stucks because 0th worker thread is busy.

[xtne6f]

@andrew2311 I would be happy if you could reply something or closing this.
@bel2125 I think there are no differences between ALTERNATIVE_QUEUE and NO_ALT... for Civetweb users, so no reason to optionaize them. Existing of several approaches increases maintenance cost and prevents understanding the flow of code. I would recommend choosing one as soon as you can.

[jodogne]

If only one implementation should be kept, I would highly recommend keeping NO_ALTERNATIVE_QUEUE, as it is the only one that is compatible with Linux Standard Base (LSB).