haproxy 2.4 opentracing not send span if daemon option is configured

[bandrea83]

Detailed description of the problem

if in global section you enable daemon option trace is not sended to zipkin endpoint

Expected behavior

it also send span to zipkin server if daemon option is configured

Steps to reproduce the behavior

enable daemon in [global] section

1. enable OT
2. in global section enable daemon option
3. start haproxy

Do you have any idea what may have caused this?

Do you have an idea how to solve the issue?

What is your configuration?

global
    daemon
#   nbthread 1
    maxconn 5000
    hard-stop-after 10s
#   log localhost:514 local7 debug
#   debug
    stats socket /tmp/haproxy-fe.sock mode 666 level admin

defaults
    log     global
    option  httplog
    option  dontlognull
    retries 3
    maxconn 4000
    timeout connect 5000
    timeout client  50000
    timeout server  50000

listen stats
    mode http
    bind *:8001
    stats uri /
    stats admin if TRUE
    stats refresh 10s

frontend ot-test-fe-frontend
    filter opentracing id ot-test-fe config fe/ot.cfg
    bind *:10080
    mode http
    default_backend servers-backend


backend servers-backend
    mode http
    server server-1 127.0.0.1:11080

Output of haproxy -vv and uname -a

HA-Proxy version 2.4-dev13-09cc669 2021/03/19 - https://haproxy.org/
Status: development branch - not safe for use in production.
Known bugs: https://github.com/haproxy/haproxy/issues?q=is:issue+is:open
Running on: Linux 3.10.0-1160.15.2.el7.x86_64 #1 SMP Thu Jan 21 16:15:07 EST 2021 x86_64
Build options :
  TARGET  = linux-glibc
  CPU     = generic
  CC      = cc
  CFLAGS  = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -Wno-unused-label -DTCP_USER_TIMEOUT=18
  OPTIONS = USE_EPOLL=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_THREAD=1 USE_LINUX_TPROXY=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_DL=1 USE_OT=1
  DEBUG   =

Feature list : +EPOLL -KQUEUE +NETFILTER +PCRE +PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -CLOSEFROM +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS +OT -QUIC

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=4).
Built with OpenSSL version : OpenSSL 1.1.1e  17 Mar 2020
Running on OpenSSL version : OpenSSL 1.1.1e  17 Mar 2020
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.5
Built with network namespace support.
Built with OpenTracing support.
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 4.8.5 20150623 (Red Hat 4.8.5-44)

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
              h2 : mode=HTTP       side=FE|BE     mux=H2
            fcgi : mode=HTTP       side=BE        mux=FCGI
       <default> : mode=HTTP       side=FE|BE     mux=H1
       <default> : mode=TCP        side=FE|BE     mux=PASS

Available services : none

Available filters :
        [  OT] opentracing
        [SPOE] spoe
        [CACHE] cache
        [FCGI] fcgi-app
        [COMP] compression
        [TRACE] trace

$ uname -a
Linux 184260ebf98c 3.10.0-1160.15.2.el7.x86_64 #1 SMP Thu Jan 21 16:15:07 EST 2021 x86_64 x86_64 x86_64 GNU/Linux

[zaga00]

Hello @bandrea83,

thank you for reporting the bug. The problem is that the OpenTracing library is initialized before HAProxy initialize the daemon mode. Establishing this mode kills the OpenTracing thread, after which the correct operation of the OpenTracing filter is no longer possible. Also, HAProxy crashes on deinitialization of the OpenTracing library. That will be resolved tomorrow.

[zaga00]

Hello @bandrea83,

have you tried whether the OpenTracing filter now sends data to the tracer?

[bandrea83]

Hi @zaga00 , seem to work as expected now

Nice work and thank you so much! haproxy team is doing a so great work ;)