Backends with SRV records in server-template aren't resolved and used

[casKd-dev]

Detailed description of the problem

HAProxy 2.4.1 doesn't resolve the SRV record in server-template anymore.

Expected behavior

SRV records are resolved properly and the backend is used.

Steps to reproduce the behavior

1. Use minimal working example (see configuration below)
2. Start HAProxy 2.4.1 in foreground using haproxy -db -f /etc/redxen/haproxy/main.cfg
3. Get no [WARNING] xxx/xxx (xxx) : backend-example/example1 changed its IP from (none) to 10.x.x.x by DNS additional record.

Do you have any idea what may have caused this?

Recent server-template related commits might have caused them.

Do you have an idea how to solve the issue?

No.

What is your configuration?

Minimal example: (DNS server irrelevant as it is never queried)

global
  stats socket /run/haproxy.sock mode 600
  stats timeout 2m

defaults
  mode http
  default-server resolvers local

resolvers local
  nameserver example 127.0.0.1:53

frontend http
  bind ipv4@:80,ipv6@:80
  use_backend backend-example

backend backend-example
  server-template example 1 _http._tcp.domain

Output of haproxy -vv and uname -a

HAProxy version 2.4.1-1ce7d49 2021/06/17 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2026.
Known bugs: http://www.haproxy.org/bugs/bugs-2.4.1.html
Running on: Linux 5.12.8-0-edge #1-Alpine SMP PREEMPT Sat, 29 May 2021 06:59:58 +0000 x86_64
Build options :
  TARGET  = linux-musl
  CPU     = generic
  CC      = cc
  CFLAGS  = -Os -fomit-frame-pointer
  OPTIONS = USE_PCRE=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_NS=1 USE_PROMEX=1
  DEBUG   = 

Feature list : +EPOLL -KQUEUE +NETFILTER +PCRE -PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED -BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -CLOSEFROM +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS -OT -QUIC +PROMEX -MEMORY_PROFILING

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=16).
Built with OpenSSL version : OpenSSL 1.1.1k  25 Mar 2021
Running on OpenSSL version : OpenSSL 1.1.1k  25 Mar 2021
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.4.3
Built with the Prometheus exporter as a service
Built with network namespace support.
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE version : 8.44 2020-02-12
Running on PCRE version : 8.44 2020-02-12
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Encrypted password support via crypt(3): yes
Built with gcc compiler version 10.3.1 20210424

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
              h2 : mode=HTTP       side=FE|BE     mux=H2       flags=HTX|CLEAN_ABRT|HOL_RISK|NO_UPG
            fcgi : mode=HTTP       side=BE        mux=FCGI     flags=HTX|HOL_RISK|NO_UPG
       <default> : mode=HTTP       side=FE|BE     mux=H1       flags=HTX
              h1 : mode=HTTP       side=FE|BE     mux=H1       flags=HTX|NO_UPG
       <default> : mode=TCP        side=FE|BE     mux=PASS     flags=
            none : mode=TCP        side=FE|BE     mux=PASS     flags=NO_UPG

Available services : prometheus-exporter
Available filters :
	[SPOE] spoe
	[CACHE] cache
	[FCGI] fcgi-app
	[COMP] compression
	[TRACE] trace

Linux navi 5.12.8-0-edge #1-Alpine SMP PREEMPT Sat, 29 May 2021 06:59:58 +0000 x86_64 GNU/Linux

Additional information (if helpful)

• This only happens with SRV records of format _service._proto.domain and for others it works.
• Version 2.2.14 handles server-template correctly

[jmagnin]

Hello,

declaring the resolvers section is not sufficient. You need it referenced on server lines as well.
if you don't, the server name is only resolved when haproxy starts, and SRV records are not supported in this mode.
see also https://cbonte.github.io/haproxy-dconv/2.4/configuration.html#5.3.1.

[capflam]

I don't understand. As @jmagnin noticed, there is no "resolvers" option on your server-template definition. However, the message you reported is emitted by HAproxy when a response to a SRV request is received and an additional record is found for your server. Thus, in your scenario, a request was sent to the DNS server and an IP was assigned to your server via an additional record.

Please, try to better explain your problem and be sure it is really a bug.

[capflam]

Your description is not accurate. However, working on another issue, I found a bug. when a server-template relies on a SRV record, the first server is never resolved. The same is true for a server relying on a SRV record. I'll provide a fix tomorrow. The bug only affects the 2.4.1 and the 2.5-dev.

[capflam]

@casKd-dev, can you at least confirm it is you bug ? For instance by configuring 2 or more slot in your server-template definition. Thanks !