[BUG] httpd don't handle any requests

[MosinAnton]

OpenSIPS version you are running

version: opensips 3.2.0 (x86_64/linux)
flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, HP_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll, sigio_rt, select.
main.c compiled on 17:30:17 Jul 21 2021 with gcc 4.8.5

Describe the bug
httpd don't handle requests in 3.2.0

To Reproduce

• Clean install from official repo on fresh updated Centos 7.
• Added loadmodule "httpd.so" loadmodule "mi_http.so" in default config.
• try "curl -v http://127.0.0.1:8888/mi"
• opensips listen 8888 port, but don't replied

tcp    LISTEN     3      32        *:8888                  *:*                   users:(("opensips",pid=1507,fd=51))


12:30:03.070064 IP 127.0.0.1.51990 > 127.0.0.1.8888: Flags [S], seq 3189146424, win 43690, options [mss 65495,sackOK,TS val 146025 ecr 0,nop,wscale 7], length 0
12:30:03.070123 IP 127.0.0.1.8888 > 127.0.0.1.51990: Flags [S.], seq 3712388354, ack 3189146425, win 43690, options [mss 65495,sackOK,TS val 146025 ecr 146025,nop,wscale 7], length 0
12:30:03.070185 IP 127.0.0.1.51990 > 127.0.0.1.8888: Flags [.], ack 1, win 342, options [nop,nop,TS val 146026 ecr 146025], length 0
12:30:03.070369 IP 127.0.0.1.51990 > 127.0.0.1.8888: Flags [P.], seq 1:79, ack 1, win 342, options [nop,nop,TS val 146026 ecr 146025], length 78
12:30:03.070380 IP 127.0.0.1.8888 > 127.0.0.1.51990: Flags [.], ack 79, win 342, options [nop,nop,TS val 146026 ecr 146026], length 0

OS/environment information

• Operating System: CentOS Linux release 7.9.2009 (Core)

Additional context
libmicrohttpd.x86_64 0.9.33-2.el7

[MosinAnton]

I replaced libmicrohttpd library file on 0.9.59-2.el7 in test enviroment and HTTPD module started working. But we can't use this version because it is not in the repository

[bogdan-iancu]

Hi @MosinAnton , with the culprit version, if you run opensips with debugs (log_level 4), do you see any logs when trying to send some request to the 8888 port? I want to see if there is any reading triggered there.
Also, could you do an "opensips-cli trap", to check the status of the OpenSIPS processes ?

[MosinAnton]

There is nothing in the logs.
gdb_opensips_20210726_104249.txt

[vasilevalex]

We had the same problem. And we just built libmicrohttpd.x86_64 0.9.59-1.el7 to make it work. The problem appears after httpd was migrated to the reactor framework. Seems that old version (0.9.33) can't work in this mode. But we didn't investigate more.

[bogdan-iancu]

Hi @vasilevalex , in ded, the new httpd code is using the MHD_USE_POLL option, which seems to be available starting 0.9.50. This is checked at the code level, when compiling the code (and using MHD_USE_EPOLL_LINUX_ONLY for older versions).
Bottom line, this issue is solved at code / compile time, but not at packages level. I guess we need to improve the package dependencies for the httpd module - let me check this with @nikbyte

[ovidiusas]

Maybe we should package libmicrohttpd just like wolfssl is packaged.

[bogdan-iancu]

Not sure if this is possible . What we need to do is to enforce, for the opensips-modules-httpd package , as install dependency, a requirement to a libmicrohttpd newer than 0.9.50. And this should address the issue.

[nikbyte]

On the repository there is a remark that build for centos uses epel repository. Opensips built using some libraries from epel, which has newer libhttpd.

[bogdan-iancu]

@nikbyte , maybe here https://github.com/OpenSIPS/opensips/blob/master/packaging/redhat_fedora/opensips.spec#L351 we should add a version requirement too

[vasilevalex]

I build OpenSIPS for CentOS7 from git with mock (using epel) and there is only 0.9.33-2 version available. And I've seen that compiling is conditional, comparing version with 0.9.50, but seems, that 0.9.33 already has flag MHD_USE_EPOLL_LINUX_ONLY, but don't work with it properly (not 100% sure). We took libmicrohttpd.spec for 0.9.59-1 (the same is in CentOS 8). So may be the simplest way is to put libmicrohttpd.x86_64 0.9.59-1.el7.rpm to CentOS7 repo for OpenSIPS? Not so conservative distros already have working version of this library.

[bogdan-iancu]

OK, after some brainstorming with @razvancrainea , we decided not do any version based dependency, but rather to better do a runtime detection, on opensips startup, by using the MHD_is_feature_supported() function, to check if the MHD_FEATURE_POLL feature is available - this will generate a negative return code, if not available.
It is not a fix, but it will clearly indicate you the problem

[nikbyte]

Just for info. I checked the build server and it uses 0.9.33-2 to build on el7.

[vasilevalex]

And 0.9.59-2 uses different API: libmicrohttpd.so.10 (0.9.33-2) and libmicrohttpd.so.12 (0.9.59-1), so it's impossible just to install OpenSIPS from package and then install library on this server. OpenSIPS must be built with library with the same API.

[MosinAnton]

we took the packages from here https://cbs.centos.org/koji/buildinfo?buildID=22621 and put them in our repository. Built Opensips with version 0.9.59-2.el7 and everything installs and works fine