Compile issues with openssl 1.1.0c

[kittel]

ISSUE TYPE

• Bug Report

DIONAEA VERSION

"627d98311820b27a15ecb8f872fda82648e4f969"

CONFIGURATION

No Dionaea config yet

OS / ENVIRONMENT

• Debian 9.0

SUMMARY

The code does not compile with openssl 1.1.0c due to (at least) the following error messages:
(Additional warnings attached below)

Problem: With openssl1.0.1 "struct ssl_st" was defined in "openssl/ssl.h" [1]. In openssl1.1.0c it is defined in the file "ssl_locl.h" [2] which is not part of the include directory.

connection.c: In function 'myssl_dh_configure':
connection.c:2268:4: error: dereferencing pointer to incomplete type 'DH {aka struct dh_st}'
  dh->p = BN_bin2bn(p, plen, NULL);
    ^~
connection.c: In function 'connection_dtls_io_in_cb':
connection.c:4021:29: error: dereferencing pointer to incomplete type 'SSL {aka struct ssl_st}'
     peer->transport.dtls.ssl->d1->listen = 1;
                             ^~

[1] https://fossies.org/dox/openssl-1.0.2j/structssl__st.html
[2] https://fossies.org/dox/openssl-1.1.0c/structssl__st.html

STEPS TO REPRODUCE

• Dependencies taken from: https://dionaea.readthedocs.io/en/latest/installation.html
• Minor: libnl-dev does not exist in debian, thus using libnl-3-dev (also true for jessie).

debootstrap stretch stretch http://deb.debian.org/debian/
chroot stretch

apt-get update && apt-get install -y git autoconf automake build-essential check cython3 libcurl4-openssl-dev libemu-dev libev-dev libglib2.0-dev libloudmouth1-dev libnetfilter-queue-dev libnl-3-dev libpcap-dev libssl-dev libtool libudns-dev python3 python3-dev python3-yaml

cd /opt && git clone https://github.com/DinoTools/dionaea.git && cd - && cd /opt/dionaea && autoreconf -vi && ./configure --disable-werror --prefix=/opt/dionaea --with-python=/usr/bin/python3 --with-cython-dir=/usr/bin --with-ev-include=/usr/include --with-ev-lib=/usr/lib --with-emu-lib=/usr/lib/libemu --with-emu-include=/usr/include --with-nl-include=/usr/include/libnl3 --with-nl-lib=/usr/lib && make && make install && cd -

EXPECTED RESULTS

Compile finishes successful.

ACTUAL RESULTS

gcc -DHAVE_CONFIG_H -I. -I..    -I/usr/include -DEV_COMPAT3=0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -pthread -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -pthread -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include    -I../include -I .. -fno-strict-aliasing -std=c99 -D_GNU_SOURCE -D_GNU_SOURCE -I/usr/include -DEV_COMPAT3=0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -pthread -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -pthread -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include  -Wall  -Wstrict-prototypes -g  -MT dionaea-connection.o -MD -MP -MF .deps/dionaea-connection.Tpo -c -o dionaea-connection.o `test -f 'connection.c' || echo './'`connection.c
connection.c: In function 'connection_new':
connection.c:129:3: warning: 'DTLSv1_method' is deprecated [-Wdeprecated-declarations]
   con->transport.tls.meth = DTLSv1_method();
   ^~~
In file included from /usr/include/openssl/ct.h:13:0,
                 from /usr/include/openssl/ssl.h:61,
                 from ../include/connection.h:41,
                 from connection.c:69:
/usr/include/openssl/ssl.h:1614:1: note: declared here
 DEPRECATEDIN_1_1_0(__owur const SSL_METHOD *DTLSv1_method(void)) /* DTLSv1.0 */
 ^
connection.c: In function 'connection_listen':
connection.c:474:57: warning: passing argument 2 of 'SSL_CTX_set_cookie_verify_cb' from incompatible pointer type [-Wincompatible-pointer-types]
   SSL_CTX_set_cookie_verify_cb(con->transport.dtls.ctx, dtls_verify_cookie_cb);
                                                         ^~~~~~~~~~~~~~~~~~~~~
In file included from ../include/connection.h:41:0,
                 from connection.c:69:
/usr/include/openssl/ssl.h:646:6: note: expected 'int (*)(SSL *, const unsigned char *, unsigned int) {aka int (*)(struct ssl_st *, const unsigned char *, unsigned int)}' but argument is of type 'int (*)(SSL *, unsigned char *, unsigned int) {aka int (*)(struct ssl_st *, unsigned char *, unsigned int)}'
 void SSL_CTX_set_cookie_verify_cb(SSL_CTX *ctx,
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
connection.c: In function 'myssl_dh_configure':
connection.c:2268:4: error: dereferencing pointer to incomplete type 'DH {aka struct dh_st}'
  dh->p = BN_bin2bn(p, plen, NULL);
    ^~
connection.c: In function 'ssl_tmp_key_init_rsa':
connection.c:2404:2: warning: 'RSA_generate_key' is deprecated [-Wdeprecated-declarations]
  if( !(con->transport.tls.pTmpKeys[idx] = RSA_generate_key(bits, RSA_F4, NULL, NULL)) )
  ^~
In file included from /usr/include/openssl/rsa.h:13:0,
                 from /usr/include/openssl/x509.h:31,
                 from /usr/include/openssl/x509v3.h:14,
                 from connection.c:50:
/usr/include/openssl/rsa.h:193:1: note: declared here
 DEPRECATEDIN_0_9_8(RSA *RSA_generate_key(int bits, unsigned long e, void
 ^
connection.c: In function 'mkcert':
connection.c:2583:2: warning: 'RSA_generate_key' is deprecated [-Wdeprecated-declarations]
  rsa=RSA_generate_key(bits,RSA_F4,callback,NULL);
  ^~~
In file included from /usr/include/openssl/rsa.h:13:0,
                 from /usr/include/openssl/x509.h:31,
                 from /usr/include/openssl/x509v3.h:14,
                 from connection.c:50:
/usr/include/openssl/rsa.h:193:1: note: declared here
 DEPRECATEDIN_0_9_8(RSA *RSA_generate_key(int bits, unsigned long e, void
 ^
connection.c: In function 'connection_dtls_io_in_cb':
connection.c:4021:29: error: dereferencing pointer to incomplete type 'SSL {aka struct ssl_st}'
     peer->transport.dtls.ssl->d1->listen = 1;
                             ^~
Makefile:562: recipe for target 'dionaea-connection.o' failed
make[2]: *** [dionaea-connection.o] Error 1
make[2]: Leaving directory '/opt/dionaea/src'
Makefile:491: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/opt/dionaea'
Makefile:399: recipe for target 'all' failed
make: *** [all] Error 2

[phibos]

Thanks for reporting the issue and providing all the information. I will have a look at it.

[phibos]

Some parts of the connection functions have been rewritten and a CI test for Debian 9 has been added. Please feel free to use the latest source from the master branch and rebuild dionaea.

[phibos]

The issue should be fixed. Please feel free to reopen or create a new issue if you think there are still issues.