Openssl1.1

[rtorrentuser]

Unable to build libtorrent against openssl1.1 on Debian stretch. Everything was OK with openssl1.0.2
...
diffie_hellman.cc: In constructor ‘torrent::DiffieHellman::DiffieHellman(const unsigned char*, int, const unsigned char*, int)’:
diffie_hellman.cc:58:7: error: invalid use of incomplete type ‘DH {aka struct dh_st}’
m_dh->p = BN_bin2bn(prime, primeLength, NULL);
^~
In file included from /usr/include/openssl/bn.h:32:0,
from diffie_hellman.cc:43:
/usr/include/openssl/ossl_typ.h:104:16: note: forward declaration of ‘DH {aka struct dh_st}’
typedef struct dh_st DH;
^~~~~
diffie_hellman.cc:59:7: error: invalid use of incomplete type ‘DH {aka struct dh_st}’
m_dh->g = BN_bin2bn(generator, generatorLength, NULL);
^~
In file included from /usr/include/openssl/bn.h:32:0,
from diffie_hellman.cc:43:
/usr/include/openssl/ossl_typ.h:104:16: note: forward declaration of ‘DH {aka struct dh_st}’
typedef struct dh_st DH;
^~~~~
diffie_hellman.cc: In member function ‘bool torrent::DiffieHellman::is_valid() const’:
diffie_hellman.cc:77:30: error: invalid use of incomplete type ‘DH {aka struct dh_st}’
return m_dh != NULL && m_dh->pub_key != NULL;
^~
In file included from /usr/include/openssl/bn.h:32:0,
from diffie_hellman.cc:43:
/usr/include/openssl/ossl_typ.h:104:16: note: forward declaration of ‘DH {aka struct dh_st}’
typedef struct dh_st DH;
^~~~~
diffie_hellman.cc: In member function ‘bool torrent::DiffieHellman::compute_secret(const unsigned char*, unsigned int)’:
diffie_hellman.cc:95:17: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
return m_size != -1;
~~~~~~~^~~~~
In file included from diffie_hellman.cc:43:0:
diffie_hellman.cc: In member function ‘void torrent::DiffieHellman::store_pub_key(unsigned char*, unsigned int)’:
diffie_hellman.cc:106:39: error: invalid use of incomplete type ‘DH {aka struct dh_st}’
if ((int)length >= BN_num_bytes(m_dh->pub_key))
^
In file included from /usr/include/openssl/bn.h:32:0,
from diffie_hellman.cc:43:
/usr/include/openssl/ossl_typ.h:104:16: note: forward declaration of ‘DH {aka struct dh_st}’
typedef struct dh_st DH;
^~~~~
diffie_hellman.cc:107:19: error: invalid use of incomplete type ‘DH {aka struct dh_st}’
BN_bn2bin(m_dh->pub_key, dest + length - BN_num_bytes(m_dh->pub_key));
^~
In file included from /usr/include/openssl/bn.h:32:0,
from diffie_hellman.cc:43:
/usr/include/openssl/ossl_typ.h:104:16: note: forward declaration of ‘DH {aka struct dh_st}’
typedef struct dh_st DH;
^~~~~
In file included from diffie_hellman.cc:43:0:
diffie_hellman.cc:107:63: error: invalid use of incomplete type ‘DH {aka struct dh_st}’
BN_bn2bin(m_dh->pub_key, dest + length - BN_num_bytes(m_dh->pub_key));
^
In file included from /usr/include/openssl/bn.h:32:0,
from diffie_hellman.cc:43:
/usr/include/openssl/ossl_typ.h:104:16: note: forward declaration of ‘DH {aka struct dh_st}’
typedef struct dh_st DH;
^~~~~
Makefile:422: recipe for target 'diffie_hellman.lo' failed
make[3]: *** [diffie_hellman.lo] Error 1
...

[dhewg]

See 0007-Add-the-dh-openssl-1.1-patch-to-fix-the-build.patch attached to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=828414

[rtorrentuser]

It works! Thanks.

[rakshasa]

Could you turn that into a proper pull request I can apply?

[dhewg]

I'm not the patch author and don't want to take credit, but here's a git am ready patch.
0001-Fix-the-DH-parameters-generation-with-OpenSSL-1.1.patch.txt

[rakshasa]

The feature-bind branch now includes the changes above, with a check to see if openssl 1.1 is used.