libssh2: compile error

[stintel]

Maintainer: @jslachta
Environment: OpenWrt master r13174-73fa1aba94 on octeon

Description:
Building an image for my ERL fails, due to a compile error in libssh2, which is not even enabled in my .config.

stijn@taz ~/Development/LEDE/source $ grep ssh2 .config
# CONFIG_PACKAGE_libssh2 is not set

[stintel]

compile.txt

[neheb]

#11796

[stintel]

I can apparently work around by adding this to the Makefile:

PKG_BUILD_DEPENDS:=libmbedtls

Don't know if that's an acceptable solution, or why libssh2 is being built in the first place.

[neheb]

It is not. And no clue.

libssh2 can use either mbedtls or openssl.

[micmac1]

I would say the problem is that some package has a "lazy" dependency on libssh2. For instance nmap. It has "+libssh2" for nmap-full. If instead it had "+PACKAGE_nmap-full:libssh2" then libssh2 wouldn't be built before nmap while nmap-full is not selected while, for instance, ncat is selected.

[micmac1]

For cases where you have something like "+ARIA2_SFTP:libssh2" it seems not possible to avoid this. In that case adding PKG_BUILD_DEPENDs to libssh2 might be best. Although I would probably try to make PKG_BUILD_DEPENDs dependent on the abscence of CONFIG_LIBSSH2_MBEDTLS and CONFIG_LIBSSH2_OPENSSL.

Edit: actually "+ARIA2_SFTP:libssh2" should not pull in libssh2 if CONFIG_ARIA2_SFTP is not set, for the same reason that libssh2 doesn't pull in libmbedtls if CONFIG_LIBSSH2_MBEDTLS is unset. So this wouldn't be a "lazy" dependency. Still, PKG_BUILD_DEPENDs for libssh2 may be a good workaround for "lazy" dependencies like nmap-full.

[cotequeiroz]

I would say the problem is that some package has a "lazy" dependency on libssh2. For instance nmap. It has "+libssh2" for nmap-full. If instead it had "+PACKAGE_nmap-full:libssh2" then libssh2 wouldn't be built before nmap while nmap-full is not selected while, for instance, ncat is selected.

Why fixing it at the source by adding +PACKAGE_nmap-full:libssh2 to nmap? It tackles the problem right at the origin. The generated dependencies can be seen at tmp/.packagedeps. The packages that depend on libssh2 unconditionally can be seen with:

$ grep -E '.*=.*libssh2/compile($|[^)])' tmp/.packagedeps
$(curdir)/feeds/packages/nmap/compile += $(curdir)/feeds/packages/libssh2/compile $(curdir)/feeds/packages/pcre/compile $(curdir)/feeds/packages/python3/compile $(curdir)/libs/libpcap/compile $(curdir)/libs/openssl/compile $(curdir)/libs/toolchain/compile $(curdir)/libs/zlib/compile $(curdir)/utils/lua5.3/compile $(if $(CONFIG_GCC_LIBSSP),$(curdir)/libs/toolchain/compile) $(if $(CONFIG_USE_GLIBC),$(curdir)/libs/toolchain/compile)

The problem with adding mbedtls to PKG_BUILD_DEPENDS, and not tacking nmap is that you're always forcing a build of libssh2 and mbedtls even if you don't need them (you're not selecting nmap-full.

On its own, adding mbedtls to libssh2 PKG_BUILD_DEPENDS will case, in an eventual failure to build mbedtls, libssh2 to fail as well, even in cases where mbedtls is not used.

Besides, it is a good to not have to build both libssh2 and mbedtls, if you're not going to build nmap-full. In the times of python2 and python3, this strategy was more widely used, as the annoyance of having to needlessly build another version of python was far greater.

PS: I will document this as a guideline when writing multiple packages in one Makefile, to always prepend the +PACKAGE_foo:bar when bar is only used in part of that Makefile's packages.

[micmac1]

The problem with adding mbedtls to PKG_BUILD_DEPENDS, and not tacking nmap is that you're always forcing a build of libssh2 and mbedtls even if you don't need them (you're not selecting nmap-full. On its own, adding mbedtls to libssh2 PKG_BUILD_DEPENDS will case, in an eventual failure to build mbedtls, libssh2 to fail as well, even in cases where mbedtls is not used. Besides, it is a good to not have to build both libssh2 and mbedtls, if you're not going to build nmap-full. In the times of python2 and python3, this strategy was more widely used, as the annoyance of having to needlessly build another version of python was far greater.

I agree with you. But fixing libssh2's PKG_BUILD_DEPENDS _as well_ would allow it to be used in other packages PKG_BUILD_DEPENDS, without libssh2 being selected in menuconfig.

[cotequeiroz]

There are no other packages in this situation.

$ grep -E 'libssh2/compile\)' tmp/.packagedeps
$(curdir)/feeds/packages/aria2/compile += $(curdir)/libs/toolchain/compile $(curdir)/libs/zlib/compile $(if $(CONFIG_ARIA2_ASYNC_DNS),$(curdir)/feeds/packages/c-ares/compile) $(if $(CONFIG_ARIA2_COOKIE),$(curdir)/feeds/packages/sqlite3/compile) $(if $(CONFIG_ARIA2_EXPAT),$(curdir)/feeds/packages/expat/compile) $(if $(CONFIG_ARIA2_GMP),$(curdir)/libs/gmp/compile) $(if $(CONFIG_ARIA2_GNUTLS),$(curdir)/feeds/packages/gnutls/compile) $(if $(CONFIG_ARIA2_LIBGCRYPT),$(curdir)/feeds/packages/libgcrypt/compile) $(if $(CONFIG_ARIA2_LIBXML2),$(curdir)/feeds/packages/libxml2/compile) $(if $(CONFIG_ARIA2_NETTLE),$(curdir)/libs/nettle/compile) $(if $(CONFIG_ARIA2_OPENSSL),$(curdir)/libs/openssl/compile) $(if $(CONFIG_ARIA2_SFTP),$(curdir)/feeds/packages/libssh2/compile) $(if $(CONFIG_GCC_LIBSSP),$(curdir)/libs/toolchain/compile) $(if $(CONFIG_USE_GLIBC),$(curdir)/libs/toolchain/compile)
$(curdir)/network/utils/curl/compile += $(curdir)/libs/toolchain/compile $(curdir)/system/ca-certificates/compile $(if $(CONFIG_GCC_LIBSSP),$(curdir)/libs/toolchain/compile) $(if $(CONFIG_LIBCURL_GNUTLS),$(curdir)/feeds/packages/gnutls/compile) $(if $(CONFIG_LIBCURL_LDAP),$(curdir)/feeds/packages/openldap/compile) $(if $(CONFIG_LIBCURL_LIBIDN2),$(curdir)/feeds/packages/libidn2/compile) $(if $(CONFIG_LIBCURL_MBEDTLS),$(curdir)/libs/mbedtls/compile) $(if $(CONFIG_LIBCURL_NGHTTP2),$(curdir)/libs/nghttp2/compile) $(if $(CONFIG_LIBCURL_OPENSSL),$(curdir)/libs/openssl/compile) $(if $(CONFIG_LIBCURL_SSH2),$(curdir)/feeds/packages/libssh2/compile) $(if $(CONFIG_LIBCURL_THREADED_RESOLVER),$(curdir)/libs/toolchain/compile) $(if $(CONFIG_LIBCURL_WOLFSSL),$(curdir)/libs/wolfssl/compile) $(if $(CONFIG_LIBCURL_ZLIB),$(curdir)/libs/zlib/compile) $(if $(CONFIG_USE_GLIBC),$(curdir)/libs/toolchain/compile)
$(curdir)/feeds/packages/mc/compile += $(curdir)/feeds/packages/glib2/compile $(curdir)/libs/ncurses/compile $(curdir)/libs/toolchain/compile $(curdir)/utils/util-linux/compile $(if $(CONFIG_BUILD_NLS),$(curdir)/libs/libiconv-full/compile) $(if $(CONFIG_BUILD_NLS),,$(curdir)/libs/gettext/compile) $(if $(CONFIG_BUILD_NLS),,$(curdir)/libs/libiconv/compile) $(if $(CONFIG_GCC_LIBSSP),$(curdir)/libs/toolchain/compile) $(if $(CONFIG_MC_VFS),$(curdir)/feeds/packages/libssh2/compile) $(if $(CONFIG_MC_VFS),$(curdir)/feeds/packages/libtirpc/compile) $(if $(CONFIG_USE_GLIBC),$(curdir)/libs/toolchain/compile)

• aria2 will build libssh2 only if ARIA2_SFTP is selected; ARIA2_SFTP will select PACKAGE_libssh2
• curl will build libssh2 only if LIBCURL_SSH2 is selected; LIBCURL_SSH2 selects PACKAGE_libssh2
• mc will build libssh2 only if MC_VFS is selected; MC_VFS selects PACKAGE_libssh2.

The problem with nmap is that any package that's part of feeds/packages/nmap/compile will call feeds/packages/libssh2/compile--this is what tmp/.packagedeps is defining. However, only nmap-full is selecting PACKAGE_libssh2. When PACKAGE_libssh2 is built without being selected, no configuration symbols are defined. So both LIBSSH2_MBEDTLS and LIBSSH2_OPENSSL are undefined.
Another, IMO better, option to make it more fail proof is to change the condition to compile libmbedtls, from +LIBSSH2_MBEDTLS:libmbedtls to +!LIBSSH2_OPENSSL:libmbedtls. This way, no unnecessary packages are built.