Switch to OpenSSL 3.0 breaks packages

[multiduplikator]

Describe the bug
After latest update in x64-k3.2, openssl got upgraded to 3.0 branch. At least weechat, rtorrent and curl are throwing error messages:

"error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory"

Seems like they did not get built against the new openssl.

Environment (please complete the following information):

• Entware feed: x64-k3.2
• Device: Qnap TS-470pro
• Firmware version 4.3.6.2232

[zyxmon]

Opkg is too straightforward and simple. Try this - https://github.com/Entware/Entware/wiki#%EF%B8%8F-package-upgrade-notes
If it does not help you can try something like
opkg list-installed|grep -v "libc "|sed 's/ - .*$//'|grep lib|grep -v libpthread|grep -v libgcc|xargs -n 5 opkg --force-reinstall install

[multiduplikator]

Added a ^ before the grep for lib. That worked. Now having issues with some upgrade versions mismatch...Will deal with that. But the issue is resolved

opkg list-installed|grep -v "libc "|sed 's/ - .*$//'|grep ^lib|grep -v libpthread|grep -v libgcc|xargs -n 5 opkg --force-reinstall install

[multiduplikator]

Had to edit the install status file manually and remove libattr and libx264 (in my case). After that, all is back to normal.

[OneCDOnly]

I think I'm getting the same thing with the Deluge packages (deluge & deluge-ui-web).

Here's the Deluge daemon startup:

'/opt/bin/deluged -L info --logfile /share/CACHEDEV2_DATA/.qpkg/Deluge-server/config/Deluge-server.log --config /share/CACHEDEV2_DATA/.qpkg/Deluge-server/config/ --pidfile /var/run/Deluge-server.pid'
OK
file /var/run/Deluge-server.pid: exists
Traceback (most recent call last):
  File "/opt/bin/deluged", line 33, in <module>
    sys.exit(load_entry_point('deluge==2.1.1', 'gui_scripts', 'deluged')())
  File "/opt/lib/python3.10/site-packages/deluge/core/daemon_entry.py", line 87, in start_daemon
    from deluge.core.daemon import is_daemon_running
  File "/opt/lib/python3.10/site-packages/deluge/core/daemon.py", line 19, in <module>
    from deluge.core.core import Core
  File "/opt/lib/python3.10/site-packages/deluge/core/core.py", line 40, in <module>
    from deluge.core.rpcserver import export
  File "/opt/lib/python3.10/site-packages/deluge/core/rpcserver.py", line 28, in <module>
    from deluge.crypto_utils import check_ssl_keys, get_context_factory
  File "/opt/lib/python3.10/site-packages/deluge/crypto_utils.py", line 12, in <module>
    from OpenSSL import crypto
  File "/opt/lib/python3.10/site-packages/OpenSSL/__init__.py", line 8, in <module>
  File "/opt/lib/python3.10/site-packages/OpenSSL/crypto.py", line 17, in <module>
  File "/opt/lib/python3.10/site-packages/OpenSSL/_util.py", line 6, in <module>
  File "/opt/lib/python3.10/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 14, in <module>
ImportError: /opt/lib/python3.10/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so: undefined symbol: FIPS_mode

I've reinstalled all libs as shown earlier in this issue, I've reinstalled python*, I even resorted to reinstalling Entware entirely, but I still get this error with OpenSSL. Is Deluge now broken?

[zyxmon]

It looks like deluge is broken.

[zyxmon]

@OneCDOnly
Please check if the deluge is working. It is better to make full Entware reinstall
Fixed via
Entware/entware-packages@31802a7
and
8da0e37

[OneCDOnly]

@zyxmon yes, that's fixed it. Cheers mate! 👍🏽

[victor-rds]

In my case also broke "wget", which in turn broke opkg:

$ opkg update
Downloading http://bin.entware.net/x64-k3.2/Packages.gz
wget: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
*** Failed to download the package list from http://bin.entware.net/x64-k3.2/Packages.gz

Collected errors:
 * opkg_download: Failed to download http://bin.entware.net/x64-k3.2/Packages.gz, wget returned 127.

Removing the wget package fixes opkg, now install wget-ssl to get wget back

[ryzhovau]

I'll keep this one open for few days, while this topic stays hot.

[gilgrissom]

These apparently break as well...

mariadb-client - 10.9.3-1
mariadb-client-base - 10.4.18-1a
mariadb-client-extra - 10.9.3-1
mariadb-common - 10.2.24-1
mariadb-server - 10.9.3-1
mariadb-server-base - 10.9.3-1
mariadb-server-extra - 10.9.3-1

/opt/bin/mysql: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory

Had to downgrade, but unsure what other collaterals of going with 1.1.1s are now:
opkg install --force-downgrade libopenssl_1.1.1s-1_armv7-2.6.ipk libopenssl-conf_1.1.1s-1_armv7-2.6.ipk openssl-util_1.1.1s-1_armv7-2.6.ipk

python3-openssl - 3.10.7-1 => 3.10.7-3 also breaks all connectivity with mysql DB above

xxx@host:/mnt/xxx# /opt/xxx.py
Traceback (most recent call last):
  File "/opt/xxx.py", line 25, in <module>
    db = pymysql.connect(
  File "/opt/lib/python3.10/site-packages/pymysql/connections.py", line 282, in __init__
NotImplementedError: ssl module not found

[cthu1hoo]

Monit seems to be broken:

# ldd `which monit`
/opt/bin/monit: /usr/lib/libcrypto.so.1.1: version `OPENSSL_1_1_0' not found (required by /opt/bin/monit)
        linux-vdso.so.1 (0x77dcc000)
        libm.so.6 => /opt/lib/libm.so.6 (0x77c92000)
        libz.so.1 => /opt/lib/libz.so.1 (0x77c6d000)
        libpthread.so.0 => /opt/lib/libpthread.so.0 (0x77c3f000)
        libcrypt.so.1 => /opt/lib/libcrypt.so.1 (0x77bfe000)
        libresolv.so.2 => /opt/lib/libresolv.so.2 (0x77bd8000)
        libnsl.so.1 => /opt/lib/libnsl.so.1 (0x77bb0000)
        libssl.so.1.1 => not found
        libcrypto.so.1.1 => /usr/lib/libcrypto.so.1.1 (0x778f3000)
        libgcc_s.so.1 => /opt/lib/libgcc_s.so.1 (0x778cd000)
        libc.so.6 => /opt/lib/libc.so.6 (0x7773a000)
        /opt/lib/ld.so.1 (0x77d9a000)
        libatomic.so.1 => /opt/lib/libatomic.so.1 (0x77724000)
        libssp.so.0 => /opt/lib/libssp.so.0 (0x77712000)
        libc.so => /usr/lib/libc.so (0x7762d000)

EDIT: nevermind, just needed a reinstall.

in the end I had to reinstall everything that showed up here:

find /opt/bin/ /opt/sbin/ -type f | while read F; do ldd $F | grep -q ssl.so.1 && opkg search $F; done

[crkinard]

git is suffering the same issues as well.

[sacc-leo]

copy the two files llibcrypto.so.1.1 and libssl.so.1.1 to /opt/lib can fix this issue temporarily.

[zyxmon]

copy the two files llibcrypto.so.1.1 and libssl.so.1.1 to /opt/lib can fix this issue temporarily.

There are correct fixes in this discussion. Do not use incorrect ones.

[gilgrissom]

copy the two files llibcrypto.so.1.1 and libssl.so.1.1 to /opt/lib can fix this issue temporarily.

There are correct fixes in this discussion. Do not use incorrect ones.

@zyxmon
Hey, Andrey. I see several workarounds in chronology that worked to some extent but not for everyone and not for every issue.

Could you please elaborate more on THE "correct fix"? :)

[zyxmon]

@gilgrissom - what workarounds you have tried? What were your results?

[Weegley]

curl - 7.86.0-1 also broken
cmake - 3.25.2-1 too

-bash-5.2# curl
curl: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory

[original-birdman]

In my case also broke "wget", which in turn broke opkg:

If you run the upgrade as:

PATH=/usr/bin:$PATH opkg upgrade

it will use the system wget, which lets it download the rest of the things.

[Shorrer]

Reinstalling everything seemed to help the issue. At least curl works now, so it is something that is worth a try. As said above -- you're probably better off reinstalling entire Entware, but this is not a solution that's going to work with everyone.

So, maybe someone will find information there useful.
!!! IT IS NOT CHECKED THOROUGHLY !!! This is more of a list of advices and things to know than a proper guide. Only do this if you're sure what you're doing and how it affects your system. Do a backup that you will be able to restore quickly if you need to.

0. Do a backup of your configuration and stuff (and better yet -- dump the whole filesystem somewhere)
1. Generate a package list
   It is better to have this list at hand somewhere in file because if things didn't go the way they should you'd at least have list of installed packages. What seems to be an OK command for me is
   opkg list-installed | awk '!/^(busybox|opkg|lib(c|pthread|gcc)) / { print $1 }'
2. Create package cache
   It is very easy to lose ability to download packages when doing the reinstall. If you have a package cache then all you have to have is busybox (without it opkg will not be able to install anything), and if you didn't remove libc and libgcc it is possible to just copy compatible busybox binary to /opt/bin and create symlinks.
   You can do this by
   cd /path/to/package/cache && opkg list-installed | awk '{ print $1 }' | xargs opkg download && cd -
3. Reinstall the world
   opkg list-installed | awk '!/^(busybox|opkg|lib(c|pthread|gcc)) / { print $1 }' | xargs opkg --cache=/path/to/package/cache install

If things go wrong:

• libgcc and libc are essential for sure. libpthread is too, but I'm not sure. If you don't have them you're on your own -- full reinstall might actually be the best solution at this point if you're in a hurry.
• If you have the above packages then it's recoverable. You basically want to restore opkg, wget and busybox if you've lost them. Hopefully you have made the package cache -- for some reason I can't install libopenssl without it as it always wants to download even when I give OPKG the filename. If not, then you can create an empty directory that you will use as package cache and place *.ipk files in there.
• You might've lost the ca-bundle. This is not a huge problem, but if you have to download anything wget will fail and opkg will report that it failed. --no-check-certificate is a solution there.
• If you've lost the Busybox then OPKG doesn't want to install anything (and it's a pain to lookup directory contents through shell completion). As long as you have essential libraries mentioned above you can grab a compatible busybox binary (e.g. from Entware install archive) and drop it into /opt/bin/ (or wherever you have your Entware installed). Depending on the way you do it you might end up with executable without executable permissions, and if you've lost the Busybox you've probably lost the chmod as well. It is not a problem: try /opt/lib/ld-2.27.so /opt/bin/busybox chmod +x /opt/bin/busybox (your ld might have some other version). It, apparently, ignores the fact that there are no executable permissions on the file. Then it's the case of creating the symlinks:
  cd /opt/bin && /opt/bin/busybox --list | /opt/bin/busybox xargs -n1 /opt/bin/busybox ln -s /opt/bin/busybox && cd -
  After all this you should have a bunch of symlinks to Busybox in /opt/bin/ and a useable system. Don't forget to install Busybox using OPKG so that you would continue to get updates to it (and to do whatever postinstall script might want to do).
• If you've lost OPKG then you should also be able to just copy it.

Hopefully it'll help someone.