deluged -d throws ffi.callback() error on M1 mac

[Amar1729]

$ deluged -d

00:15:41 [ERROR   ][deluge.core.daemon_entry      :132 ]
Unable to start deluged: Cannot allocate write+execute memory for ffi.callback().
You might be running on a system that prevents this.
For more information, see https://cffi.readthedocs.io/en/latest/using.html#callbacks

Since M1 Macs have stricter codesigning requirements than Intel Macs, this issue seems to only be present on M1. Also, it does NOT happen when simply running a local deluged binary (e.g. inside a python virtual environment from venv/bin/pip install deluge) since (i think?) that binary would be linker-signed. Since this is a homebrew-sandbox-built binary it's different ...?

The cffi docs note that on Mac OS X, applications that use ffi.callback() must use the entitlement com.apple.security.cs.allow-unsigned-executable-memory.

I've tried something along the lines of the steps listed in this blog post:

Save into entitlements.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
</dict>
</plist>

Run adhoc codesign and then check attributes with -d to ensure they're there:

$ codesign --force -s - deluged --entitlements entitlements.plist -vvvvvv

deluged: signed generic [deluged-4da4dc2480ab686d2d76406c065a0f26e4d08446]

$  codesign -dv -r- --entitlements - deluged

Executable=/Users/amar1729/Downloads/deluge/deluged
Identifier=deluged-4da4dc2480ab686d2d76406c065a0f26e4d08446
Format=generic
CodeDirectory v=20100 size=289 flags=0x2(adhoc) hashes=1+5 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
host => identifier "com.apple.bash" and anchor apple
# designated => cdhash H"53fc2c0675f51fefa89a47f2c5df1dcf02d0324d" or cdhash H"6b06683631d1634d2640ec75d9ff0a352ec426c5"
[Dict]
        [Key] com.apple.security.cs.allow-unsigned-executable-memory
        [Value]
                [Bool] true

Regardless of codesign telling me this binary is signed with the proper entitlement, running it still throws back the same unable to start error as above (???).

[Amar1729]

One complexity that should have been obvious is that the final brew bin/deluged isn't a binary, it's an executable script that execs libexec/bin/deluged - which is also an executable script:

#!/opt/homebrew/Cellar/deluge-meta/2.0.5/libexec/bin/python3.9
# -*- coding: utf-8 -*-
import re
import sys
from deluge.core.daemon_entry import start_daemon
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(start_daemon())

This keg's libexec is a python virtual environment (built by the formula defintion) so it is odd to me that this call can throw the codesigning error whereas installing deluge inside a user-created virtual environment and running it from there works just fine. I've tried codesigning a bunch of things in the keg:

fd -t x -HI -x \
    sudo codesign -fs - --entitlements ~/entitlements.plist {} -vvvvvv

fd -e py -HI -x \
    sudo codesign -fs - --entitlements ~/entitlements.plist {} -vvvvvv

But neither lets deluged run unhindered.

[Amar1729]

(Pdb) n
> /opt/homebrew/Cellar/deluge-meta/2.0.5/libexec/lib/python3.9/site-packages/OpenSSL/SSL.py(1128)set_verify()
-> self._verify_helper = _VerifyHelper(callback)
(Pdb) where
  /opt/homebrew/Cellar/deluge-meta/2.0.5/libexec/bin/deluged(8)<module>()
-> sys.exit(start_daemon())
  /opt/homebrew/Cellar/deluge-meta/2.0.5/libexec/lib/python3.9/site-packages/deluge/core/daemon_entry.py(146)start_daemon()
-> return run_profiled(
  /opt/homebrew/Cellar/deluge-meta/2.0.5/libexec/lib/python3.9/site-packages/deluge/common.py(1351)run_profiled()
-> return func(*args)
  /opt/homebrew/Cellar/deluge-meta/2.0.5/libexec/lib/python3.9/site-packages/deluge/core/daemon_entry.py(115)run_daemon()
-> daemon = Daemon(
  /opt/homebrew/Cellar/deluge-meta/2.0.5/libexec/lib/python3.9/site-packages/deluge/core/daemon.py(130)__init__()
-> self.rpcserver = RPCServer(
  /opt/homebrew/Cellar/deluge-meta/2.0.5/libexec/lib/python3.9/site-packages/deluge/core/rpcserver.py(412)__init__()
-> port, self.factory, get_context_factory(cert, pkey), interface=hostname
  /opt/homebrew/Cellar/deluge-meta/2.0.5/libexec/lib/python3.9/site-packages/deluge/crypto_utils.py(75)get_context_factory()
-> ctx = cert_options.getContext()
  /opt/homebrew/Cellar/deluge-meta/2.0.5/libexec/lib/python3.9/site-packages/twisted/internet/_sslverify.py(1632)getContext()
-> self._context = self._makeContext()
  /opt/homebrew/Cellar/deluge-meta/2.0.5/libexec/lib/python3.9/site-packages/twisted/internet/_sslverify.py(1663)_makeContext()
-> ctx.set_verify(verifyFlags, _verifyCallback)
> /opt/homebrew/Cellar/deluge-meta/2.0.5/libexec/lib/python3.9/site-packages/OpenSSL/SSL.py(1128)set_verify()
-> self._verify_helper = _VerifyHelper(callback)
(Pdb) n
MemoryError: Cannot allocate write+execute memory for ffi.callback(). You might be running on a system that prevents this. For more information, see https://cffi.readthedocs.io/en/latest/using.html#callbacks
> /opt/homebrew/Cellar/deluge-meta/2.0.5/libexec/lib/python3.9/site-packages/OpenSSL/SSL.py(1128)set_verify()
-> self._verify_helper = _VerifyHelper(callback)

Traceback found with pdb. Issue is in pyopenssl which deluge is still using. pyca/pyopenssl#873

[Amar1729]

related PR does fix this issue, as it bumps cffi from 1.15.1 to 1.16, which (as mentioned in the linked pyopenssl issue) includes the bugfix for this issue.