[Core] Implement SSL peers support #432

rcarpa · 2023-09-05T12:29:00Z

This feature is interesting when multiple deluge instances are managed by the same administrator who uses it to transfer private data across a non-secure network.

A separate port has to be allocated for incoming SSL connections from peers. Libtorrent already supports this. It's enough to add the suffix 's' when configuring libtorrent's listen_interfaces. Implement a way to activate listening on an SSL port via the configuration.

To actually allow SSL connection between peers, one has to also configure a x509 certificate, private_key and diffie-hellman for each affected torrent. This is achieved by calling libtorrent's handle->set_ssl_certificate. The certificates are only kept in-memory, so they have to be explicitly re-added after each restart. Implement two ways to set these certificates:

either by putting them in a directory with predefined names and letting deluge set them when receiving the corresponding alert;
or by using a core api call.

rcarpa · 2023-09-05T12:32:56Z

Some core-functions added in this PR rely on a pending PR in libtorrent to expose the needed function in the python bindings: arvidn/libtorrent#7509 . So will be available in the next libtorrent release the earliest. Is there any convention on how to handle cases of requiring a specific libtorrent version for functions which are not required for normal deluge operation, but may be useful for some users?

cas--

This should be a useful addition, I've added comments for changes and happy to hear your thoughts on how you expect the feature to be used.

If possible it would be quite nice to have some tests.

cas-- · 2023-09-18T14:07:18Z

deluge/core/torrentmanager.py

+                    certificate_path, private_key_path, dh_params_path
+                )
+            except RuntimeError:
+                log.debug(


This should be log.error and if the exception message is useful then it should also be included.

cas-- · 2023-09-18T14:08:44Z

deluge/core/core.py

@@ -675,6 +675,43 @@ def connect_peer(self, torrent_id: str, ip: str, port: int):
        if not self.torrentmanager[torrent_id].connect_peer(ip, port):
            log.warning('Error adding peer %s:%s to %s', ip, port, torrent_id)

+    @export
+    def set_ssl_certificate(


I'm not sure about having this as an exported method since it requires prior knowledge of the server setup and to me indicates this should be just an internal implementation.

We should also avoid exposing libtorrent implementation details and instead look at what the end-user needs for exported methods. This also applies to naming, so this method it should really be named set_ssl_torrent_cert.

I would consider the simplest option to have one core method accept cert, private key and DH param which are stored to the default SSL cert location for the daemon.

If really need the cert storage could be controlled by an optional parameter if the certs only want to be retained in memory but would that even be a use-case??

cas-- · 2023-09-18T16:17:19Z

deluge/core/preferencesmanager.py

+    'ssl_peers': {
+        'enabled': False,
+        'random_port': True,
+        'listen_ports': [6892, 6896],
+        'listen_random_port': None,
+        'certificate_location': os.path.join(
+            deluge.configmanager.get_config_dir(), 'ssl_peers_certificates'
+        ),
+    },


For naming of this feature we should use the more common term 'SSL Torrents' that end-users are more likely to know and use. I would also prefer that we keep with the existing layout for config, partially since our validator doesn't parse nested dicts.

For the random port option we could simplify it so that the SSL random port is listen_random_port + 1.

This narrows our set to just three config options:

Suggested change

'ssl_peers': {

'enabled': False,

'random_port': True,

'listen_ports': [6892, 6896],

'listen_random_port': None,

'certificate_location': os.path.join(

deluge.configmanager.get_config_dir(), 'ssl_peers_certificates'

),

},

'ssl_torrents': False,

'ssl_listen_ports': [6892, 6896],

'ssl_torrents_certs': os.path.join(

deluge.configmanager.get_config_dir(), 'ssl_torrents_certs'

),

},

cas-- · 2023-09-18T17:58:40Z

deluge/core/torrent.py

+                certificate_path, private_key_path, dh_params_path, password
+            )
+        except RuntimeError as ex:
+            log.debug('Unable to set ssl certificate from file: %s', ex)


Should be log.error

cas-- · 2023-09-18T18:10:43Z

deluge/core/core.py

+    @export
+    def get_ssl_listen_port(self) -> int:
+        """Returns the active SSL listen port"""
+        return self.session.ssl_listen_port()


This is a tricky one since we are both adding a new RPC method and relying on a new libtorrent exposed method...

Sometimes we check for specific libtorrent version but it likely is easier in this case to catch the AttributeError and log a warning. For the clients we could either re-raise the exception with a nice message or return an invalid port number, not sure which would be better long-term

Suggested change

return self.session.ssl_listen_port()

try:

return self.session.ssl_listen_port()

except AttributeError:

log.warning("libtorrent version >=2.x required to get active SSL listen port")

return -1

cas-- · 2023-09-18T18:39:34Z

deluge/core/torrentmanager.py

+        certificate_path = None
+        private_key_path = None
+        dh_params_path = None
+        for file_name in [torrent_id + '.dh', 'default.dh']:
+            params_path = os.path.join(base_path, file_name)
+            if os.path.isfile(params_path):
+                dh_params_path = params_path
+                break
+        if dh_params_path:
+            for file_name in [torrent_id, 'default']:
+                crt_path = os.path.join(base_path, file_name)
+                key_path = crt_path + '.key'
+                if os.path.isfile(crt_path) and os.path.isfile(key_path):
+                    certificate_path = crt_path
+                    private_key_path = private_key_path
+                    break


Is there a consensus on file naming convention here? I notice that the cert file is missing an extension which doesn't seem quite right.

I feel that the files should end in .pem to indicate the encoding. The libtorrent tests use:

dhparams.pem

<torrent_id>.pem

<torrent_id>_key.pem

Also these default files will need documented.

I wonder if we can make the code cleaner with pathlib and for else:

Suggested change

certificate_path = None

private_key_path = None

dh_params_path = None

for file_name in [torrent_id + '.dh', 'default.dh']:

params_path = os.path.join(base_path, file_name)

if os.path.isfile(params_path):

dh_params_path = params_path

break

if dh_params_path:

for file_name in [torrent_id, 'default']:

crt_path = os.path.join(base_path, file_name)

key_path = crt_path + '.key'

if os.path.isfile(crt_path) and os.path.isfile(key_path):

certificate_path = crt_path

private_key_path = private_key_path

break

certs_dir = Path(self.config['ssl_torrents_certs'])

dh_params_file = certs_dir / f'{torrent_id}.dh'

if not dh_params.is_file():

dh_params_file = certs_dir / 'dhparams.pem'

for filename in [torrent_id, 'default']:

cert_file = certs_dir / f'{filename}.pem'

priv_key_file = certs_dir / f'{filename}_key.pem'

if dh_params_file.is_file() and cert_file.is_file() and priv_key_file.is_file():

break

else:

log.error('<Certs not found...>')

return

In fact this feels like reusable code for a common helper function

rcarpa · 2023-11-02T17:48:16Z

Thank you for the review. I updated the pull request. I slightly deviated from the naming convention you proposed for the files on the disk, because it seemed strange to have a specific naming for dh_params.pem, while calling it <torrent_id>.dh.pem for per-torrent configuration. Files are now automatically saved on disk when set via RPC. I also added a test which creates a seeder and a leecher instance and performs an actual transfer between the 2 using SSL torrents. I did some minor adjustments to facilitate adding this test case. I also rebased on top of #430 which allows to slightly simplify the test case (create and return the torrent on the seeder in one call) + avoids merge conflicts because both PRs change the signature of create_torrent.

This feature is interesting when multiple deluge instances are managed by the same administrator who uses it to transfer private data across a non-secure network. A separate port has to be allocated for incoming SSL connections from peers. Libtorrent already supports this. It's enough to add the suffix 's' when configuring libtorrent's listen_interfaces. Implement a way to activate listening on an SSL port via the configuration. To actually allow SSL connection between peers, one has to also configure a x509 certificate, private_key and diffie-hellman for each affected torrent. This is achieved by calling libtorrent's handle->set_ssl_certificate. Add a new exported method to perform this goal. By default, this method will persist certificates on disk. Allowing them to be re-loaded automatically on restart. Cleanup the certificates of a torrent when it is removed.

rcarpa · 2023-11-20T10:27:30Z

Just rebased. No functional changes in last force push.

cas-- · 2023-11-27T16:01:06Z

Thanks for all the changes and tests! I'll try to get to this PR next, oh and don't worry about rebasing

rcarpa force-pushed the ssl_peers branch from 3b92fac to 9acee05 Compare September 5, 2023 12:41

cas-- requested changes Sep 18, 2023

View reviewed changes

rcarpa force-pushed the ssl_peers branch from 9acee05 to eb035b9 Compare November 2, 2023 17:46

rcarpa force-pushed the ssl_peers branch from eb035b9 to 51f54d7 Compare November 3, 2023 15:11

rcarpa force-pushed the ssl_peers branch from 51f54d7 to 5d176b9 Compare November 20, 2023 10:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Core] Implement SSL peers support #432

[Core] Implement SSL peers support #432

rcarpa commented Sep 5, 2023

rcarpa commented Sep 5, 2023 •

edited

cas-- left a comment •

edited

cas-- Sep 18, 2023

cas-- Sep 18, 2023

cas-- Sep 18, 2023

cas-- Sep 18, 2023

cas-- Sep 18, 2023

cas-- Sep 18, 2023 •

edited

rcarpa commented Nov 2, 2023

rcarpa commented Nov 20, 2023

cas-- commented Nov 27, 2023

-        return self.session.ssl_listen_port()
+        try:
+            return self.session.ssl_listen_port()
+       except AttributeError:
+            log.warning("libtorrent version >=2.x required to get active SSL listen port")
+            return -1

[Core] Implement SSL peers support #432

Are you sure you want to change the base?

[Core] Implement SSL peers support #432

Conversation

rcarpa commented Sep 5, 2023

rcarpa commented Sep 5, 2023 • edited

cas-- left a comment • edited

Choose a reason for hiding this comment

cas-- Sep 18, 2023

Choose a reason for hiding this comment

cas-- Sep 18, 2023

Choose a reason for hiding this comment

cas-- Sep 18, 2023

Choose a reason for hiding this comment

cas-- Sep 18, 2023

Choose a reason for hiding this comment

cas-- Sep 18, 2023

Choose a reason for hiding this comment

cas-- Sep 18, 2023 • edited

Choose a reason for hiding this comment

rcarpa commented Nov 2, 2023

rcarpa commented Nov 20, 2023

cas-- commented Nov 27, 2023

rcarpa commented Sep 5, 2023 •

edited

cas-- left a comment •

edited

cas-- Sep 18, 2023 •

edited