urllib ResourceWarning in case of usage of FTP

[icegood]

BPO	42076

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-10-18.23:00:52.403>
labels = ['3.8', 'library', '3.9', '3.10', 'performance']
title = 'urllib ResourceWarning in case of usage of FTP'
updated_at = <Date 2020-10-18.23:00:52.403>
user = 'https://bugs.python.org/icegood'

bugs.python.org fields:

activity = <Date 2020-10-18.23:00:52.403>
actor = 'icegood'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-10-18.23:00:52.403>
creator = 'icegood'
dependencies = []
files = []
hgrepos = []
issue_num = 42076
keywords = []
message_count = 1.0
messages = ['378911']
nosy_count = 1.0
nosy_names = ['icegood']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue42076'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[icegood]

In case when FTP url is successfully connected via default FTPHandler in FTPHandler.connect_ftp then release of this stuff becomes total responsibility of user i.e. socket remains open, which leads to ResourceWarning in the garbage collector, in case if the user does nothing.

Something like this solves issue in user area:

class SafeFTPHandler(urllib.request.FTPHandler):
          ftp_object = None

          def __init__(self):
            super().__init__()

          def connect_ftp(self, user, passwd, host, port, dirs, timeout):
              self.ftp_object = super().connect_ftp(user, passwd, host, port, dirs, timeout)
              return self.ftp_object

          def ftp_response(self, req, response):
              self._close_ftp()
              return response

          def ftp_open(self, req):
            try:
              return super().ftp_open(req)
            except:
              self._close_ftp()
              raise

          def _close_ftp(self):
            if self.ftp_object:
                self.ftp_object.close()

and further usage in OpenerDirector...

In general case FTPHandler should be able to close immediately (in case if it raises as well) while re-using of the socket should be implemented only within CacheFTPHandler

[iritkatriel]

@icegood Why are you calling connect_ftp rather than ftp_open?

[iritkatriel]

Closing as the problem is not clear and there was no response from the OP to followup questions. We can reopen if there is an actual issue here aferall.