Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load data local in file fails with AttributeError: 'NoneType' object has no attribute 'settimeout' on connection reset #989

Closed
bruceduhamel opened this issue Jul 12, 2021 · 1 comment · Fixed by #1116
Milestone

Comments

@bruceduhamel
Copy link

Describe the bug

When loading data into a MariaDB database server using the LOAD DATA LOCAL INFILE syntax, if the database server resets the connection with a ConnectionResetError, the PyMySQL client will re-raise it as an OperationalError, and also attempt to send an empty packet to let the server know we're done sending data. When sending that empty packet, PyMySQL encounters the following exception: AttributeError: 'NoneType' object has no attribute 'settimeout'.

Here's the relevant code from connections.py

class LoadLocalFile:
    def __init__(self, filename, connection):
        self.filename = filename
        self.connection = connection

    def send_data(self):
        """Send data packets from the local file to the server"""
        if not self.connection._sock:
            raise err.InterfaceError(0, "")
        conn = self.connection

        try:
            with open(self.filename, "rb") as open_file:
                packet_size = min(
                    conn.max_allowed_packet, 16 * 1024
                )  # 16KB is efficient enough
                while True:
                    chunk = open_file.read(packet_size)
                    if not chunk:
                        break
                    conn.write_packet(chunk)  # DEBUG: raises ConnectionResetError, force closes connection, sets socket to None
        except IOError:
            raise err.OperationalError(1017, f"Can't find file '{self.filename}'")
        finally:
            # send the empty packet to signify we are done sending data
            conn.write_packet(b"")  # DEBUG: raises AttributeError, attempts to call socket.settimeout on a None reference

I added comments in-line prefaced with DEBUG to help explain the situation.

Here's a snippet where we can see the connection is force closed on a ConnectionResetError (which inherits ConnectionError, which is an alias for IOError)

    def _write_bytes(self, data):
        self._sock.settimeout(self._write_timeout)
        try:
            self._sock.sendall(data)
        except IOError as e:
            self._force_close()
            raise err.OperationalError(
                CR.CR_SERVER_GONE_ERROR, "MySQL server has gone away (%r)" % (e,)
            )

Here's force close

    def _force_close(self):
        """Close connection without QUIT message"""
        if self._sock:
            try:
                self._sock.close()
            except:  # noqa
                pass
        self._sock = None 
        self._rfile = None

To Reproduce
I'll put this together soon.

Schema:
I'll put this together soon.

Code:
I'll put this together soon.

Expected behavior
Instead of an AttributeError, an OperationalError should be raised. This allows clients to attempt to retry the failure.

Environment

  • OS: Ubuntu 18.04 w/ Python 3.8.11
  • Server and version: MariaDB 10.3.23 on AWS RDS
  • PyMySQL version: 1.0.2

Additional context

Traceback (most recent call last):
  File "/root/foreman/venv/lib/python3.8/site-packages/pymysql/connections.py", line 756, in _write_bytes
    self._sock.sendall(data)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/foreman/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1362, in send_data
    conn.write_packet(chunk)
  File "/root/foreman/venv/lib/python3.8/site-packages/pymysql/connections.py", line 680, in write_packet
    self._write_bytes(data)
  File "/root/foreman/venv/lib/python3.8/site-packages/pymysql/connections.py", line 759, in _write_bytes
    raise err.OperationalError(
pymysql.err.OperationalError: (2006, "MySQL server has gone away (ConnectionResetError(104, 'Connection reset by peer'))")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/foreman/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1209, in _read_load_local_packet
    sender.send_data()
  File "/root/foreman/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1367, in send_data
    conn.write_packet(b"")
  File "/root/foreman/venv/lib/python3.8/site-packages/pymysql/connections.py", line 680, in write_packet
    self._write_bytes(data)
  File "/root/foreman/venv/lib/python3.8/site-packages/pymysql/connections.py", line 754, in _write_bytes
    self._sock.settimeout(self._write_timeout)
AttributeError: 'NoneType' object has no attribute 'settimeout'
@aigr
Copy link

aigr commented Oct 29, 2021

as an example, i think the same behaviour can be seen in https://github.com/transferwise/pipelinewise/issues/503

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants