Writing to FTP fails with error "503 ASCII (Text) data type is not supported for file transfer operations. Please configure your FTP client to use IMAGE (Binary) type and try again"

[beck3905]

Problem description

I am trying to write to FTP using smart_open. However, when I do I receive an error that states"

503 ASCII (Text) data type is not supported for file transfer operations. Please configure your FTP client to use IMAGE (Binary) type and try again

I get the above error regardless of whether I use mode "w" or "wb".

I also noticed in the source code https://github.com/RaRe-Technologies/smart_open/blob/develop/smart_open/ftp.py#L102 that the default mode is "r" even though the docstring states that the mode must be "rb" or "wb".

Steps/code to reproduce the problem

import smart_open

with smart_open.open("ftp://<username>:<password>@<ftp-server>/my-file.txt", mode="wb") as fp:
    fp.write(b"this is a test")

Tracback

error_perm                                Traceback (most recent call last)
Cell In[33], line 1
----> 1 with smart_open.open("ftp://<username>:<password>@<server-url>/test", mode="wb") as fp:
      2     fp.write(b"this is a test")

File python3.9/site-packages/smart_open/smart_open_lib.py:224, in open(uri, mode, buffering, encoding, errors, newline, closefd, opener, compression, transport_params)
    221 except ValueError as ve:
    222     raise NotImplementedError(ve.args[0])
--> 224 binary = _open_binary_stream(uri, binary_mode, transport_params)
    225 decompressed = so_compression.compression_wrapper(binary, binary_mode, compression)
    227 if 'b' not in mode or explicit_encoding is not None:

File python3.9/site-packages/smart_open/smart_open_lib.py:400, in _open_binary_stream(uri, mode, transport_params)
    398 scheme = _sniff_scheme(uri)
    399 submodule = transport.get_transport(scheme)
--> 400 fobj = submodule.open_uri(uri, mode, transport_params)
    401 if not hasattr(fobj, 'name'):
    402     fobj.name = uri

File python3.9/site-packages/smart_open/ftp.py:58, in open_uri(uri, mode, transport_params)
     56 scheme = parsed_uri.pop("scheme")
     57 secure_conn = True if scheme == "ftps" else False
---> 58 return open(uri_path, mode, secure_connection=secure_conn,
     59         transport_params=transport_params, **parsed_uri)
...
    253 if c == '5':
--> 254     raise error_perm(resp)
    255 raise error_proto(resp)

error_perm: 503 ASCII (Text) data type is not supported for file transfer operations. Please configure your FTP client to use IMAGE (Binary) type and try again.

Versions

Please provide the output of:

macOS-13.5-arm64-arm-64bit
Python 3.9.16 (main, Mar 13 2023, 13:10:31) 
[Clang 14.0.0 (clang-1400.0.29.202)]
smart_open 6.3.0

Possible Solution

I looked at the source code for both smart_open.ftp and the ftplib module in Python and noticed something in the storbinary method that isn't used in smart_open.ftp.open.

The following is the implementation of ftplib.FTP.storbinary:

self.voidcmd('TYPE I')
with self.transfercmd(cmd, rest) as conn:
    while 1:
        buf = fp.read(blocksize)
        if not buf:
            break
        conn.sendall(buf)
        if callback:
            callback(buf)
    # shutdown ssl layer
    if _SSLSocket is not None and isinstance(conn, _SSLSocket):
        conn.unwrap()
return self.voidresp()

Notice the self.voidcmd('TYPE I') before calling self.transfercmd.

smart_open does not use the TYPE I command. https://github.com/RaRe-Technologies/smart_open/blob/develop/smart_open/ftp.py#L149.

I ran a test where I updated my local smart_open.ftp.py module on line 149 to

conn.voidcmd('TYPE I')
socket = conn.transfercmd(f"{ftp_mode} {path}")

and it successfully wrote to my FTP server.

Would you mind if I submit a PR with this fix and an update of the default mode to "rb" instead of "r"?

[beck3905]

PR submitted: #781