Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server listdir fails with files saved during Feburary, August, and December #985

Closed
bz2 opened this issue Jun 6, 2017 · 1 comment
Closed

Comments

@bz2
Copy link
Contributor

bz2 commented Jun 6, 2017

From downstream bug lp:1522916, a paramiko server using a locale such as fr_CH.UTF-8 will 'randomly' fail with IOError: Failure when listdir() is used.

To reproduce the server must have a suitable locale installed, by for instance running sudo dpkg-reconfigure locales on a Debian-based system. Then export the locale with the LANG envvar and call setlocale(LC_ALL, '') or similar in the server process. Finally, list a directory with files of suitable vintage. For French, the abbreviated month names 'févr.' 'août' and 'déc.' will trigger the problem.

On the server side, the following traceback is recorded:

Traceback (most recent call last):
  File ".../paramiko/sftp_server.py", line 111, in start_subsystem
    self._process(t, request_number, msg)
  File ".../paramiko/sftp_server.py", line 386, in _process
    self._read_folder(request_number, folder)
  File ".../paramiko/sftp_server.py", line 239, in _read_folder
    msg.add_string(attr)
  File ".../paramiko/message.py", line 273, in add_string
    s = asbytes(s)
  File ".../paramiko/common.py", line 170, in asbytes
    raise Exception('Unknown type')
Exception: Unknown type

Or with code is modified to allow the original exception to propagate:

Traceback (most recent call last):
  File ".../paramiko/sftp_server.py", line 111, in start_subsystem
    self._process(t, request_number, msg)
  File ".../paramiko/sftp_server.py", line 386, in _process
    self._read_folder(request_number, folder)
  File ".../paramiko/sftp_server.py", line 239, in _read_folder
    msg.add_string(attr)
  File ".../paramiko/message.py", line 273, in add_string
    s = asbytes(s)
  File ".../paramiko/common.py", line 168, in asbytes
    s = s.asbytes()
  File ".../paramiko/sftp_attr.py", line 223, in asbytes
    return b(str(self))
  File ".../paramiko/sftp_attr.py", line 220, in __str__
    return '%s   1 %-8d %-8d %8d %-12s %s' % (ks, uid, gid, size, datestr, filename)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xb7 in position 47: ordinal not in range(128)

The solution is to decode the result from time.strftime as the documentation suggests. For Python 3, this has already been done by the standard library so the issue does not appear. Another way to avoid the issue is using a language neutral locale like C.UTF-8.

bz2 added a commit to bz2/paramiko that referenced this issue Jun 6, 2017
Fixes paramiko#985

SFTPAttributes uses the locale-aware %b directive for the
abbrieviated month name with time.strftime, but was not
decoding the result on Python 2.7.

Add a strftime alias in py3compat that will always return
unicode, and resolve the mixing of bytes and text in
SFTPAttributes methods.

Also adds a test at the listdir level, and a more specific
test for the asbytes serialisation of SFTPAttributes.
bz2 added a commit to bz2/paramiko that referenced this issue Jun 6, 2017
Fixes paramiko#985

SFTPAttributes uses the locale-aware %b directive for the
abbreviated month name with time.strftime, but was not
decoding the result on Python 2.7.

Add a strftime alias in py3compat that will always return
unicode, and resolve the mixing of bytes and text in
SFTPAttributes methods.

Add a test at the listdir level, and a more specific test
for the SFTPAttributes asbytes method.
@bitprophet
Copy link
Member

Rolling into PR.

bz2 added a commit to bz2/paramiko that referenced this issue Jun 11, 2017
Fixes paramiko#985

SFTPAttributes uses the locale-aware %b directive for the
abbreviated month name with time.strftime, but was not
decoding the result on Python 2.7.

Add a strftime alias in py3compat that will always return
unicode, and resolve the mixing of bytes and text in
SFTPAttributes methods.

Add a test at the listdir level, and a more specific test
for the SFTPAttributes asbytes method.
bz2 added a commit to bz2/paramiko that referenced this issue Jun 12, 2017
Fixes paramiko#985

SFTPAttributes uses the locale-aware %b directive for the
abbreviated month name with time.strftime, but was not
decoding the result on Python 2.7.

Add a strftime alias in py3compat that will always return
unicode, and resolve the mixing of bytes and text in
SFTPAttributes methods.

Add a test at the listdir level, and a more specific test
for the SFTPAttributes asbytes method.
bitprophet pushed a commit that referenced this issue Nov 29, 2021
Fixes #985

SFTPAttributes uses the locale-aware %b directive for the
abbreviated month name with time.strftime, but was not
decoding the result on Python 2.7.

Add a strftime alias in py3compat that will always return
unicode, and resolve the mixing of bytes and text in
SFTPAttributes methods.

Add a test at the listdir level, and a more specific test
for the SFTPAttributes asbytes method.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants