BufferedFile readline() does not check for newline characters if buffer size is met or exceeded #428

achapp opened this Issue Nov 4, 2014 · 2 comments


None yet

2 participants

achapp commented Nov 4, 2014

Paramiko version: 1.15
Python version: 2.6.6

I'm running into an issue with the readline() function in BufferedFile. It appears that when the buffer size is met or exceeded, the function will return a buffer size chunk of the file and not attempt to locate newline characters.

Documentation of readline()

    Read one entire line from the file.  A trailing newline character is
    kept in the string (but may be absent when a file ends with an
    incomplete line).  If the size argument is present and non-negative, it
    is a maximum byte count (including the trailing newline) and an
    incomplete line may be returned.

My interpretation of the documentation is that readline() should read up to the next newline character or buffer size bytes, whichever comes first. If readline() is given a file greater than or equal to the buffer size it should always look for a newline character or, failing that, return up to buffer size bytes. If my understanding is correct, then I think there is a problem in the following code.

From BufferedFile readline():

 219     # check size before looking for a linefeed, in case we already have
 220     # enough.
 221     if (size is not None) and (size >= 0):
 222           if len(line) >= size:
 223               # truncate line and return
 224               self._rbuffer = line[size:]
 225               line = line[:size]
 226               self._pos += len(line)
 227               return line if self._flags & self.FLAG_BINARY else u(line)
 228           n = size - len(line)

The problem lies with line 227.

 227               return line if self._flags & self.FLAG_BINARY else u(line)

This exits the function with a buffer size chunk before any search is made for newline characters. It's possible there are lines within that buffer size chunk, but we never look so they are disregarded in this case.

Thanks for taking the time to read this and for your work on Paramiko!

achapp commented Nov 25, 2014

Rolling into PR.

@bitprophet bitprophet closed this Dec 15, 2014
@bitprophet bitprophet added a commit that referenced this issue Dec 18, 2014
@bitprophet bitprophet Changelog closes #428 fc59b7d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment