New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BufferedFile readline() does not check for newline characters if buffer size is met or exceeded #428

Closed
achapp opened this Issue Nov 4, 2014 · 2 comments

Comments

Projects
None yet
2 participants
@achapp
Contributor

achapp commented Nov 4, 2014

Paramiko version: 1.15
Python version: 2.6.6

I'm running into an issue with the readline() function in BufferedFile. It appears that when the buffer size is met or exceeded, the function will return a buffer size chunk of the file and not attempt to locate newline characters.

Documentation of readline()

    Read one entire line from the file.  A trailing newline character is
    kept in the string (but may be absent when a file ends with an
    incomplete line).  If the size argument is present and non-negative, it
    is a maximum byte count (including the trailing newline) and an
    incomplete line may be returned.

My interpretation of the documentation is that readline() should read up to the next newline character or buffer size bytes, whichever comes first. If readline() is given a file greater than or equal to the buffer size it should always look for a newline character or, failing that, return up to buffer size bytes. If my understanding is correct, then I think there is a problem in the following code.

From BufferedFile readline():

 219     # check size before looking for a linefeed, in case we already have
 220     # enough.
 221     if (size is not None) and (size >= 0):
 222           if len(line) >= size:
 223               # truncate line and return
 224               self._rbuffer = line[size:]
 225               line = line[:size]
 226               self._pos += len(line)
 227               return line if self._flags & self.FLAG_BINARY else u(line)
 228           n = size - len(line)

The problem lies with line 227.

 227               return line if self._flags & self.FLAG_BINARY else u(line)

This exits the function with a buffer size chunk before any search is made for newline characters. It's possible there are lines within that buffer size chunk, but we never look so they are disregarded in this case.

Thanks for taking the time to read this and for your work on Paramiko!

@achapp

This comment has been minimized.

Contributor

achapp commented Nov 25, 2014

@bitprophet

This comment has been minimized.

Member

bitprophet commented Dec 15, 2014

Rolling into PR.

@bitprophet bitprophet closed this Dec 15, 2014

bitprophet added a commit that referenced this issue Dec 18, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment