Avoid overfilling buffer when reading from Azure #767
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ronreiter
changed the title
ronreiter
changed the title
mpenkov
approved these changes
Sep 6, 2023
mpenkov
changed the title
|
Thank you @ronreiter ! |
beck3905
pushed a commit
to beck3905/smart_open
that referenced
this pull request
Sep 6, 2023
mpenkov
pushed a commit
that referenced
this pull request
Sep 7, 2023
* fix: ignore seek requests to the current position * fix: adjust test to match new seek behavior * run seek if it is the first time * Add required import for example to work (#756) If a person were to simply copy this code block it would use the built in `open` and would not work. Adding in the correct import makes this block a bit easier for a simple copy paste. * run tests on py3.11 (#774) * add type command to ftp (#781) * Add python 3.11 to setup.py (#775) * Fixes KeyError when retrieving empty but existing object from S3 (#771) * fix: Fixes KeyError when retrieving empty file from S3 * Add test * bugfix: when read size > chunk size, return read size and not chunk size (#767) * undo formatting * fix whitespace * undo formatting --------- Co-authored-by: Rusty Conover <rusty@conover.me> Co-authored-by: Christian Jensen <christian@orbik.com> Co-authored-by: tooptoop4 <33283496+tooptoop4@users.noreply.github.com> Co-authored-by: Raphaël Cohen <raphael.cohen.utt@gmail.com> Co-authored-by: Ron Reiter <ron.reiter@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title
when read size > chunk size, return read size and not chunk size.
Motivation
The read function expects to return the maximum size it can read from a stream and not the chunk size. This causes serious bugs when reading with a parameter such as x.read(MAX_READ_SIZE), which does not expect you to re-read the buffer until you get to MAX_READ_SIZE.
This bug does not happen when the read size is not specified.
If you're fixing a bug, link to the issue number like so:
If you're adding a new feature, then consider opening a ticket and discussing it with the maintainers before you actually do the hard work.
Tests
If you're fixing a bug, consider test-driven development:
If you're implementing a new feature, include unit tests for it.
Make sure all existing unit tests pass.
You can run them locally using:
If there are any failures, please fix them before creating the PR (or mark it as WIP, see below).
Work in progress
If you're still working on your PR, include "WIP" in the title.
We'll skip reviewing it for the time being.
Once you're ready to review, remove the "WIP" from the title, and ping one of the maintainers (e.g. mpenkov).
Checklist
Before you create the PR, please make sure you have:
Workflow
Please avoid rebasing and force-pushing to the branch of the PR once a review is in progress.
Rebasing can make your commits look a bit cleaner, but it also makes life more difficult from the reviewer, because they are no longer able to distinguish between code that has already been reviewed, and unreviewed code.