Fix the issue #109 #110
Merged
Fix the issue #109 #110
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Mar 17, 2017
tmylk
pushed a commit
that referenced
this pull request
Mar 27, 2017
…and #110. (#112) * Better support for custom S3 servers. This patch adds support for custom S3 servers in the connection string. It also adds explicit support for setting the server port, and whether or not to use SSL, both as paramaters to the smart_open function as well as within the connection string. These changes are neccessary to be able to connect to s3proxy and other custom s3 servers which don't run on the default port, or neccessarily use SSL. * Fix unit tests * updated README.rst with new s3 mode. * Added a new unit test for the unsecured calling form * Updated style and unit test. * Check that the port argument isnt normally passed. * Add generic HTTP and HTTPS streaming support. Adds support for opening vanilla HTTP and HTTPS addresses. Supports efficient streaming, gzip and bz2 compression, as well as Kerberos and username/password (basic) http authentication. * removed previous merge artifact; * Raise exception instead of returning it :/ * Raise http exceptions properly * neccessary import * python 3 compatibility * Reverted make_closing -> closing We still want to maintain Python 2.6 compatibility, so don't rely on contextlib.closing. * Refactor the code to get the Python version * Refactored the GZfile and BZ2File compression wrappers. * Refactored HttpOpenRead unit tests. Now they don't require internet access, and will test for Basic authentication in the HTTP header. * Clean up http unit tests. http => https, and remove old versions of the tests. * Cosmetic changes and doc updates. * Re-use the open filehandle rather than open a new one. This allows one to use any filehandle-like object instead of just local posix. It also avoids unneccessary filesystem syscalls. * merge artifact * Add unit tests for compressed httpd reads. This breaks out the http tests into their own test class. Also fixed a few behaviors in the HttpReader uncovered by the new tests (yay). * fixed import for python3 * removed stray import * Handle some python3 byte vs unicode incompatibilityes. Works now on Python 2 as well as Python 3.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add tests to read/write compresed format.
Fix issue 109 to make tests pass.
Left for another PR:
Add tests for zipped http and https format