Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Google Cloud Storage (GCS) #404

Merged
merged 87 commits into from
Jan 24, 2020
Merged

Google Cloud Storage (GCS) #404

merged 87 commits into from
Jan 24, 2020

Commits on Nov 11, 2019

  1. GCS support working

    Peter Dannemann committed Nov 11, 2019
    Configuration menu
    Copy the full SHA
    6eeb5b4 View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2019

  1. Start updating setup.py and README

    Peter Dannemann committed Nov 12, 2019
    Configuration menu
    Copy the full SHA
    45e1019 View commit details
    Browse the repository at this point in the history

  2. Add .idea/ to .gitignore

    Peter Dannemann committed Nov 12, 2019
    Configuration menu
    Copy the full SHA
    ae6c252 View commit details
    Browse the repository at this point in the history

  3. Move integration tests to their proper location

    Peter Dannemann committed Nov 12, 2019
    Configuration menu
    Copy the full SHA
    38905ae View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2019

  1. All unit tests passing with mocks

    Peter Dannemann committed Nov 15, 2019
    Configuration menu
    Copy the full SHA
    d054a87 View commit details
    Browse the repository at this point in the history

  2. Add gcs integration tests

    Peter Dannemann committed Nov 15, 2019
    Configuration menu
    Copy the full SHA
    a5e2a6e View commit details
    Browse the repository at this point in the history

Commits on Nov 19, 2019

  1. Add .env to .gitignore

    Peter Dannemann committed Nov 19, 2019
    Configuration menu
    Copy the full SHA
    4fc43fa View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2020

  1. Integration tests passing

    Peter Dannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    67feb7b View commit details
    Browse the repository at this point in the history

  2. Remove kms integration test

    Peter Dannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    62252f0 View commit details
    Browse the repository at this point in the history

  3. Fix failing test

    Peter Dannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    40cbb7d View commit details
    Browse the repository at this point in the history

  4. Change writer from buffer size to min part size

    Peter Dannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    5769e9b View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' into gcs

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    056abe9 View commit details
    Browse the repository at this point in the history

  6. Fix typo

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    c31f79d View commit details
    Browse the repository at this point in the history

  7. Change buffer size to min part size

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    e2be283 View commit details
    Browse the repository at this point in the history

  8. Fix another missed min_part_size

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    0eca1d3 View commit details
    Browse the repository at this point in the history

  9. Fix broken example

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    9357d48 View commit details
    Browse the repository at this point in the history

  10. Change smart_open to open

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    eb2817b View commit details
    Browse the repository at this point in the history

  11. Make WHENCE_CHOICES a tuple

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    00f7c86 View commit details
    Browse the repository at this point in the history

  12. Specify client type

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    2544aca View commit details
    Browse the repository at this point in the history

  13. Make init one line instead of multiple

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    d3e0d8c View commit details
    Browse the repository at this point in the history

  14. Shorten _parse_uri_gcs

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    b2f38cf View commit details
    Browse the repository at this point in the history

  15. Use open instead of smart_open

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    0afb007 View commit details
    Browse the repository at this point in the history

  16. Fix memory issue with RawReader

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    5b95e87 View commit details
    Browse the repository at this point in the history

  17. Fix type hints

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    81f50ae View commit details
    Browse the repository at this point in the history

  18. Fix exists in test

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    2f8f22c View commit details
    Browse the repository at this point in the history

  19. Fix typo in smart_open_lib about schemes

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    7e07484 View commit details
    Browse the repository at this point in the history

  20. Remove RawReaders and BufferedInputBase

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    b779ccd View commit details
    Browse the repository at this point in the history

  21. Remove unneeded arg passing in tests

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    66a0b33 View commit details
    Browse the repository at this point in the history

  22. Fix bug with seek

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    ca90299 View commit details
    Browse the repository at this point in the history

  23. Internal and doc stringed constants

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    ca4c26b View commit details
    Browse the repository at this point in the history

  24. Internal functions and doc strings

    @petedannemann
    petedannemann committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    c4d1595 View commit details
    Browse the repository at this point in the history

Commits on Jan 6, 2020

  1. Fix issue with double issue on upload and add __str__ and __repr__

    @petedannemann
    petedannemann committed Jan 6, 2020
    Configuration menu
    Copy the full SHA
    79ee333 View commit details
    Browse the repository at this point in the history

  2. Minor cleanup

    @petedannemann
    petedannemann committed Jan 6, 2020
    Configuration menu
    Copy the full SHA
    9622bc6 View commit details
    Browse the repository at this point in the history

  3. Fix flake8 errors

    @petedannemann
    petedannemann committed Jan 6, 2020
    Configuration menu
    Copy the full SHA
    780c549 View commit details
    Browse the repository at this point in the history

  4. Additional flake8 resolution

    @petedannemann
    petedannemann committed Jan 6, 2020
    Configuration menu
    Copy the full SHA
    4382710 View commit details
    Browse the repository at this point in the history

  5. Add source code encoding to test file

    @petedannemann
    petedannemann committed Jan 6, 2020
    Configuration menu
    Copy the full SHA
    13872de View commit details
    Browse the repository at this point in the history

  6. Docstrings in imperative mode

    Peter Dannemann committed Jan 6, 2020
    Configuration menu
    Copy the full SHA
    9b5c802 View commit details
    Browse the repository at this point in the history

  7. Test grammar

    Peter Dannemann committed Jan 6, 2020
    Configuration menu
    Copy the full SHA
    daae9fe View commit details
    Browse the repository at this point in the history

  8. Fix mock_gcs docstring

    Peter Dannemann committed Jan 6, 2020
    Configuration menu
    Copy the full SHA
    86c24db View commit details
    Browse the repository at this point in the history

  9. Only support gs scheme

    Peter Dannemann committed Jan 6, 2020
    Configuration menu
    Copy the full SHA
    f4c4781 View commit details
    Browse the repository at this point in the history

  10. Remove additional occurences of removed gcs scheme

    Peter Dannemann committed Jan 6, 2020
    Configuration menu
    Copy the full SHA
    e03e5fe View commit details
    Browse the repository at this point in the history

  11. Add test_read_past_end

    @petedannemann
    petedannemann committed Jan 6, 2020
    Configuration menu
    Copy the full SHA
    7321bbd View commit details
    Browse the repository at this point in the history

Commits on Jan 7, 2020

  1. Clean up tests with class level decorator

    @petedannemann
    petedannemann committed Jan 7, 2020
    Configuration menu
    Copy the full SHA
    5d1baf8 View commit details
    Browse the repository at this point in the history

  2. Remove stub function

    @petedannemann
    petedannemann committed Jan 7, 2020
    Configuration menu
    Copy the full SHA
    56112cd View commit details
    Browse the repository at this point in the history

  3. Use equality instead of in for scheme

    Peter Dannemann committed Jan 7, 2020
    Configuration menu
    Copy the full SHA
    f4ad80b View commit details
    Browse the repository at this point in the history

  4. Fix repr and mock import

    @petedannemann
    petedannemann committed Jan 7, 2020
    Configuration menu
    Copy the full SHA
    8611dba View commit details
    Browse the repository at this point in the history

  5. Merge branch 'gcs' of https://github.com/petedannemann/smart_open int… 

    …o gcs
    @petedannemann
    petedannemann committed Jan 7, 2020
    Configuration menu
    Copy the full SHA
    8eb3e0a View commit details
    Browse the repository at this point in the history

  6. Specify ImportError

    @petedannemann
    petedannemann committed Jan 7, 2020
    Configuration menu
    Copy the full SHA
    03ac2a9 View commit details
    Browse the repository at this point in the history

Commits on Jan 8, 2020

  1. Use BytesIO in integration test

    @petedannemann
    petedannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    93e4b0f View commit details
    Browse the repository at this point in the history

  2. Explicit encoding in integration tests

    @petedannemann
    petedannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    2bcf8e3 View commit details
    Browse the repository at this point in the history

  3. Remove unneeded variable in integration test 

    Co-Authored-By: Michael Penkov <m@penkov.dev>
    @petedannemann @mpenkov
    petedannemann and mpenkov committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    0a48ba4 View commit details
    Browse the repository at this point in the history

  4. Remove unneeded data variable 

    Co-Authored-By: Michael Penkov <m@penkov.dev>
    @petedannemann @mpenkov
    petedannemann and mpenkov committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    18ca5f8 View commit details
    Browse the repository at this point in the history

  5. Remove .format from test_gcs

    @petedannemann
    petedannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    ffa8083 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'gcs' of https://github.com/petedannemann/smart_open int… 

    …o gcs
    @petedannemann
    petedannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    f56909c View commit details
    Browse the repository at this point in the history

  7. Move RESUMEABLE_SESSION_URI_TEMPLATE to module scope

    @petedannemann
    petedannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    9579b1d View commit details
    Browse the repository at this point in the history

  8. Remove unnecessary explicit encoding 

    Co-Authored-By: Michael Penkov <m@penkov.dev>
    @petedannemann @mpenkov
    petedannemann and mpenkov committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    aa90d64 View commit details
    Browse the repository at this point in the history

  9. Remove unnecessary variable in read 

    Co-Authored-By: Michael Penkov <m@penkov.dev>
    @petedannemann @mpenkov
    petedannemann and mpenkov committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    adf5d56 View commit details
    Browse the repository at this point in the history

  10. Change assertion to use _REQUIRED_CHUNK_MULTIPLE 

    Co-Authored-By: Michael Penkov <m@penkov.dev>
    @petedannemann @mpenkov
    petedannemann and mpenkov committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    03a7d72 View commit details
    Browse the repository at this point in the history

  11. Import clamp from s3

    Peter Dannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    115940a View commit details
    Browse the repository at this point in the history

  12. Add comment on buffering being read-only

    Peter Dannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    b735b7e View commit details
    Browse the repository at this point in the history

  13. Fix client docstring

    Peter Dannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    1d4805c View commit details
    Browse the repository at this point in the history

  14. Make SeekableRawReader internal

    Peter Dannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    0b5797f View commit details
    Browse the repository at this point in the history

  15. Add return value to seek in _SeekableRawReader

    Peter Dannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    6dd7e76 View commit details
    Browse the repository at this point in the history

  16. Allow integration test to take a prefix

    Peter Dannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    94ce2bd View commit details
    Browse the repository at this point in the history

  17. Add doc to NotFound exception

    Peter Dannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    d2a4c2e View commit details
    Browse the repository at this point in the history

  18. Fix misleading log statement

    Peter Dannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    674ad80 View commit details
    Browse the repository at this point in the history

  19. Various formatting changes

    Peter Dannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    910bdde View commit details
    Browse the repository at this point in the history

  20. Add additional assertion for min_part_size

    Peter Dannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    fb44cd2 View commit details
    Browse the repository at this point in the history

  21. Clean up _upload_next_part

    @petedannemann
    petedannemann committed Jan 8, 2020
    Configuration menu
    Copy the full SHA
    f17b9b1 View commit details
    Browse the repository at this point in the history

Commits on Jan 9, 2020

  1. Improve UploadFailedError

    @petedannemann
    petedannemann committed Jan 9, 2020
    Configuration menu
    Copy the full SHA
    5b86291 View commit details
    Browse the repository at this point in the history

  2. Add docstring to terminate

    @petedannemann
    petedannemann committed Jan 9, 2020
    Configuration menu
    Copy the full SHA
    52cb2d1 View commit details
    Browse the repository at this point in the history

  3. Add additional clean up to close

    @petedannemann
    petedannemann committed Jan 9, 2020
    Configuration menu
    Copy the full SHA
    e02fdab View commit details
    Browse the repository at this point in the history

  4. Remove useless terminate in SeekableBufferedInputBase

    @petedannemann
    petedannemann committed Jan 9, 2020
    Configuration menu
    Copy the full SHA
    dcef33d View commit details
    Browse the repository at this point in the history

  5. Fix data type for status_code in UploadFailedError

    @petedannemann
    petedannemann committed Jan 9, 2020
    Configuration menu
    Copy the full SHA
    8ddb98d View commit details
    Browse the repository at this point in the history

  6. Clean up UploadedFailedError msg

    @petedannemann
    petedannemann committed Jan 9, 2020
    Configuration menu
    Copy the full SHA
    c3fc44a View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2020

  1. Start on mock tests

    @petedannemann
    Peter Dannemann authored and petedannemann committed Jan 19, 2020
    Configuration menu
    Copy the full SHA
    7cc0509 View commit details
    Browse the repository at this point in the history

Commits on Jan 20, 2020

  1. Add tests for mocks

    @petedannemann
    petedannemann committed Jan 20, 2020
    Configuration menu
    Copy the full SHA
    bec8a6d View commit details
    Browse the repository at this point in the history

  2. Clean up registering dependencies

    @petedannemann
    petedannemann committed Jan 20, 2020
    Configuration menu
    Copy the full SHA
    be21104 View commit details
    Browse the repository at this point in the history

  3. Get initialize_bucket to work without gsutil

    @petedannemann
    petedannemann committed Jan 20, 2020
    Configuration menu
    Copy the full SHA
    a760f71 View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2020

  1. Change buffering to buffer_size

    Peter Dannemann committed Jan 23, 2020
    Configuration menu
    Copy the full SHA
    dadda49 View commit details
    Browse the repository at this point in the history

  2. Add copyright header, fix logging styles, and move result outside ctx… 

    … manager
    Peter Dannemann committed Jan 23, 2020
    Configuration menu
    Copy the full SHA
    343d5a2 View commit details
    Browse the repository at this point in the history

  3. Change _upload_empty_part to debug msg

    Peter Dannemann committed Jan 23, 2020
    Configuration menu
    Copy the full SHA
    8373740 View commit details
    Browse the repository at this point in the history

  4. Clean up patching style

    Peter Dannemann committed Jan 23, 2020
    Configuration menu
    Copy the full SHA
    ab4d936 View commit details
    Browse the repository at this point in the history

  5. Add tests for smart_open_lib

    Peter Dannemann committed Jan 23, 2020
    Configuration menu
    Copy the full SHA
    f8dd13b View commit details
    Browse the repository at this point in the history

  6. Add blank lines before constants to help readability

    Peter Dannemann committed Jan 23, 2020
    Configuration menu
    Copy the full SHA
    1d2613c View commit details
    Browse the repository at this point in the history

  7. Add missing clean up of raw_reader in close

    Peter Dannemann committed Jan 23, 2020
    Configuration menu
    Copy the full SHA
    9ec0b40 View commit details
    Browse the repository at this point in the history

  8. Remove aws related credentials from gs tests

    Peter Dannemann committed Jan 23, 2020
    Configuration menu
    Copy the full SHA
    a8c272e View commit details
    Browse the repository at this point in the history