Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Serialize and Deserialize from S3/URLs #685

Merged
merged 86 commits into from Aug 2, 2019
Merged

Serialize and Deserialize from S3/URLs #685

merged 86 commits into from Aug 2, 2019

Commits on Jul 23, 2019

  1. Added smart-open to requirements

    @jeremyliweishih
    jeremyliweishih committed Jul 23, 2019
    Copy the full SHA
    c6b4e7f View commit details
    Browse the repository at this point in the history

  2. Added import to replace open with smart_open

    @jeremyliweishih
    jeremyliweishih committed Jul 23, 2019
    Copy the full SHA
    6add696 View commit details
    Browse the repository at this point in the history

  3. Added smart_open for writing

    @jeremyliweishih
    jeremyliweishih committed Jul 23, 2019
    Copy the full SHA
    f6de2a8 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2019

  1. First changes to serialize

    @jeremyliweishih
    jeremyliweishih committed Jul 24, 2019
    Copy the full SHA
    c6879b6 View commit details
    Browse the repository at this point in the history

  2. Fixed serialization

    @jeremyliweishih
    jeremyliweishih committed Jul 24, 2019
    Copy the full SHA
    2388c46 View commit details
    Browse the repository at this point in the history

  3. Added first half of tests using moto

    @jeremyliweishih
    jeremyliweishih committed Jul 24, 2019
    Copy the full SHA
    29650d1 View commit details
    Browse the repository at this point in the history

  4. Fixed serializing

    @jeremyliweishih
    jeremyliweishih committed Jul 24, 2019
    Copy the full SHA
    186717a View commit details
    Browse the repository at this point in the history

  5. Added url deserialization

    @jeremyliweishih
    jeremyliweishih committed Jul 24, 2019
    Copy the full SHA
    42f05f0 View commit details
    Browse the repository at this point in the history

  6. Added s3 tests

    @jeremyliweishih
    jeremyliweishih committed Jul 24, 2019
    Copy the full SHA
    84c5b52 View commit details
    Browse the repository at this point in the history

  7. Fixed import order

    @jeremyliweishih
    jeremyliweishih committed Jul 24, 2019
    Copy the full SHA
    1c3b8b3 View commit details
    Browse the repository at this point in the history

  8. Added s3 test for feature serialization using moto

    @jeremyliweishih
    jeremyliweishih committed Jul 24, 2019
    Copy the full SHA
    08b53f5 View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2019

  1. Added read from real s3_url test and url tests

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    71e1f13 View commit details
    Browse the repository at this point in the history

  2. Added using default profile for s3 or use session parameter

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    244ef09 View commit details
    Browse the repository at this point in the history

  3. fixed real s3 test name

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    02b40ca View commit details
    Browse the repository at this point in the history

  4. Seperated s3 and urls

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    f4f1cae View commit details
    Browse the repository at this point in the history

  5. Added support to s3 and URLs to feature serialization

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    3bcb41d View commit details
    Browse the repository at this point in the history

  6. Added tests for feature serialization

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    6de475a View commit details
    Browse the repository at this point in the history

  7. Fixed import order

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    80dd85b View commit details
    Browse the repository at this point in the history

  8. Merge branch 'master' into read_from_url

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    44b0438 View commit details
    Browse the repository at this point in the history

  9. erge branch 'master' of https://github.com/featuretools/featuretools … 

    …into read_from_url
    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    f188315 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'read_from_url' of https://github.com/featuretools/featu… 

    …retools into read_from_url
    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    cd9e86b View commit details
    Browse the repository at this point in the history

  11. Added pathlib for py < 3

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    a879bed View commit details
    Browse the repository at this point in the history

  12. Fixed version of pathlib

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    736e634 View commit details
    Browse the repository at this point in the history

  13. Added s3fs for anon users

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    45b1130 View commit details
    Browse the repository at this point in the history

  14. Fixed s3 without credentials for serialization

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    e679b30 View commit details
    Browse the repository at this point in the history

  15. Fixed s3 without credentials for feature serialization

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    a2b27e3 View commit details
    Browse the repository at this point in the history

  16. Fixed tests for s3 without credentials

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    77ccccd View commit details
    Browse the repository at this point in the history

  17. Fixed tests for s3 without credentials for feature serialization

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    5b4f557 View commit details
    Browse the repository at this point in the history

  18. Fixed lint issues

    @jeremyliweishih
    jeremyliweishih committed Jul 25, 2019
    Copy the full SHA
    85cfb42 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2019

  1. Fixed reading bytes from json for py3.5

    @jeremyliweishih
    jeremyliweishih committed Jul 26, 2019
    Copy the full SHA
    ff42ff4 View commit details
    Browse the repository at this point in the history

  2. Added encoding for json loading

    @jeremyliweishih
    jeremyliweishih committed Jul 26, 2019
    Copy the full SHA
    e9b2e43 View commit details
    Browse the repository at this point in the history

  3. Added sorting for unit tests as feature ordering is different in 3.5 … 

    …vs 3.6+
    @jeremyliweishih
    jeremyliweishih committed Jul 26, 2019
    Copy the full SHA
    a5104d3 View commit details
    Browse the repository at this point in the history

  4. Added changes to changelog

    @jeremyliweishih
    jeremyliweishih committed Jul 26, 2019
    Copy the full SHA
    99edd99 View commit details
    Browse the repository at this point in the history

  5. Edited comments and made stylistic changes

    @jeremyliweishih
    jeremyliweishih committed Jul 26, 2019
    Copy the full SHA
    cac9074 View commit details
    Browse the repository at this point in the history

  6. Fixed s3fs version

    @jeremyliweishih
    jeremyliweishih committed Jul 26, 2019
    Copy the full SHA
    7ab3ded View commit details
    Browse the repository at this point in the history

  7. Fixed lint issues

    @jeremyliweishih
    jeremyliweishih committed Jul 26, 2019
    Copy the full SHA
    0a9fa2c View commit details
    Browse the repository at this point in the history

  8. Replaced urllib for python 2

    @jeremyliweishih
    jeremyliweishih committed Jul 26, 2019
    Copy the full SHA
    27e9f06 View commit details
    Browse the repository at this point in the history

  9. Fixed space in changelog

    @jeremyliweishih
    jeremyliweishih committed Jul 26, 2019
    Copy the full SHA
    60cfb2d View commit details
    Browse the repository at this point in the history

  10. Fixed checking url

    @jeremyliweishih
    jeremyliweishih committed Jul 26, 2019
    Copy the full SHA
    66e0ef6 View commit details
    Browse the repository at this point in the history

  11. Changed to use backport.tempfile for py2.7

    @jeremyliweishih
    jeremyliweishih committed Jul 26, 2019
    Copy the full SHA
    10bd93c View commit details
    Browse the repository at this point in the history

  12. Fixed checking URLs for features

    @jeremyliweishih
    jeremyliweishih committed Jul 26, 2019
    Copy the full SHA
    9ca4bba View commit details
    Browse the repository at this point in the history

  13. Fixed writing to s3 using s3fs

    @jeremyliweishih
    jeremyliweishih committed Jul 26, 2019
    Copy the full SHA
    5397307 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2019

  1. Added AWS profile as paramater for serialization

    @jeremyliweishih
    jeremyliweishih committed Jul 29, 2019
    Copy the full SHA
    6c1b83b View commit details
    Browse the repository at this point in the history

  2. Added tests for checking AWS profile

    @jeremyliweishih
    jeremyliweishih committed Jul 29, 2019
    Copy the full SHA
    7e35b4c View commit details
    Browse the repository at this point in the history

  3. Added aws_profile as a parameter

    @jeremyliweishih
    jeremyliweishih committed Jul 29, 2019
    Copy the full SHA
    99777ad View commit details
    Browse the repository at this point in the history

  4. Added profile_name as parameter

    @jeremyliweishih
    jeremyliweishih committed Jul 29, 2019
    Copy the full SHA
    cc4ce08 View commit details
    Browse the repository at this point in the history

  5. Fixed lint

    @jeremyliweishih
    jeremyliweishih committed Jul 29, 2019
    Copy the full SHA
    3485a8a View commit details
    Browse the repository at this point in the history

  6. Fixed comments with path_name parameter

    @jeremyliweishih
    jeremyliweishih committed Jul 29, 2019
    Copy the full SHA
    1168ab4 View commit details
    Browse the repository at this point in the history

  7. Slight fix to logic

    @jeremyliweishih
    jeremyliweishih committed Jul 29, 2019
    Copy the full SHA
    e34482a View commit details
    Browse the repository at this point in the history

  8. Removed try in tests

    @jeremyliweishih
    jeremyliweishih committed Jul 29, 2019
    Copy the full SHA
    9f603dd View commit details
    Browse the repository at this point in the history

  9. Added fixtures for mock_s3 tests

    @jeremyliweishih
    jeremyliweishih committed Jul 29, 2019
    Copy the full SHA
    5004ca8 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2019

  1. Moved moto to test-requirements

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    16b5769 View commit details
    Browse the repository at this point in the history

  2. Cleaned up verbose directory creation

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    1d0a63a View commit details
    Browse the repository at this point in the history

  3. Fixed forward slash for arc name

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    076a15e View commit details
    Browse the repository at this point in the history

  4. Added helper functions to clean code

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    c73bc0a View commit details
    Browse the repository at this point in the history

  5. Moved url checking functions to utils

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    ab0cce5 View commit details
    Browse the repository at this point in the history

  6. Fixed if style

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    25eef1f View commit details
    Browse the repository at this point in the history

  7. Fixed test config name

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    2c184c5 View commit details
    Browse the repository at this point in the history

  8. Changed behavior for aws credentials

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    1
    Copy the full SHA
    254121e View commit details
    Browse the repository at this point in the history

  9. Fixed checking profile tests and resetting environment variables

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    88974b5 View commit details
    Browse the repository at this point in the history

  10. Removed unused imports

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    626be31 View commit details
    Browse the repository at this point in the history

  11. Fixed profile tests

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    9d28710 View commit details
    Browse the repository at this point in the history

  12. Added python 2 specific importing

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    3a9a699 View commit details
    Browse the repository at this point in the history

  13. Fixed wrong function call

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    a8c3e0b View commit details
    Browse the repository at this point in the history

  14. Fixed requirements to be python2 only for backports

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    8bcbac7 View commit details
    Browse the repository at this point in the history

  15. Fixed checking python version in requirements

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    6a88969 View commit details
    Browse the repository at this point in the history

  16. Fixed if logic

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    6e4baf4 View commit details
    Browse the repository at this point in the history

  17. Fixed order of tests for python2

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    4256a6c View commit details
    Browse the repository at this point in the history

  18. Added tests to explicitly test anon profiles

    @jeremyliweishih
    jeremyliweishih committed Jul 30, 2019
    Copy the full SHA
    544d89c View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2019

  1. Fixed logic and made more consistent

    @jeremyliweishih
    jeremyliweishih committed Jul 31, 2019
    Copy the full SHA
    44968d3 View commit details
    Browse the repository at this point in the history

  2. Moved open statements into utils and fixed logic

    @jeremyliweishih
    jeremyliweishih committed Jul 31, 2019
    Copy the full SHA
    9ec6588 View commit details
    Browse the repository at this point in the history

  3. Made tests consitent with other serialization tests

    @jeremyliweishih
    jeremyliweishih committed Jul 31, 2019
    Copy the full SHA
    bbf87eb View commit details
    Browse the repository at this point in the history

  4. Removed faulty encoding parameter for py2

    @jeremyliweishih
    jeremyliweishih committed Jul 31, 2019
    Copy the full SHA
    5940734 View commit details
    Browse the repository at this point in the history

  5. Added profile specific tests

    @jeremyliweishih
    jeremyliweishih committed Jul 31, 2019
    Copy the full SHA
    73d7ab5 View commit details
    Browse the repository at this point in the history

  6. Fix lint

    @jeremyliweishih
    jeremyliweishih committed Jul 31, 2019
    Copy the full SHA
    7b4702b View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2019

  1. Fixed descriptions

    @jeremyliweishih
    jeremyliweishih committed Aug 2, 2019
    Copy the full SHA
    7db7701 View commit details
    Browse the repository at this point in the history

  2. Fixed imports

    @jeremyliweishih
    jeremyliweishih committed Aug 2, 2019
    Copy the full SHA
    cdd8b49 View commit details
    Browse the repository at this point in the history

  3. Moved repetitive code into funciton

    @jeremyliweishih
    jeremyliweishih committed Aug 2, 2019
    Copy the full SHA
    4970a4e View commit details
    Browse the repository at this point in the history

  4. Add todo

    @jeremyliweishih
    jeremyliweishih committed Aug 2, 2019
    Copy the full SHA
    dc9f1a7 View commit details
    Browse the repository at this point in the history

  5. Clean up setting up test profile

    @jeremyliweishih
    jeremyliweishih committed Aug 2, 2019
    Copy the full SHA
    281c9cc View commit details
    Browse the repository at this point in the history

  6. Move repetitive code into function

    @jeremyliweishih
    jeremyliweishih committed Aug 2, 2019
    Copy the full SHA
    7b18ebc View commit details
    Browse the repository at this point in the history

  7. Reduced repetitive tests

    @jeremyliweishih
    jeremyliweishih committed Aug 2, 2019
    Copy the full SHA
    6a04a11 View commit details
    Browse the repository at this point in the history

  8. Added expectation of tar format

    @jeremyliweishih
    jeremyliweishih committed Aug 2, 2019
    Copy the full SHA
    95644c1 View commit details
    Browse the repository at this point in the history

  9. Fixed description

    @jeremyliweishih
    jeremyliweishih committed Aug 2, 2019
    Copy the full SHA
    bec2e3c View commit details
    Browse the repository at this point in the history

  10. Parameterize testing s3

    @jeremyliweishih
    jeremyliweishih committed Aug 2, 2019
    Copy the full SHA
    d29411a View commit details
    Browse the repository at this point in the history

  11. Merge branch 'master' into read_from_url

    @jeremyliweishih
    jeremyliweishih committed Aug 2, 2019
    Copy the full SHA
    eac8821 View commit details
    Browse the repository at this point in the history