Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Ray remote backend and Dask distributed preprocessing #1090

Merged
merged 128 commits into from
Mar 19, 2021
Merged

Commits on Oct 18, 2020

  1. Configuration menu
    Copy the full SHA
    3b8713d View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2020

  1. Configuration menu
    Copy the full SHA
    197314c View commit details
    Browse the repository at this point in the history
  2. Removed debug code

    tgaddair committed Oct 22, 2020
    Configuration menu
    Copy the full SHA
    6bf7083 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    dd52d5c View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2020

  1. Added DataProcessingEngine

    tgaddair committed Oct 23, 2020
    Configuration menu
    Copy the full SHA
    1f228b7 View commit details
    Browse the repository at this point in the history
  2. Fixed split

    tgaddair committed Oct 23, 2020
    Configuration menu
    Copy the full SHA
    12bfea7 View commit details
    Browse the repository at this point in the history
  3. Fixed API

    tgaddair committed Oct 23, 2020
    Configuration menu
    Copy the full SHA
    b39d372 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2020

  1. Fixed data processing

    tgaddair committed Oct 24, 2020
    Configuration menu
    Copy the full SHA
    5b5fc60 View commit details
    Browse the repository at this point in the history
  2. Drop index

    tgaddair committed Oct 24, 2020
    Configuration menu
    Copy the full SHA
    8b2b594 View commit details
    Browse the repository at this point in the history
  3. Added Petastorm dataset

    tgaddair committed Oct 24, 2020
    Configuration menu
    Copy the full SHA
    b7f9546 View commit details
    Browse the repository at this point in the history
  4. Cleaned up dataset creation

    tgaddair committed Oct 24, 2020
    Configuration menu
    Copy the full SHA
    c952130 View commit details
    Browse the repository at this point in the history
  5. Added Dataset

    tgaddair committed Oct 24, 2020
    Configuration menu
    Copy the full SHA
    2c93b60 View commit details
    Browse the repository at this point in the history
  6. Train from dataset

    tgaddair committed Oct 24, 2020
    Configuration menu
    Copy the full SHA
    ea6b4a7 View commit details
    Browse the repository at this point in the history
  7. Fixed bugs

    tgaddair committed Oct 24, 2020
    Configuration menu
    Copy the full SHA
    b203fa6 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2020

  1. Fixed string_utils

    tgaddair committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    6b3bb08 View commit details
    Browse the repository at this point in the history
  2. Fixed tests

    tgaddair committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    ef2a314 View commit details
    Browse the repository at this point in the history
  3. Fixed temp dataset

    tgaddair committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    9a743fe View commit details
    Browse the repository at this point in the history
  4. Added Backend

    tgaddair committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    a630f14 View commit details
    Browse the repository at this point in the history
  5. Plumb through backend

    tgaddair committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    945d56e View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    2aab9c5 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    0a0a7c4 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    3419178 View commit details
    Browse the repository at this point in the history
  9. Fixed Pandas processing

    tgaddair committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    9d13c71 View commit details
    Browse the repository at this point in the history
  10. Added cache management

    tgaddair committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    95a7952 View commit details
    Browse the repository at this point in the history
  11. Fixed unit tests

    tgaddair committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    22b7538 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    fd7cbab View commit details
    Browse the repository at this point in the history
  13. Added numerical test

    tgaddair committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    b63b316 View commit details
    Browse the repository at this point in the history
  14. RayBackend -> DaskBackend

    tgaddair committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    0941ecd View commit details
    Browse the repository at this point in the history
  15. Fixed read_xsv

    tgaddair committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    77a59f9 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2020

  1. Fixed set feature

    tgaddair committed Oct 26, 2020
    Configuration menu
    Copy the full SHA
    cab90a1 View commit details
    Browse the repository at this point in the history
  2. Untracked Netflix example

    tgaddair committed Oct 26, 2020
    Configuration menu
    Copy the full SHA
    755c204 View commit details
    Browse the repository at this point in the history
  3. Added Dask requirements

    tgaddair committed Oct 26, 2020
    Configuration menu
    Copy the full SHA
    a105e41 View commit details
    Browse the repository at this point in the history
  4. Fixed bag feature

    tgaddair committed Oct 26, 2020
    Configuration menu
    Copy the full SHA
    0cbb582 View commit details
    Browse the repository at this point in the history
  5. Fixed vector feature

    tgaddair committed Oct 26, 2020
    Configuration menu
    Copy the full SHA
    87f6e4b View commit details
    Browse the repository at this point in the history
  6. Fixed h3

    tgaddair committed Oct 26, 2020
    Configuration menu
    Copy the full SHA
    5e089d4 View commit details
    Browse the repository at this point in the history
  7. Fixed date

    tgaddair committed Oct 26, 2020
    Configuration menu
    Copy the full SHA
    4f6c0ba View commit details
    Browse the repository at this point in the history
  8. Fixed timeseries

    tgaddair committed Oct 26, 2020
    Configuration menu
    Copy the full SHA
    93cbccd View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2020

  1. Configuration menu
    Copy the full SHA
    0e93043 View commit details
    Browse the repository at this point in the history
  2. Fixed reshaping

    tgaddair committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    bb33fc0 View commit details
    Browse the repository at this point in the history
  3. Fixed tests

    tgaddair committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    7e8d3c3 View commit details
    Browse the repository at this point in the history
  4. Removed debug print

    tgaddair committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    8924ef6 View commit details
    Browse the repository at this point in the history
  5. Fixed image processing

    tgaddair committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    8e95be2 View commit details
    Browse the repository at this point in the history
  6. Added tests for exceptions

    tgaddair committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    b19ea58 View commit details
    Browse the repository at this point in the history
  7. meta_kwargs -> map_objects

    tgaddair committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    0afade5 View commit details
    Browse the repository at this point in the history
  8. Removed unused methods

    tgaddair committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    aabb582 View commit details
    Browse the repository at this point in the history
  9. Removed prints

    tgaddair committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    ac29b9d View commit details
    Browse the repository at this point in the history
  10. Reduced runtime

    tgaddair committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    e3e7a14 View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2020

  1. Configuration menu
    Copy the full SHA
    318da2f View commit details
    Browse the repository at this point in the history
  2. Added dask extra

    tgaddair committed Oct 28, 2020
    Configuration menu
    Copy the full SHA
    1a98f33 View commit details
    Browse the repository at this point in the history
  3. Fixed concatenation

    tgaddair committed Oct 28, 2020
    Configuration menu
    Copy the full SHA
    f625402 View commit details
    Browse the repository at this point in the history
  4. Fixed split empty dataset

    tgaddair committed Oct 28, 2020
    Configuration menu
    Copy the full SHA
    418600a View commit details
    Browse the repository at this point in the history
  5. Fixed subselect

    tgaddair committed Oct 28, 2020
    Configuration menu
    Copy the full SHA
    ed9451b View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    a3de815 View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2020

  1. Moved meta.json

    tgaddair committed Oct 30, 2020
    Configuration menu
    Copy the full SHA
    eacfd63 View commit details
    Browse the repository at this point in the history
  2. Fixed cache key

    tgaddair committed Oct 30, 2020
    Configuration menu
    Copy the full SHA
    4d8e690 View commit details
    Browse the repository at this point in the history
  3. Updated Petastorm

    tgaddair committed Oct 30, 2020
    Configuration menu
    Copy the full SHA
    cd70992 View commit details
    Browse the repository at this point in the history
  4. Spawn Dask tests

    tgaddair committed Oct 30, 2020
    Configuration menu
    Copy the full SHA
    6c94f22 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    cb8bb91 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    985b5bd View commit details
    Browse the repository at this point in the history
  7. Added tables

    tgaddair committed Oct 30, 2020
    Configuration menu
    Copy the full SHA
    60ad4f4 View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2020

  1. Fixed image features

    tgaddair committed Oct 31, 2020
    Configuration menu
    Copy the full SHA
    dff8461 View commit details
    Browse the repository at this point in the history
  2. Fixed string_utils.py

    tgaddair committed Oct 31, 2020
    Configuration menu
    Copy the full SHA
    0469097 View commit details
    Browse the repository at this point in the history
  3. Fixed kfold

    tgaddair committed Oct 31, 2020
    Configuration menu
    Copy the full SHA
    1922f35 View commit details
    Browse the repository at this point in the history
  4. Fixed test splits

    tgaddair committed Oct 31, 2020
    Configuration menu
    Copy the full SHA
    8952e23 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    5057be6 View commit details
    Browse the repository at this point in the history
  6. Fixed test_visualization.py

    tgaddair committed Oct 31, 2020
    Configuration menu
    Copy the full SHA
    20351e0 View commit details
    Browse the repository at this point in the history
  7. Fixed Dask

    tgaddair committed Oct 31, 2020
    Configuration menu
    Copy the full SHA
    92d64c1 View commit details
    Browse the repository at this point in the history
  8. Fixed test_experiment.py

    tgaddair committed Oct 31, 2020
    Configuration menu
    Copy the full SHA
    25ab59b View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2020

  1. Configuration menu
    Copy the full SHA
    9f92c38 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f13fbe5 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    1963305 View commit details
    Browse the repository at this point in the history
  4. Added Ray test

    tgaddair committed Nov 6, 2020
    Configuration menu
    Copy the full SHA
    bbfe0de View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    498f788 View commit details
    Browse the repository at this point in the history
  6. Refactored into mixins

    tgaddair committed Nov 6, 2020
    Configuration menu
    Copy the full SHA
    4d57e18 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    e0bebc2 View commit details
    Browse the repository at this point in the history
  8. Added remote utils

    tgaddair committed Nov 6, 2020
    Configuration menu
    Copy the full SHA
    4ab92cf View commit details
    Browse the repository at this point in the history
  9. Fixed ECD serialization

    tgaddair committed Nov 6, 2020
    Configuration menu
    Copy the full SHA
    61262ef View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2020

  1. Added sync_model to backend

    tgaddair committed Nov 8, 2020
    Configuration menu
    Copy the full SHA
    5fa2b17 View commit details
    Browse the repository at this point in the history
  2. Created RayPredictor

    tgaddair committed Nov 8, 2020
    Configuration menu
    Copy the full SHA
    e15e93b View commit details
    Browse the repository at this point in the history
  3. Fixed kwargs

    tgaddair committed Nov 8, 2020
    Configuration menu
    Copy the full SHA
    a9bb14b View commit details
    Browse the repository at this point in the history
  4. Fixed Horovod in Ray

    tgaddair committed Nov 8, 2020
    Configuration menu
    Copy the full SHA
    e461c00 View commit details
    Browse the repository at this point in the history
  5. Added return_on_master

    tgaddair committed Nov 8, 2020
    Configuration menu
    Copy the full SHA
    1294a21 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    dc17f92 View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2020

  1. Refactored broadcast_return

    tgaddair committed Nov 9, 2020
    Configuration menu
    Copy the full SHA
    1e59b3c View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2020

  1. Configuration menu
    Copy the full SHA
    004902f View commit details
    Browse the repository at this point in the history
  2. Refactored horovod_utils

    tgaddair committed Nov 10, 2020
    Configuration menu
    Copy the full SHA
    cbcaaa1 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    21a3660 View commit details
    Browse the repository at this point in the history
  4. Replaced is_on_master

    tgaddair committed Nov 10, 2020
    Configuration menu
    Copy the full SHA
    f343a64 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    83c9aad View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    3cb9185 View commit details
    Browse the repository at this point in the history
  7. master -> coordinator

    tgaddair committed Nov 10, 2020
    Configuration menu
    Copy the full SHA
    594ca61 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    902e8b7 View commit details
    Browse the repository at this point in the history
  9. Fixed hyperopt

    tgaddair committed Nov 10, 2020
    Configuration menu
    Copy the full SHA
    1e65b21 View commit details
    Browse the repository at this point in the history
  10. Added requirements for Ray

    tgaddair committed Nov 10, 2020
    Configuration menu
    Copy the full SHA
    b245055 View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2020

  1. Configuration menu
    Copy the full SHA
    08bf843 View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2021

  1. Merged from master

    tgaddair committed Jan 17, 2021
    Configuration menu
    Copy the full SHA
    2febe92 View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2021

  1. Merged master

    tgaddair committed Feb 4, 2021
    Configuration menu
    Copy the full SHA
    e8ed0f5 View commit details
    Browse the repository at this point in the history
  2. Removed processor module

    tgaddair committed Feb 4, 2021
    Configuration menu
    Copy the full SHA
    5b979a3 View commit details
    Browse the repository at this point in the history
  3. Removed remote_utils

    tgaddair committed Feb 4, 2021
    Configuration menu
    Copy the full SHA
    fe07d3f View commit details
    Browse the repository at this point in the history
  4. Fixed Dask tests

    tgaddair committed Feb 4, 2021
    Configuration menu
    Copy the full SHA
    273c85d View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2021

  1. Temp fix tests

    tgaddair committed Feb 7, 2021
    Configuration menu
    Copy the full SHA
    b10393a View commit details
    Browse the repository at this point in the history
  2. Spawn test

    tgaddair committed Feb 7, 2021
    Configuration menu
    Copy the full SHA
    73c5ad1 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2021

  1. Merged master

    tgaddair committed Feb 21, 2021
    Configuration menu
    Copy the full SHA
    9fab810 View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2021

  1. Configuration menu
    Copy the full SHA
    77e1b1f View commit details
    Browse the repository at this point in the history

Commits on Mar 2, 2021

  1. Set Ray as Dask backend

    tgaddair committed Mar 2, 2021
    Configuration menu
    Copy the full SHA
    7c2d65b View commit details
    Browse the repository at this point in the history
  2. Fixed comments

    tgaddair committed Mar 2, 2021
    Configuration menu
    Copy the full SHA
    1822501 View commit details
    Browse the repository at this point in the history
  3. Merge

    tgaddair committed Mar 2, 2021
    Configuration menu
    Copy the full SHA
    745c96c View commit details
    Browse the repository at this point in the history
  4. Removed link

    tgaddair committed Mar 2, 2021
    Configuration menu
    Copy the full SHA
    ff795a4 View commit details
    Browse the repository at this point in the history
  5. Added test

    tgaddair committed Mar 2, 2021
    Configuration menu
    Copy the full SHA
    8f65656 View commit details
    Browse the repository at this point in the history
  6. TEST: skip experiment.py

    tgaddair committed Mar 2, 2021
    Configuration menu
    Copy the full SHA
    b87e7bc View commit details
    Browse the repository at this point in the history
  7. Split backend tests

    tgaddair committed Mar 2, 2021
    Configuration menu
    Copy the full SHA
    f6e3686 View commit details
    Browse the repository at this point in the history
  8. Added pytest.ini

    tgaddair committed Mar 2, 2021
    Configuration menu
    Copy the full SHA
    15920c1 View commit details
    Browse the repository at this point in the history
  9. backend -> distributed

    tgaddair committed Mar 2, 2021
    Configuration menu
    Copy the full SHA
    1094f78 View commit details
    Browse the repository at this point in the history
  10. Reordered tests

    tgaddair committed Mar 2, 2021
    Configuration menu
    Copy the full SHA
    7c324c1 View commit details
    Browse the repository at this point in the history
  11. Configure Dask parallelism

    tgaddair committed Mar 2, 2021
    Configuration menu
    Copy the full SHA
    ff598e3 View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2021

  1. TEST: disable fiber

    tgaddair committed Mar 4, 2021
    Configuration menu
    Copy the full SHA
    7ea761f View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2021

  1. Test without Ray hyperopt

    tgaddair committed Mar 6, 2021
    Configuration menu
    Copy the full SHA
    23b4be0 View commit details
    Browse the repository at this point in the history
  2. Test ray only

    tgaddair committed Mar 6, 2021
    Configuration menu
    Copy the full SHA
    214f718 View commit details
    Browse the repository at this point in the history
  3. Test all distributed

    tgaddair committed Mar 6, 2021
    Configuration menu
    Copy the full SHA
    65f3293 View commit details
    Browse the repository at this point in the history
  4. Run distributed

    tgaddair committed Mar 6, 2021
    Configuration menu
    Copy the full SHA
    f6a4227 View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2021

  1. Only ray

    tgaddair committed Mar 7, 2021
    Configuration menu
    Copy the full SHA
    740ed9d View commit details
    Browse the repository at this point in the history

Commits on Mar 10, 2021

  1. Serialize on load

    tgaddair committed Mar 10, 2021
    Configuration menu
    Copy the full SHA
    d10e14f View commit details
    Browse the repository at this point in the history
  2. Only return the weights

    tgaddair committed Mar 10, 2021
    Configuration menu
    Copy the full SHA
    faf0052 View commit details
    Browse the repository at this point in the history
  3. Revert test changes

    tgaddair committed Mar 10, 2021
    Configuration menu
    Copy the full SHA
    0259e07 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    97139a6 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    54f1939 View commit details
    Browse the repository at this point in the history

Commits on Mar 16, 2021

  1. Resolved merge conflicts

    tgaddair committed Mar 16, 2021
    Configuration menu
    Copy the full SHA
    eb9fb2e View commit details
    Browse the repository at this point in the history

Commits on Mar 19, 2021

  1. Addressed comments

    tgaddair committed Mar 19, 2021
    Configuration menu
    Copy the full SHA
    6dfbce6 View commit details
    Browse the repository at this point in the history