Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Barrier support for task plugins #18

Merged
merged 104 commits into from
Oct 15, 2019
Merged

Barrier support for task plugins #18

merged 104 commits into from
Oct 15, 2019

Commits on Sep 9, 2019

  1. Bump IDL/Plugins versions (#4) 

    Bumping versions to pick up Flyte IDL change deprecating Hive query collections.

https://github.com/lyft/flyteplugins/releases/tag/v0.1.2
https://github.com/lyft/flyteplugins/releases/tag/v0.1.3
https://github.com/lyft/flyteidl/releases/tag/v0.14.0
    @wild-endeavor
    wild-endeavor authored Sep 9, 2019
    Configuration menu
    Copy the full SHA
    780b79d View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2019

  1. Cache task executions to DataCatalog

    @chanadian
    chanadian committed Sep 10, 2019
    Configuration menu
    Copy the full SHA
    c81a0a8 View commit details
    Browse the repository at this point in the history

  2. Correct metadata for execution name

    @chanadian
    chanadian committed Sep 10, 2019
    Configuration menu
    Copy the full SHA
    8ba3946 View commit details
    Browse the repository at this point in the history

  3. Specify insecure connection with config

    @chanadian
    chanadian committed Sep 10, 2019
    Configuration menu
    Copy the full SHA
    2404756 View commit details
    Browse the repository at this point in the history

  4. Add more comments

    @chanadian
    chanadian committed Sep 10, 2019
    Configuration menu
    Copy the full SHA
    876c9a9 View commit details
    Browse the repository at this point in the history

  5. go-lint cleanup

    @chanadian
    chanadian committed Sep 10, 2019
    Configuration menu
    Copy the full SHA
    4366de7 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2019

  1. Update flyteplugins to 0.1.5

    @EngHabu
    EngHabu committed Sep 11, 2019
    Configuration menu
    Copy the full SHA
    e91e3de View commit details
    Browse the repository at this point in the history

  2. Merge pull request #6 from lyft/update-plugins-1.5 

    Update flyteplugins to 0.1.5
    @EngHabu
    EngHabu authored Sep 11, 2019
    Configuration menu
    Copy the full SHA
    8de0681 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #5 from lyft/datacatalog-cache 

    Datacatalog cache
    @chanadian
    chanadian authored Sep 11, 2019
    Configuration menu
    Copy the full SHA
    2a9e28f View commit details
    Browse the repository at this point in the history

  4. bump-versions flyteidl and flyteplugins

    @vsbus
    vsbus authored Sep 11, 2019
    Configuration menu
    Copy the full SHA
    cad9f00 View commit details
    Browse the repository at this point in the history

  5. github.com/dgrijalva/jwt-go was missing (#8) 

    Could not compile. Apparently dep ensure doesn't update transitive dependencies. This just runs dep ensure -update.
    @wild-endeavor
    wild-endeavor authored Sep 11, 2019
    Configuration menu
    Copy the full SHA
    6df59d7 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2019

  1. Merge branch 'master' into plugin-refactor

    Ketan Umare committed Sep 12, 2019
    Configuration menu
    Copy the full SHA
    f1a8f23 View commit details
    Browse the repository at this point in the history

  2. merge issue

    Ketan Umare committed Sep 12, 2019
    Configuration menu
    Copy the full SHA
    4b30072 View commit details
    Browse the repository at this point in the history

  3. Final top workflow tests in progress

    Ketan Umare committed Sep 12, 2019
    Configuration menu
    Copy the full SHA
    37bf8dd View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2019

  1. Tests working and compiling

    Ketan Umare committed Sep 13, 2019
    Configuration menu
    Copy the full SHA
    413086e View commit details
    Browse the repository at this point in the history

  2. Linter Fix

    Ketan Umare committed Sep 13, 2019
    Configuration menu
    Copy the full SHA
    e45a775 View commit details
    Browse the repository at this point in the history

  3. Adjust logging levels

    @chanadian
    chanadian committed Sep 13, 2019
    Configuration menu
    Copy the full SHA
    4729c1d View commit details
    Browse the repository at this point in the history

  4. loaded container plugin

    Ketan Umare committed Sep 13, 2019
    Configuration menu
    Copy the full SHA
    a5b1836 View commit details
    Browse the repository at this point in the history

  5. correct metadata to pass workflow exec name

    @chanadian
    chanadian committed Sep 13, 2019
    Configuration menu
    Copy the full SHA
    08dd99f View commit details
    Browse the repository at this point in the history

  6. Clarify key name for execution name

    @chanadian
    chanadian committed Sep 13, 2019
    Configuration menu
    Copy the full SHA
    841ff8f View commit details
    Browse the repository at this point in the history

  7. Adjust Catalog logs (#9) 

    Adjust Catalog logs
    @chanadian
    chanadian authored Sep 13, 2019
    Configuration menu
    Copy the full SHA
    86ed7f8 View commit details
    Browse the repository at this point in the history

  8. More unit tests done and eventing tested

    Ketan Umare committed Sep 13, 2019
    Configuration menu
    Copy the full SHA
    58648b9 View commit details
    Browse the repository at this point in the history

  9. remove backwards compatibility

    Ketan Umare committed Sep 13, 2019
    Configuration menu
    Copy the full SHA
    a92d1ca View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2019

  1. Working simple task

    Ketan Umare committed Sep 14, 2019
    Configuration menu
    Copy the full SHA
    b2e2c7c View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2019

  1. Dynamic nodes working

    Ketan Umare committed Sep 16, 2019
    Configuration menu
    Copy the full SHA
    13cc97d View commit details
    Browse the repository at this point in the history

  2. k8s array being called successfully

    Ketan Umare committed Sep 16, 2019
    Configuration menu
    Copy the full SHA
    a40b620 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2019

  1. Improved Plugin manager

    Ketan Umare committed Sep 17, 2019
    Configuration menu
    Copy the full SHA
    1d7d521 View commit details
    Browse the repository at this point in the history

  2. Updated workflow

    Ketan Umare committed Sep 17, 2019
    Configuration menu
    Copy the full SHA
    e0cb7b9 View commit details
    Browse the repository at this point in the history

  3. branch node

    Surinder Singh committed Sep 17, 2019
    Configuration menu
    Copy the full SHA
    c50e2c4 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'plugin-refactor' into branch-wf-node

    Surinder Singh committed Sep 17, 2019
    Configuration menu
    Copy the full SHA
    cccd8b7 View commit details
    Browse the repository at this point in the history

  5. Update plugins to 0.1.7 (#11)

    @EngHabu
    EngHabu authored Sep 17, 2019
    Configuration menu
    Copy the full SHA
    f9f959d View commit details
    Browse the repository at this point in the history

  6. All unit tests working and linter issues fixed

    Ketan Umare committed Sep 17, 2019
    Configuration menu
    Copy the full SHA
    8f74d13 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2019

  1. work in progress

    Ketan Umare committed Sep 18, 2019
    Configuration menu
    Copy the full SHA
    31ad97c View commit details
    Browse the repository at this point in the history

  2. Metrics captured for offending workflows

    Ketan Umare committed Sep 18, 2019
    Configuration menu
    Copy the full SHA
    147eb44 View commit details
    Browse the repository at this point in the history

  3. Another metric to measure time taken to gather inputs

    Ketan Umare committed Sep 18, 2019
    Configuration menu
    Copy the full SHA
    7ec1adc View commit details
    Browse the repository at this point in the history

  4. Merge pull request #13 from lyft/more-metrics 

    Metrics captured for offending workflows
    Ketan Umare authored Sep 18, 2019
    Configuration menu
    Copy the full SHA
    cff87f1 View commit details
    Browse the repository at this point in the history

  5. Variable created would shadow external variable

    Ketan Umare committed Sep 18, 2019
    Configuration menu
    Copy the full SHA
    058d343 View commit details
    Browse the repository at this point in the history

  6. Merge pull request #14 from lyft/fix-typo 

    Variable created would shadow external variable
    Ketan Umare authored Sep 18, 2019
    Configuration menu
    Copy the full SHA
    5f7503b View commit details
    Browse the repository at this point in the history

  7. updated lock file

    Ketan Umare committed Sep 18, 2019
    Configuration menu
    Copy the full SHA
    b2c6fe3 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'master' into plugin-refactor

    Ketan Umare committed Sep 18, 2019
    Configuration menu
    Copy the full SHA
    179a19f View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2019

  1. fixing tests

    Surinder Singh committed Sep 19, 2019
    Configuration menu
    Copy the full SHA
    474adaf View commit details
    Browse the repository at this point in the history

  2. Advanced options to configure kube client for large scale deployments

    Ketan Umare committed Sep 19, 2019
    Configuration menu
    Copy the full SHA
    d6fbfd3 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #15 from lyft/configurable-kube-client 

    Advanced options to configure kube client for large scale deployments
    @matthewphsmith
    matthewphsmith authored Sep 19, 2019
    Configuration menu
    Copy the full SHA
    daa7e99 View commit details
    Browse the repository at this point in the history

  4. some renaming

    Surinder Singh committed Sep 19, 2019
    Configuration menu
    Copy the full SHA
    20cb713 View commit details
    Browse the repository at this point in the history

  5. consistent naming

    Surinder Singh committed Sep 19, 2019
    Configuration menu
    Copy the full SHA
    2f4c0ac View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2019

  1. Cache Dynamic WF Spec (#16) 

    * Cache Dynamic WF Spec

* Cleanup

* Update Deps
    @EngHabu
    EngHabu authored Sep 20, 2019
    Configuration menu
    Copy the full SHA
    8ab3272 View commit details
    Browse the repository at this point in the history

  2. cr feedback

    Surinder Singh committed Sep 20, 2019
    Configuration menu
    Copy the full SHA
    4b70288 View commit details
    Browse the repository at this point in the history

  3. cr feedback

    Surinder Singh committed Sep 20, 2019
    Configuration menu
    Copy the full SHA
    b117448 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'plugin-refactor' into branch-wf-node

    Surinder Singh committed Sep 20, 2019
    Configuration menu
    Copy the full SHA
    99bf9eb View commit details
    Browse the repository at this point in the history

  5. Merge pull request #10 from lyft/branch-wf-node 

    branch node
    @surindersinghp
    surindersinghp authored Sep 20, 2019
    Configuration menu
    Copy the full SHA
    a6dd147 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'master' into plugin-refactor

    Ketan Umare committed Sep 20, 2019
    Configuration menu
    Copy the full SHA
    4c8b06d View commit details
    Browse the repository at this point in the history

  7. tests working and fixed compilation errors

    Ketan Umare committed Sep 20, 2019
    Configuration menu
    Copy the full SHA
    8fdae34 View commit details
    Browse the repository at this point in the history

  8. linter fixed

    Ketan Umare committed Sep 20, 2019
    Configuration menu
    Copy the full SHA
    007f5f3 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'plugin-refactor' of github.com:lyft/flytepropeller into… 

    … plugin-refactor
    Ketan Umare committed Sep 20, 2019
    Configuration menu
    Copy the full SHA
    ecf8e75 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2019

  1. removed unused methods

    Ketan Umare committed Sep 21, 2019
    Configuration menu
    Copy the full SHA
    81a0c75 View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2019

  1. Move common code to pluginmachinery

    Ketan Umare committed Sep 23, 2019
    Configuration menu
    Copy the full SHA
    8ddac46 View commit details
    Browse the repository at this point in the history

  2. Not compiling - Cycle

    Ketan Umare committed Sep 23, 2019
    Configuration menu
    Copy the full SHA
    d11b77c View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2019

  1. Catalog

    Ketan Umare committed Sep 24, 2019
    Configuration menu
    Copy the full SHA
    ee6ca34 View commit details
    Browse the repository at this point in the history

  2. fixing unit tests

    Ketan Umare committed Sep 24, 2019
    Configuration menu
    Copy the full SHA
    a512597 View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2019

  1. Updated handler

    Ketan Umare committed Sep 25, 2019
    Configuration menu
    Copy the full SHA
    c09438e View commit details
    Browse the repository at this point in the history

  2. fixed unit tests

    Ketan Umare committed Sep 25, 2019
    Configuration menu
    Copy the full SHA
    efeb27d View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2019

  1. Enable k8s && aws batch

    @EngHabu
    EngHabu committed Sep 27, 2019
    Configuration menu
    Copy the full SHA
    9b7d9bb View commit details
    Browse the repository at this point in the history

  2. Barrier support for task plugins

    Ketan Umare committed Sep 27, 2019
    Configuration menu
    Copy the full SHA
    e342c82 View commit details
    Browse the repository at this point in the history

  3. tests fixed

    Ketan Umare committed Sep 27, 2019
    Configuration menu
    Copy the full SHA
    af5a5c4 View commit details
    Browse the repository at this point in the history

  4. linter fixes

    Ketan Umare committed Sep 27, 2019
    Configuration menu
    Copy the full SHA
    1289c93 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'plugin-refactor' into barrier-impl

    Ketan Umare committed Sep 27, 2019
    Configuration menu
    Copy the full SHA
    64004f3 View commit details
    Browse the repository at this point in the history

  6. linter fix

    Ketan Umare committed Sep 27, 2019
    Configuration menu
    Copy the full SHA
    4b3374c View commit details
    Browse the repository at this point in the history

  7. Merge branch 'plugin-refactor' of github.com:lyft/flytepropeller into… 

    … plugin-refactor
    Ketan Umare committed Sep 27, 2019
    Configuration menu
    Copy the full SHA
    6f5ac61 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2019

  1. Dep ensure

    @EngHabu
    EngHabu committed Sep 28, 2019
    Configuration menu
    Copy the full SHA
    99597ed View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2019

  1. Aborting all running nodes on failure or external abort

    Ketan Umare committed Sep 29, 2019
    Configuration menu
    Copy the full SHA
    2ee6a13 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2019

  1. Merge origin

    @EngHabu
    EngHabu committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    beef93e View commit details
    Browse the repository at this point in the history

  2. Only enable container and k8s array plugins locally

    @EngHabu
    EngHabu committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    62d643c View commit details
    Browse the repository at this point in the history

  3. Fix enabled plugins check

    @EngHabu
    EngHabu committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    a69b88c View commit details
    Browse the repository at this point in the history

  4. Fix enabled-plugins check

    @EngHabu
    EngHabu committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    1de8966 View commit details
    Browse the repository at this point in the history

  5. updated setupcontext

    Ketan Umare committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    a95192a View commit details
    Browse the repository at this point in the history

  6. Merge branch 'plugin-refactor' into barrier-impl

    Ketan Umare committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    dae2d25 View commit details
    Browse the repository at this point in the history

  7. merge bugs

    Ketan Umare committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    a091264 View commit details
    Browse the repository at this point in the history

  8. Fixes from master

    @EngHabu
    EngHabu committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    5a6ad00 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'plugin-refactor' of github.com:lyft/flytepropeller into… 

    … plugin-refactor
    @EngHabu
    EngHabu committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    d0ae8f1 View commit details
    Browse the repository at this point in the history

  10. Better plugin wrangling logic

    Ketan Umare committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    1c89ac7 View commit details
    Browse the repository at this point in the history

  11. update config

    Ketan Umare committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    772a332 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'plugin-refactor' of github.com:lyft/flytepropeller into… 

    … plugin-refactor
    @EngHabu
    EngHabu committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    94d2898 View commit details
    Browse the repository at this point in the history

  13. Merge branch 'plugin-refactor' of github.com:lyft/flytepropeller into… 

    … plugin-refactor
    Ketan Umare committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    0c33208 View commit details
    Browse the repository at this point in the history

  14. plugin manager

    Ketan Umare committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    c3d7dd2 View commit details
    Browse the repository at this point in the history

  15. Working cases

    Ketan Umare committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    f747771 View commit details
    Browse the repository at this point in the history

  16. rename discovery -> catalog

    Ketan Umare committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    44f852f View commit details
    Browse the repository at this point in the history

  17. Merge branch 'plugin-refactor' into barrier-impl

    Ketan Umare committed Sep 30, 2019
    Configuration menu
    Copy the full SHA
    22b989e View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2019

  1. Barrier improvements

    Ketan Umare committed Oct 1, 2019
    Configuration menu
    Copy the full SHA
    3e0f6ed View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2019

  1. Handler updated

    Ketan Umare committed Oct 9, 2019
    Configuration menu
    Copy the full SHA
    5dfac3e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'plugin-refactor' into barrier-impl

    Ketan Umare committed Oct 9, 2019
    Configuration menu
    Copy the full SHA
    8f8fd5e View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2019

  1. test for best effort type too

    Ketan Umare committed Oct 10, 2019
    Configuration menu
    Copy the full SHA
    d72675f View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2019

  1. Merge branch 'plugin-refactor' into barrier-impl

    Ketan Umare committed Oct 11, 2019
    Configuration menu
    Copy the full SHA
    9128d30 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'plugin-refactor' into barrier-impl

    Ketan Umare committed Oct 11, 2019
    Configuration menu
    Copy the full SHA
    afe9561 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'plugin-refactor' into barrier-impl

    Ketan Umare committed Oct 11, 2019
    Configuration menu
    Copy the full SHA
    158956a View commit details
    Browse the repository at this point in the history

  4. double import

    Ketan Umare committed Oct 11, 2019
    Configuration menu
    Copy the full SHA
    e1b7145 View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2019

  1. Merge branch 'plugin-refactor' into barrier-impl

    Ketan Umare committed Oct 13, 2019
    Configuration menu
    Copy the full SHA
    5ce2df0 View commit details
    Browse the repository at this point in the history

  2. max duration updated to 30 minutes

    Ketan Umare committed Oct 13, 2019
    Configuration menu
    Copy the full SHA
    79f6b8c View commit details
    Browse the repository at this point in the history

  3. Merge branch 'plugin-refactor' into barrier-impl

    Ketan Umare committed Oct 13, 2019
    Configuration menu
    Copy the full SHA
    047f2ac View commit details
    Browse the repository at this point in the history

  4. Merge branch 'plugin-refactor' into barrier-impl

    Ketan Umare committed Oct 13, 2019
    Configuration menu
    Copy the full SHA
    1088dfa View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2019

  1. Merge branch 'plugin-refactor' of github.com:lyft/flytepropeller into… 

    … barrier-impl
    @EngHabu
    EngHabu committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    38c428c View commit details
    Browse the repository at this point in the history

  2. Remove best effort reference

    @EngHabu
    EngHabu committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    fdd21ff View commit details
    Browse the repository at this point in the history

  3. Merge branch 'plugin-refactor' of github.com:lyft/flytepropeller into… 

    … barrier-impl
    @EngHabu
    EngHabu committed Oct 14, 2019
    Configuration menu
    Copy the full SHA
    993258e View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2019

  1. Merge branch 'plugin-refactor' into barrier-impl

    Ketan Umare committed Oct 15, 2019
    Configuration menu
    Copy the full SHA
    0ee7831 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'barrier-impl' of github.com:lyft/flytepropeller into ba… 

    …rrier-impl
    Ketan Umare committed Oct 15, 2019
    Configuration menu
    Copy the full SHA
    5963199 View commit details
    Browse the repository at this point in the history