Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Add Azure Container Instances infrastructure block #45

Merged
merged 106 commits into from
Oct 20, 2022
Merged

Add Azure Container Instances infrastructure block #45

merged 106 commits into from
Oct 20, 2022

Commits on Sep 25, 2022

  1. Added Pycharm config directory to .gitignore

    @rpeden
    rpeden committed Sep 25, 2022
    Configuration menu
    Copy the full SHA
    679ac69 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2022

  1. Initial working implementation of Azure Container Instances flow runn… 

    …er and credentials.
    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    438f453 View commit details
    Browse the repository at this point in the history

  2. Added Azure Container Instances classes to module init

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    0535d31 View commit details
    Browse the repository at this point in the history

  3. Added ACI classes to setup.py

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    e52ff9b View commit details
    Browse the repository at this point in the history

  4. Added Azure SDK dependencies to requirements.txt

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    953723f View commit details
    Browse the repository at this point in the history

  5. Update _wait_for_task_container_start return value

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    1a271d9 View commit details
    Browse the repository at this point in the history

  6. Removed field description from credentials attribute

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    f6c6bb6 View commit details
    Browse the repository at this point in the history

  7. Formatting fixes

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    038acb4 View commit details
    Browse the repository at this point in the history

  8. Added TODO to consider alternate names for flow runner class

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    5e8c710 View commit details
    Browse the repository at this point in the history

  9. Fixed import formatting

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    f3e54f8 View commit details
    Browse the repository at this point in the history

  10. Added GPU count and SKU

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    bc1d5b6 View commit details
    Browse the repository at this point in the history

  11. Fixed type in memory description

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    6e6df25 View commit details
    Browse the repository at this point in the history

  12. Updated container configuration

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    b54b3b4 View commit details
    Browse the repository at this point in the history

  13. Fixed typo in container group creation

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    85124c5 View commit details
    Browse the repository at this point in the history

  14. Removed duplicate todo

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    40783c4 View commit details
    Browse the repository at this point in the history

  15. Updated ACI infrastructure block logo

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    dd6f3f5 View commit details
    Browse the repository at this point in the history

  16. Refactored run code into multiple shorter methods

    @rpeden
    rpeden committed Sep 26, 2022
    Configuration menu
    Copy the full SHA
    ce03635 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2022

  1. Implemented _watch_task_and_get_exit_code method

    @rpeden
    rpeden committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    d64c036 View commit details
    Browse the repository at this point in the history

  2. Updated default start timeout based on observed ACI behavior

    @rpeden
    rpeden committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    725a5c1 View commit details
    Browse the repository at this point in the history

  3. Added check to ensure command is present

    @rpeden
    rpeden committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    153b558 View commit details
    Browse the repository at this point in the history

  4. Added entrypoint field because ACI overwrites the container's normal … 

    …entrypoint
    @rpeden
    rpeden committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    9ef758e View commit details
    Browse the repository at this point in the history

  5. Added container group deletion

    @rpeden
    rpeden committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    32b2150 View commit details
    Browse the repository at this point in the history

  6. Removed unused import

    @rpeden
    rpeden committed Sep 27, 2022
    Configuration menu
    Copy the full SHA
    f66fa52 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2022

  1. Add docstrings and fixed formatting

    @rpeden
    rpeden committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    f77769e View commit details
    Browse the repository at this point in the history

  2. Updated name of credentials field

    @rpeden
    rpeden committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    cead2fa View commit details
    Browse the repository at this point in the history

  3. Updated setup.py entrypoints

    @rpeden
    rpeden committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    e020e5b View commit details
    Browse the repository at this point in the history

  4. Updated module init to avoid exposing infrastructure block at top level

    @rpeden
    rpeden committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    ea6af85 View commit details
    Browse the repository at this point in the history

  5. Removed command presence check because Infrastructure base call set… 

    …s a default if missing
    @rpeden
    rpeden committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    429868b View commit details
    Browse the repository at this point in the history

  6. Added command validator and tests

    @rpeden
    rpeden committed Sep 28, 2022
    Configuration menu
    Copy the full SHA
    97f8311 View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2022

  1. Added docstring to command validator

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    81ddfd9 View commit details
    Browse the repository at this point in the history

  2. Moved verbose Azure SDK object initialization code to separate, short… 

    …er methods
    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    56cf6e0 View commit details
    Browse the repository at this point in the history

  3. Removed unused argument

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    6707587 View commit details
    Browse the repository at this point in the history

  4. Added more test code

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    543c469 View commit details
    Browse the repository at this point in the history

  5. Fixed test imports and updated fixture use

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    b5b1f6e View commit details
    Browse the repository at this point in the history

  6. Changed type to ensure compatibility with Python <= 3.8

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    272fb65 View commit details
    Browse the repository at this point in the history

  7. Added preview method and moved environment generation to method

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    870ccbc View commit details
    Browse the repository at this point in the history

  8. Updated timeout field description

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    6ddd5c4 View commit details
    Browse the repository at this point in the history

  9. Removed incorrect pass statement

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    9bf509b View commit details
    Browse the repository at this point in the history

  10. Fixed docstring typo

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    9669f43 View commit details
    Browse the repository at this point in the history

  11. Moved container and group states to enums 

    Purpose: reduce the chance of typos while checking states, and make it more difficult to accidentally use states that the Azure docs say should not be relied on as dependencies.
    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    2995687 View commit details
    Browse the repository at this point in the history

  12. Added more tests

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    2ce0008 View commit details
    Browse the repository at this point in the history

  13. Renamed module from aci to container_instance

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    ca01b49 View commit details
    Browse the repository at this point in the history

  14. Added missing docstring

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    efdaaee View commit details
    Browse the repository at this point in the history

  15. Renamed ACITask to ContainerInstanceJob

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    8f43edc View commit details
    Browse the repository at this point in the history

  16. Change gpu_count from float to int

    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    8bd4a15 View commit details
    Browse the repository at this point in the history

  17. Extract password value when calling ImageRegistryCredential constru… 

    …ctor
    @rpeden
    rpeden committed Sep 29, 2022
    Configuration menu
    Copy the full SHA
    ca8fe63 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2022

  1. Renamed container instances credentials class to match infra block

    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    dd47d76 View commit details
    Browse the repository at this point in the history

  2. Updated test fixture docstring

    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    531cfa8 View commit details
    Browse the repository at this point in the history

  3. Moved verbose provisioning success check out of run method

    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    731bec6 View commit details
    Browse the repository at this point in the history

  4. Removed unused code, updated preview method

    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    17e17d1 View commit details
    Browse the repository at this point in the history

  5. Added more tests

    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    a8a7e2c View commit details
    Browse the repository at this point in the history

  6. Removed PREFECT_API_URL assertion 

    Can't assume which Prefect env variables will or won't be set in the CI test environment
    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    f110626 View commit details
    Browse the repository at this point in the history

  7. Updated description of the stream_output field

    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    9512349 View commit details
    Browse the repository at this point in the history

  8. Updated block type name so it won't overflow in the catalog UI

    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    3d52859 View commit details
    Browse the repository at this point in the history

  9. Updated ACI credentials block type name to avoid overflow in block ca… 

    …talog UI
    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    96b9cfc View commit details
    Browse the repository at this point in the history

  10. Updated ContainerInstanceCredentials docstring

    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    9eee677 View commit details
    Browse the repository at this point in the history

  11. Updated container instance module name in setup.py

    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    14cae57 View commit details
    Browse the repository at this point in the history

  12. Updated Azure container credentials slug 

    Trying to find a balance between brevity and something that's understandable when displayed by `prefect block type ls`
    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    86d26ab View commit details
    Browse the repository at this point in the history

  13. Updated DockerRegistry import to avoid re-registering DockerRegistry 

    Before, calling `prefect block register -m prefect_azure.container_instance` registered `AzureContainerCredentials` and `AzureContainerJob` as expected, but also re-registered `DockerRegistry`, which we don't want.
    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    3a10c7b View commit details
    Browse the repository at this point in the history

  14. Removed credential TODO 

    Managed identities would be nice, but using service principal is in what we already do for other Azure services for ML. Easy to revisit this in the future if/when needed.
    @rpeden
    rpeden committed Sep 30, 2022
    Configuration menu
    Copy the full SHA
    27c51b6 View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2022

  1. Corrected test

    @rpeden
    rpeden committed Oct 1, 2022
    Configuration menu
    Copy the full SHA
    b2554de View commit details
    Browse the repository at this point in the history

  2. Added subnets to ContainerInstanceJob

    @rpeden
    rpeden committed Oct 1, 2022
    Configuration menu
    Copy the full SHA
    2751292 View commit details
    Browse the repository at this point in the history

  3. Added test for container group subnets

    @rpeden
    rpeden committed Oct 1, 2022
    Configuration menu
    Copy the full SHA
    2dd3078 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2022

  1. Added client creation test

    @rpeden
    rpeden committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    51831a7 View commit details
    Browse the repository at this point in the history

  2. Moved container entrypoint to constant, added test, added comment

    @rpeden
    rpeden committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    7b3aa60 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2022

  1. Added output streaming to ContainerInstanceJob

    @rpeden
    rpeden committed Oct 4, 2022
    Configuration menu
    Copy the full SHA
    6cb3832 View commit details
    Browse the repository at this point in the history

  2. Added method to write streaming output lines

    @rpeden
    rpeden committed Oct 4, 2022
    Configuration menu
    Copy the full SHA
    f42bd8f View commit details
    Browse the repository at this point in the history

  3. Added output streaming test

    @rpeden
    rpeden committed Oct 4, 2022
    Configuration menu
    Copy the full SHA
    1182846 View commit details
    Browse the repository at this point in the history

  4. Removed login method from Azure container credentials

    @rpeden
    rpeden committed Oct 4, 2022
    Configuration menu
    Copy the full SHA
    66562c8 View commit details
    Browse the repository at this point in the history

  5. Made subscription ID and resource group names required, updated resou… 

    …rce group property name, updated tests
    @rpeden
    rpeden committed Oct 4, 2022
    Configuration menu
    Copy the full SHA
    69bb3d6 View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2022

  1. Removed unused method

    @rpeden
    rpeden committed Oct 5, 2022
    Configuration menu
    Copy the full SHA
    4dc4653 View commit details
    Browse the repository at this point in the history

  2. Fixed check for empty log content

    @rpeden
    rpeden committed Oct 5, 2022
    Configuration menu
    Copy the full SHA
    cff5826 View commit details
    Browse the repository at this point in the history

  3. Renamed client creation method to make it more descriptive

    @rpeden
    rpeden committed Oct 5, 2022
    Configuration menu
    Copy the full SHA
    e7eee2d View commit details
    Browse the repository at this point in the history

  4. changed _get_environment method to _environment property

    @rpeden
    rpeden committed Oct 5, 2022
    Configuration menu
    Copy the full SHA
    fd5ff15 View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2022

  1. Updated test to wrap instead of replace method call

    @rpeden
    rpeden committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    e744cf8 View commit details
    Browse the repository at this point in the history

  2. Changed type field value

    @rpeden
    rpeden committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    42452c5 View commit details
    Browse the repository at this point in the history

  3. Added Optional type to entrypoint

    @rpeden
    rpeden committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    15a801a View commit details
    Browse the repository at this point in the history

  4. Added `_log_prefix property

    @rpeden
    rpeden committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    58abfb3 View commit details
    Browse the repository at this point in the history

  5. Added logging calls to ContainerInstanceJob

    @rpeden
    rpeden committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    c8aa1b5 View commit details
    Browse the repository at this point in the history

  6. Small test updates to ensure all code paths are used during test suit… 

    …e runs
    @rpeden
    rpeden committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    5b6fb41 View commit details
    Browse the repository at this point in the history

  7. Fixed container run state condition check

    @rpeden
    rpeden committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    9b9a658 View commit details
    Browse the repository at this point in the history

  8. Added check for no container group

    @rpeden
    rpeden committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    a584ea6 View commit details
    Browse the repository at this point in the history

  9. Added tests

    @rpeden
    rpeden committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    a0d1069 View commit details
    Browse the repository at this point in the history

  10. Updated method to avoid unnecessary early return

    @rpeden
    rpeden committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    bc050a9 View commit details
    Browse the repository at this point in the history

  11. Added Container Instances to docs

    @rpeden
    rpeden committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    aecac4d View commit details
    Browse the repository at this point in the history

  12. Fixed EOF newline

    @rpeden
    rpeden committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    c032e92 View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2022

  1. Apply suggestions from code review 

    Apply my own suggestions
    @ahuang11
    ahuang11 committed Oct 13, 2022
    Configuration menu
    Copy the full SHA
    871ecf5 View commit details
    Browse the repository at this point in the history

  2. Migrate subscription_id to credentials and add example

    ahuang11 committed Oct 13, 2022
    Configuration menu
    Copy the full SHA
    e30d10e View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' of https://github.com/PrefectHQ/prefect-azure int… 

    …o add_aci_infrastructure
    ahuang11 committed Oct 13, 2022
    Configuration menu
    Copy the full SHA
    3b9e3b2 View commit details
    Browse the repository at this point in the history

  4. Fix tests

    ahuang11 committed Oct 13, 2022
    Configuration menu
    Copy the full SHA
    03f053e View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2022

  1. Apply suggestions from code review 

    Co-authored-by: Michael Adkins <contact@zanie.dev>
    @ahuang11 @zanieb
    ahuang11 and zanieb committed Oct 18, 2022
    Configuration menu
    Copy the full SHA
    ebb394a View commit details
    Browse the repository at this point in the history

  2. Revise based on suggestions

    ahuang11 committed Oct 18, 2022
    Configuration menu
    Copy the full SHA
    d2ccf69 View commit details
    Browse the repository at this point in the history

  3. Remove validator

    ahuang11 committed Oct 18, 2022
    Configuration menu
    Copy the full SHA
    3aa6230 View commit details
    Browse the repository at this point in the history

  4. Draft example

    ahuang11 committed Oct 18, 2022
    Configuration menu
    Copy the full SHA
    e176b28 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2022

  1. Fix readme example

    ahuang11 committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    63ab390 View commit details
    Browse the repository at this point in the history

  2. Fix tests

    ahuang11 committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    32c1eb0 View commit details
    Browse the repository at this point in the history

  3. Deleted poller print call

    @rpeden
    rpeden committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    f6ee7cf View commit details
    Browse the repository at this point in the history

  4. Updated README examples to match current implementation

    @rpeden
    rpeden committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    1d562f3 View commit details
    Browse the repository at this point in the history

  5. Updated container instance block to use container group name as uniqu… 

    …e identifier

And updated tests accordingly
    @rpeden
    rpeden committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    22a3718 View commit details
    Browse the repository at this point in the history

  6. Add title to subnet ID field 

    Co-authored-by: Alexander Streed <desertaxle@users.noreply.github.com>
    @rpeden @desertaxle
    rpeden and desertaxle committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    79818ee View commit details
    Browse the repository at this point in the history

  7. Removed unnecessary entrypoint from setup.py 

    Co-authored-by: Alexander Streed <desertaxle@users.noreply.github.com>
    @rpeden @desertaxle
    rpeden and desertaxle committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    45bd096 View commit details
    Browse the repository at this point in the history

  8. Made polling interval description specific about time unit 

    Co-authored-by: Alexander Streed <desertaxle@users.noreply.github.com>
    @rpeden @desertaxle
    rpeden and desertaxle committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    c0bf089 View commit details
    Browse the repository at this point in the history

  9. Removed block type slug from container instance credentials block 

    Co-authored-by: Alexander Streed <desertaxle@users.noreply.github.com>
    @rpeden @desertaxle
    rpeden and desertaxle committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    58d306a View commit details
    Browse the repository at this point in the history

  10. Renamed ContainerInstanceJob to AzureContainerInstanceJob 

    Co-authored-by: Alexander Streed <desertaxle@users.noreply.github.com>
    @rpeden @desertaxle
    rpeden and desertaxle committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    fde6244 View commit details
    Browse the repository at this point in the history

  11. Renamed ACI infrastructure and credentials blocks everywhere they are… 

    … used or discussed
    @rpeden
    rpeden committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    7f3db9d View commit details
    Browse the repository at this point in the history

  12. Moved Azure client creation to credentials class and updated tests ac… 

    …cordingly.
    @rpeden
    rpeden committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    2164719 View commit details
    Browse the repository at this point in the history