Skip to content

Commit

Permalink
misc YAML workflow tweaks (#668)
Browse files Browse the repository at this point in the history
  • Loading branch information
casperdcl committed Jul 27, 2021
1 parent 2cc00d4 commit b550936
Show file tree
Hide file tree
Showing 4 changed files with 23 additions and 21 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/checkbot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ jobs:
# check that CML container builds properly
build-container:
if: startsWith(github.event.comment.body, '/tests')
runs-on: [ubuntu-18.04]
runs-on: ubuntu-18.04
steps:
- name: React Seen
uses: actions/github-script@v2
Expand Down Expand Up @@ -49,7 +49,7 @@ jobs:
# test container all CML features, vega and actions with issues in the past
check-container:
needs: build-container
runs-on: [ubuntu-18.04]
runs-on: ubuntu-18.04
container: dvcorg/cml-test
steps:
- uses: actions/checkout@v2
Expand Down Expand Up @@ -128,7 +128,7 @@ jobs:
nvidia-smi
chatbot:
if: startsWith(github.event.comment.body, '/cml-')
runs-on: [ubuntu-latest]
runs-on: ubuntu-latest
steps:
- name: React Seen
uses: actions/github-script@v2
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/test-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ jobs:
fi
fi
echo "${registry}/cml:${tag}"
done | tr '\n' ',' | head -c-1
done | head -c-1
)"
- uses: docker/setup-buildx-action@v1
- uses: actions/cache@v2
Expand Down Expand Up @@ -156,7 +156,8 @@ jobs:
'schedule' }}
context: ./
file: ./Dockerfile
tags: ${{ steps.metadata.outputs.tags }}
tags: |
${{ steps.metadata.outputs.tags }}
build-args: |
CML_VERSION=${{ steps.metadata.outputs.cml_version }}
DVC_VERSION=${{ matrix.dvc }}
Expand Down
4 changes: 2 additions & 2 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ stages:
deploy_job:
stage: deploy
when: always
image: dvcorg/cml:0-dvc2-base1-gpu
image: iterativeai/cml:0-dvc2-base1-gpu
script:
- cml-runner --cloud aws --cloud-region us-west --cloud-type g2.2xlarge
--cloud-spot --labels=cml-runner-gpu
Expand Down Expand Up @@ -34,7 +34,7 @@ test_container:
when: on_success
tags:
- cml-runner-gpu
image: dvcorg/cml:0-dvc2-base1-gpu
image: iterativeai/cml:0-dvc2-base1-gpu
script:
- dvc --version
- pip install tensorboard
Expand Down
29 changes: 15 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,9 @@ name: your-workflow-name
on: [push]
jobs:
run:
runs-on: [ubuntu-latest]
runs-on: ubuntu-latest
# optionally use a convenient Ubuntu LTS + CUDA + DVC + CML image
# container: docker://dvcorg/cml:0-dvc2-base1-gpu
# container: docker://iterativeai/cml:0-dvc2-base1-gpu
# container: docker://ghcr.io/iterative/cml:0-dvc2-base1-gpu
steps:
- uses: actions/checkout@v2
Expand Down Expand Up @@ -113,7 +113,7 @@ jobs:
We helpfully provide CML and other useful libraries pre-installed on our
[custom Docker images](https://github.com/iterative/cml/blob/master/Dockerfile).
In the above example, uncommenting the field
`container: docker://dvcorg/cml:0-dvc2-base1-gpu` (or
`container: docker://iterativeai/cml:0-dvc2-base1-gpu` (or
`container: docker://ghcr.io/iterative/cml:0-dvc2-base1-gpu`) will make the
runner pull the CML Docker image. The image already has NodeJS, Python 3, DVC
and CML set up on an Ubuntu LTS base with CUDA libraries and
Expand Down Expand Up @@ -189,7 +189,7 @@ name: model-training
on: [push]
jobs:
run:
runs-on: [ubuntu-latest]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
Expand Down Expand Up @@ -254,7 +254,7 @@ name: model-training
on: [push]
jobs:
run:
runs-on: [ubuntu-latest]
runs-on: ubuntu-latest
container: docker://ghcr.io/iterative/cml:0-dvc2-base1
steps:
- uses: actions/checkout@v2
Expand Down Expand Up @@ -413,7 +413,7 @@ name: Train-in-the-cloud
on: [push]
jobs:
deploy-runner:
runs-on: [ubuntu-latest]
runs-on: ubuntu-latest
steps:
- uses: iterative/setup-cml@v1
- uses: actions/checkout@v2
Expand All @@ -428,10 +428,10 @@ jobs:
--cloud-region us-west \
--cloud-type t2.micro \
--labels cml-runner
model-training:
needs: [deploy-runner]
train-model:
needs: deploy-runner
runs-on: [self-hosted, cml-runner]
container: docker://dvcorg/cml:0-dvc2-base1-gpu
container: docker://iterativeai/cml:0-dvc2-base1-gpu
steps:
- uses: actions/checkout@v2
- name: Train model
Expand All @@ -457,17 +457,18 @@ newly-launched instance.
#### Docker Images

We like the CML container (`docker://dvcorg/cml`) because it comes loaded with
Python, CUDA, `git`, `node` and other essentials for full-stack data science.
Different versions of these essentials are available from different `dvcorg/cml`
image tags. The tag convention is `{CML_VER}-dvc{DVC_VER}-base{BASE_VER}{-gpu}`:
We like the CML container (`docker://iterativeai/cml`) because it comes loaded
with Python, CUDA, `git`, `node` and other essentials for full-stack data
science. Different versions of these essentials are available from different
`iterativeai/cml` image tags. The tag convention is
`{CML_VER}-dvc{DVC_VER}-base{BASE_VER}{-gpu}`:

| `{BASE_VER}` | Software included (`-gpu`) |
| ------------ | ----------------------------------------------- |
| 0 | Ubuntu 18.04, Python 2.7 (CUDA 10.1, CuDNN 7) |
| 1 | Ubuntu 20.04, Python 3.8 (CUDA 11.0.3, CuDNN 8) |

For example, `docker://dvcorg/cml:0-dvc2-base1-gpu`, or
For example, `docker://iterativeai/cml:0-dvc2-base1-gpu`, or
`docker://ghcr.io/iterative/cml:0-dvc2-base1`.

#### Arguments
Expand Down

0 comments on commit b550936

Please sign in to comment.