Skip to content

Add nightly-sweep build workflow. #172

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
xuzhao9 merged 1 commit into from
Dec 11, 2020
Merged

Add nightly-sweep build workflow. #172

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/scripts/run-docker.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,13 @@ CONFIG_VER=v0
RUN_SCRIPT=$1
CONFIG_DIR=${PWD}/score/configs/${CONFIG_VER}
CONFIG_ENV=${CONFIG_DIR}/config-${CONFIG_VER}.env
DATA_DIR=${HOME}/benchmark-results/gh${GITHUB_RUN_ID}
# Use the latest pytorch-benchmark image
TORCH_IMAGE_ID=torchbench/pytorch-benchmark:latest
if [ -z "$2" ]; then
DATA_DIR=${HOME}/benchmark-results/gh${GITHUB_RUN_ID}
else
DATA_DIR=$2
fi

# Load environment variables
set -a;
Expand Down
18 changes: 18 additions & 0 deletions .github/scripts/run-sweep-benchmark.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/bin/sh

set -xeo pipefail

for CONFIG in /output/configs/*; do
Copy link
Contributor

@wconstab wconstab Dec 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where does /output/configs come from? and what what is the format of CONFIG?

Copy link
Contributor Author

@xuzhao9 xuzhao9 Dec 11, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/output/configs is defined in as a list of "requirements.txt" files, e.g. "requirements-1.6.0.dev20200526.txt". Inside this txt file is the urls of torch/torchtext/torchvision wheels, for example, "requirements-1.6.0.dev20200526.txt" contains three lines:
https://download.pytorch.org/whl/nightly/cu102/torch-1.6.0.dev20200526-cp37-cp37m-linux_x86_64.whl
https://download.pytorch.org/whl/nightly/cu102/torchvision-0.7.0.dev20200526-cp37-cp37m-linux_x86_64.whl
https://download.pytorch.org/whl/nightly/cu102/../torchtext-0.6.0.dev20200526-cp37-cp37m-linux_x86_64.whl

# Create a new conda version from base
CONFIG_BASE=$(basename ${CONFIG})
CONFIG_VER=$(echo ${CONFIG} | sed 's/.*-\(.*\)\.txt/\1/')
conda create --name ${CONFIG_VER} python=3.7
. activate ${CONFIG_VER}
pip install -r $CONFIG
pushd /workspace/benchmark
# workaround the maskrcnn_benchmark undefined symbol problem
find . -name "*.so" -delete
python install.py
bash /workspace/benchmark/.github/scripts/run-benchmark.sh
done

11 changes: 11 additions & 0 deletions .github/scripts/run-sweep.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/sh

set -xeo pipefail

SWEEP_DIR="${HOME}/nightly-sweep"

# Run benchmark
bash ./.github/scripts/run-docker.sh \
/workspace/benchmark/.github/scripts/run-sweep-benchmark.sh \
${SWEEP_DIR}

9 changes: 9 additions & 0 deletions .github/workflows/sweep.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
name: pytorch-nightly-sweep
on:
workflow_dispatch:

jobs:
run-sweep:
runs-on: [self-hosted, bm-ec2]
steps:
- run: bash ./.github/scripts/run-sweep.sh