Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 50 additions & 55 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,66 +19,61 @@ jobs:
python-version: ["3.8", "3.9", "3.10"]
runs-on: ${{ matrix.os }}
steps:
-
name: Checking out repository
uses: actions/checkout@v4
-
name: Setting up python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
-
uses: pre-commit/action@v3.0.1
- name: Checking out repository
uses: actions/checkout@v4
- name: Setting up python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- uses: pre-commit/action@v3.0.1

tests:
strategy:
matrix:
os: [ubuntu-latest]
python-version: ["3.8", "3.9", "3.10"]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
runs-on: ${{ matrix.os }}
needs: pre-commit
steps:
-
name: Checking out repository
uses: actions/checkout@v4
-
name: Installing poetry
run: |
pipx install poetry
-
name: Setting up python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'poetry'
-
name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.CHIMEFRB_DOCKER_USERNAME }}
password: ${{ secrets.CHIMEFRB_DOCKER_PASSWORD }}
-
name: Run services
run: |
docker compose -f docker-compose.yml up buckets mongo results -d
sleep 10
docker compose -f docker-compose.yml up -d
sleep 10
docker compose logs
-
name: Run workflow tests
run: |
poetry install --no-cache
poetry run workflow workspace set development
poetry run pytest --cov workflow/ --cov-report=lcov -s -v
-
name: Upload coverage report
uses: coverallsapp/github-action@master
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
path-to-lcov: "coverage.lcov"
-
name: Kill services
if: always()
run: |
docker compose -f docker-compose.yml down -v
- name: Checking out repository
uses: actions/checkout@v4
- name: Load cached venv
uses: actions/cache@v5
with:
path: |
~/.cache/pypoetry
key: poetry-${{ matrix.os }}-${{ matrix.python-version }}-${{ hashFiles('poetry.lock') }}
- name: Installation
run: |
pip install --upgrade pip
pip install poetry==2.1.2
poetry config virtualenvs.create true
poetry config virtualenvs.in-project false
poetry config cache-dir ~/.cache/pypoetry
poetry config virtualenvs.path ~/.cache/pypoetry/venv
poetry install
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.CHIMEFRB_DOCKER_USERNAME }}
password: ${{ secrets.CHIMEFRB_DOCKER_PASSWORD }}
- name: Run services
run: |
docker compose -f docker-compose.yml up buckets mongo results -d
sleep 10
docker compose -f docker-compose.yml up -d
sleep 10
docker compose logs
- name: Run workflow tests
run: |
poetry run workflow workspace set development
poetry run pytest --cov workflow/ --cov-report=lcov -s -v
- name: Upload coverage report
uses: coverallsapp/github-action@master
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
path-to-lcov: "coverage.lcov"
- name: Kill services
if: always()
run: |
docker compose -f docker-compose.yml down -v
6 changes: 3 additions & 3 deletions Agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ interacting with the workflow system from the command line. Pipelines deposit `W
objects into `buckets`; worker processes withdraw and execute them; results are
archived to `results`.

GitHub: https://github.com/CHIMEFRB/workflow
Docs: https://chimefrb.github.io/workflow-docs/
GitHub: https://github.com/CHIMEFRB/workflow
Docs: https://chimefrb.github.io/workflow-docs/
PyPI: `workflow.core`

## System Connections
Expand Down Expand Up @@ -144,7 +144,7 @@ Slack notification config. `WORKFLOW_NOTIFY_SLACK_*` env prefix.
```python
# Deposit (create) work in the queue
work = Work(pipeline="my-pipeline", site="chime", user="shinybrar")
work.deposit(return_ids=True) # → List[str] of MongoDB IDs
work.deposit(return_ids=True) # → List[str] of MongoDB IDs

# Withdraw (claim) work for execution
work = Work.withdraw(pipeline="my-pipeline", site="chime")
Expand Down
Loading
Loading