Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
version: 2
updates:
- package-ecosystem: pip
- package-ecosystem: uv
directory: /
schedule:
interval: weekly
Expand Down
14 changes: 10 additions & 4 deletions .github/workflows/cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ on:
branches: [main, develop]
workflow_dispatch:

env:
_PAPERSCOUT_TESTING: "1"
SLACK_BOT_TOKEN: xoxb-ci-placeholder
SLACK_SIGNING_SECRET: ci-placeholder-secret

permissions:
contents: read

Expand All @@ -19,16 +24,17 @@ jobs:
steps:
- uses: actions/checkout@v6

- uses: actions/setup-python@v6
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
version: "0.8.3"
python-version: "3.12"
cache: pip

- name: Install dependencies
run: pip install -e ".[dev]"
run: uv sync --frozen --extra dev

- name: Run tests
run: python -m pytest tests/ --cov=paperscout --cov-fail-under=90 -v
run: uv run python -m pytest tests/ --cov=paperscout --cov-fail-under=90 -v

deploy:
name: Deploy
Expand Down
61 changes: 51 additions & 10 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,40 @@ on:
pull_request:
branches: [main, develop]

env:
_PAPERSCOUT_TESTING: "1"
SLACK_BOT_TOKEN: xoxb-ci-placeholder
SLACK_SIGNING_SECRET: ci-placeholder-secret

jobs:
lockfile:
name: Lockfile (uv)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6

- name: Install uv
uses: astral-sh/setup-uv@v6
with:
version: "0.8.3"

- name: Verify uv.lock matches pyproject.toml
run: uv lock --check

lint:
name: Lint (pre-commit)
runs-on: ubuntu-latest
needs: lockfile
steps:
- uses: actions/checkout@v6

- uses: actions/setup-python@v6
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
python-version: "3.12"
cache: pip
version: "0.8.3"

- name: Install dependencies
run: pip install -e ".[dev]"
run: uv sync --frozen --extra dev

- name: Cache pre-commit environments
uses: actions/cache@v5
Expand All @@ -28,11 +48,12 @@ jobs:
key: pre-commit-${{ runner.os }}-${{ hashFiles('.pre-commit-config.yaml') }}

- name: Run pre-commit
run: pre-commit run --all-files
run: uv run pre-commit run --all-files

test:
name: Test (Python ${{ matrix.python-version }})
runs-on: ubuntu-latest
needs: lockfile

strategy:
fail-fast: false
Expand All @@ -42,17 +63,18 @@ jobs:
steps:
- uses: actions/checkout@v6

- uses: actions/setup-python@v6
- name: Install uv
uses: astral-sh/setup-uv@v6
with:
version: "0.8.3"
python-version: ${{ matrix.python-version }}
cache: pip

- name: Install dependencies
run: pip install -e ".[dev]"
run: uv sync --frozen --extra dev

- name: Run tests with coverage
run: |
python -m pytest tests/ \
uv run python -m pytest tests/ \
--cov=paperscout \
--cov-report=term-missing \
--cov-report=xml \
Expand All @@ -61,7 +83,7 @@ jobs:

- name: Write coverage summary
if: always()
run: python -m coverage report --format=markdown >> $GITHUB_STEP_SUMMARY
run: uv run python -m coverage report --format=markdown >> $GITHUB_STEP_SUMMARY

- name: Upload coverage artifact
if: matrix.python-version == '3.12'
Expand All @@ -70,3 +92,22 @@ jobs:
name: coverage-xml
path: coverage.xml
retention-days: 7

benchmark:
name: Probe cycle benchmark (regression)
runs-on: ubuntu-latest
needs: lockfile
steps:
- uses: actions/checkout@v6

- name: Install uv
uses: astral-sh/setup-uv@v6
with:
version: "0.8.3"
python-version: "3.12"

- name: Install dependencies
run: uv sync --frozen --extra dev

- name: Run probe benchmark (mock HTTP)
run: uv run pytest benchmarks/ -m benchmark -v --tb=short
31 changes: 26 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,33 +22,54 @@ Thank you for your interest in improving paperscout. This document describes how

## Local checks

Install the package in editable mode with dev dependencies:
Install **[uv](https://docs.astral.sh/uv/)** (recommended) and sync the locked dev environment from the repo root:

```bash
uv sync --extra dev
```

This installs the project and all dev tools from [`uv.lock`](uv.lock) (see **Dependency lockfile** below). Alternatively, with a classic venv:

```bash
python -m venv .venv
source .venv/bin/activate # Windows Git Bash: source .venv/Scripts/activate
pip install -e ".[dev]"
# if using a classic venv instead of uv:
pre-commit install
pre-commit run --all-files
```

### Dependency lockfile

Runtime and dev dependencies are pinned in **`uv.lock`**, generated from [`pyproject.toml`](pyproject.toml). CI runs `uv lock --check` so the lockfile cannot drift.

**To add or upgrade a dependency:**

1. Edit [`pyproject.toml`](pyproject.toml) (`dependencies` or `[project.optional-dependencies] dev`).
2. Regenerate the lockfile: `uv lock`
3. Commit both `pyproject.toml` and `uv.lock`.

**To verify locally before pushing:** `uv lock --check`

### Tests and coverage

```bash
./run check # pytest + coverage, fails under 90% line coverage (matches CI)
# or: make check
```

CI runs `pre-commit run --all-files` for pushes/PRs on configured branches (currently `main` and `develop`; see `.github/workflows/ci.yml`).
CI uses **`uv sync --frozen --extra dev`** then **`uv run`** for tests and pre-commit (see `.github/workflows/ci.yml`).

### Lint and format (Ruff + pre-commit)

We use **[pre-commit](https://pre-commit.com/)** with **[Ruff](https://docs.astral.sh/ruff/)** for linting and formatting.

```bash
pre-commit install
pre-commit run --all-files
uv run pre-commit install
uv run pre-commit run --all-files
```

CI runs `pre-commit run --all-files` on every push and pull request (see the `lint` job in `.github/workflows/ci.yml`).
CI runs `uv run pre-commit run --all-files` on every push and pull request (see the `lint` job in `.github/workflows/ci.yml`).

Comment thread
coderabbitai[bot] marked this conversation as resolved.
## Expectations for changes

Expand Down
15 changes: 11 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,14 +1,18 @@
# Reproducible install: dependency graph from uv.lock (--frozen) with bytecode compile.
FROM python:3.12-slim AS builder

RUN apt-get update && apt-get install -y --no-install-recommends \
gcc libpq-dev \
&& rm -rf /var/lib/apt/lists/*

COPY --from=ghcr.io/astral-sh/uv:0.8.3 /uv /usr/local/bin/uv

WORKDIR /build
COPY pyproject.toml .
COPY pyproject.toml uv.lock ./
COPY src/ src/

RUN pip install --no-cache-dir .
ENV UV_COMPILE_BYTECODE=1
RUN uv sync --frozen --no-dev --no-editable

Comment thread
henry0816191 marked this conversation as resolved.

FROM python:3.12-slim
Expand All @@ -20,9 +24,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
RUN useradd --create-home --shell /bin/bash paperscout

WORKDIR /app
COPY --from=builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin
COPY --from=builder /build/.venv /app/.venv
COPY src/ src/

ENV PATH="/app/.venv/bin:$PATH" \
VIRTUAL_ENV="/app/.venv"

RUN mkdir -p /app/data && chown paperscout:paperscout /app/data

USER paperscout
Expand Down
5 changes: 5 additions & 0 deletions benchmarks/baseline.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"wall_seconds_median": 0.0243,
"regression_threshold_pct": 20,
"min_request_count": 6
}
61 changes: 61 additions & 0 deletions benchmarks/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
"""Pytest configuration for probe-cycle benchmarks (not collected with default ``tests/``)."""

from __future__ import annotations

import os
import sys
from pathlib import Path

# Benchmarks do not load ``tests/conftest.py``; mirror slack/test env so ``paperscout.config`` can import.
os.environ.setdefault("_PAPERSCOUT_TESTING", "1")
os.environ.setdefault("SLACK_BOT_TOKEN", "xoxb-benchmark")
os.environ.setdefault("SLACK_SIGNING_SECRET", "benchmark-secret")

import pytest

# Repo root so ``from tests.conftest import ...`` resolves when only ``benchmarks/`` is targeted.
_ROOT = Path(__file__).resolve().parents[1]
if str(_ROOT) not in sys.path:
sys.path.insert(0, str(_ROOT))


def pytest_addoption(parser):
parser.addoption(
"--bench-http-concurrency",
type=int,
default=5,
help="HTTP concurrency cap for ISOProber during benchmark (maps to Settings.http_concurrency).",
)
parser.addoption(
"--bench-poll-interval-minutes",
type=int,
default=30,
help="Settings.poll_interval_minutes (recorded in metrics; scheduler not exercised here).",
)
parser.addoption(
"--bench-per-request-delay-ms",
type=float,
default=0.15,
help="Simulated server delay per HEAD/GET in the mock transport (milliseconds).",
)


def pytest_configure(config):
if config.getoption("--bench-http-concurrency") <= 0:
raise pytest.UsageError("--bench-http-concurrency must be positive")
if config.getoption("--bench-poll-interval-minutes") <= 0:
raise pytest.UsageError("--bench-poll-interval-minutes must be positive")
if config.getoption("--bench-per-request-delay-ms") < 0:
raise pytest.UsageError("--bench-per-request-delay-ms must be non-negative")
config.addinivalue_line(
"markers",
"benchmark: probe cycle performance / regression (run via ``pytest benchmarks/ -m benchmark``).",
)
Comment thread
henry0816191 marked this conversation as resolved.


@pytest.fixture
def fake_pool():
"""Fresh in-memory pool (same as ``tests/conftest``)."""
from tests.conftest import FakePool

return FakePool()
Loading
Loading