From 515c2cdfb1a7a9fa62aabda547ff3959f6fd4163 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 30 Oct 2025 16:55:46 +0000 Subject: [PATCH 1/3] Initial plan From d8c25b5913ab6fc6cb25ac952c0365d37a2ec6d8 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 30 Oct 2025 16:58:59 +0000 Subject: [PATCH 2/3] Add GitHub Actions workflows for CI and PyPI publishing Co-authored-by: hapticPaper <677759+hapticPaper@users.noreply.github.com> --- .github/workflows/README.md | 185 ++++++++++++++++++++++++++++++++++ .github/workflows/ci.yml | 56 ++++++++++ .github/workflows/publish.yml | 83 +++++++++++++++ 3 files changed, 324 insertions(+) create mode 100644 .github/workflows/README.md create mode 100644 .github/workflows/ci.yml create mode 100644 .github/workflows/publish.yml diff --git a/.github/workflows/README.md b/.github/workflows/README.md new file mode 100644 index 0000000..6f12e89 --- /dev/null +++ b/.github/workflows/README.md @@ -0,0 +1,185 @@ +# GitHub Actions Workflows + +This repository includes two GitHub Actions workflows for continuous integration and package publishing. + +## Workflows + +### 1. CI Workflow (`ci.yml`) + +**Triggers:** +- Push to `main` or `develop` branches +- Pull requests to `main` or `develop` branches +- Manual dispatch + +**What it does:** +- Runs tests on multiple Python versions (3.8, 3.9, 3.10, 3.11, 3.12) +- Tests on multiple operating systems (Ubuntu, macOS, Windows) +- Performs code quality checks: + - Linting with flake8 + - Code formatting with black + - Type checking with mypy + - Unit tests with pytest + - Code coverage reporting + +**Code coverage** is uploaded to Codecov for the Python 3.11/Ubuntu combination. + +### 2. Publish Workflow (`publish.yml`) + +**Triggers:** +- Automatic: When a new GitHub Release is published +- Manual: Via workflow_dispatch with option to publish to TestPyPI or PyPI + +**What it does:** +1. **Build Job**: Creates source and wheel distributions +2. **Publish Job**: Publishes to PyPI or TestPyPI depending on trigger + +**Publishing options:** +- **TestPyPI** (for testing): Manual dispatch without `publish_to_pypi` flag +- **Production PyPI**: Automatically on release OR manual dispatch with `publish_to_pypi` flag + +## Setup Requirements + +### For CI Workflow + +No setup required - works out of the box. + +Optional: Set up [Codecov](https://codecov.io/) for coverage reporting: +1. Sign up at codecov.io +2. Add your repository +3. No token needed for public repositories + +### For Publish Workflow + +The workflow uses **Trusted Publishing** (OIDC), which is the recommended secure method. No API tokens needed! + +#### Setting up PyPI Trusted Publishing + +1. **For TestPyPI:** + - Go to https://test.pypi.org + - Create an account and verify email + - Go to Account Settings → Publishing + - Add a new publisher: + - PyPI Project Name: `python-schema-manager` + - Owner: `hapticPaper` + - Repository name: `python-schema-manager` + - Workflow name: `publish.yml` + - Environment name: `testpypi` + +2. **For Production PyPI:** + - Go to https://pypi.org + - Create an account and verify email + - Go to Account Settings → Publishing + - Add a new publisher: + - PyPI Project Name: `python-schema-manager` + - Owner: `hapticPaper` + - Repository name: `python-schema-manager` + - Workflow name: `publish.yml` + - Environment name: `pypi` + +#### GitHub Environment Setup (Optional) + +For additional control, set up environments in GitHub: + +1. Go to repository Settings → Environments +2. Create two environments: + - `testpypi` - for TestPyPI publishing + - `pypi` - for production PyPI publishing +3. For `pypi` environment, add protection rules: + - Required reviewers (recommended) + - Limit to `main` branch + +## Usage + +### Running Tests + +Tests run automatically on every push and pull request to `main` or `develop`. + +To manually trigger tests: +1. Go to Actions tab +2. Select "CI" workflow +3. Click "Run workflow" + +### Publishing to TestPyPI (Testing) + +1. Go to Actions tab +2. Select "Publish to PyPI" workflow +3. Click "Run workflow" +4. Leave "Publish to production PyPI" **unchecked** +5. Click "Run workflow" + +This will publish to https://test.pypi.org for testing. + +### Publishing to Production PyPI + +#### Method 1: GitHub Release (Recommended) +1. Go to Releases +2. Click "Draft a new release" +3. Create a new tag (e.g., `v0.1.0`) +4. Fill in release notes +5. Click "Publish release" + +The package will be automatically built and published to PyPI. + +#### Method 2: Manual Dispatch +1. Go to Actions tab +2. Select "Publish to PyPI" workflow +3. Click "Run workflow" +4. **Check** "Publish to production PyPI" +5. Click "Run workflow" + +## Version Management + +Before publishing, ensure the version in `pyproject.toml` is updated: + +```toml +[project] +version = "0.1.0" # Update this before release +``` + +## Testing the Package After Publishing + +### From TestPyPI: +```bash +pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple python-schema-manager +``` + +### From PyPI: +```bash +pip install python-schema-manager +``` + +## Troubleshooting + +### Build Fails +- Check that all tests pass locally: `pytest tests/` +- Verify code formatting: `black --check .` +- Check linting: `flake8 .` +- Verify types: `mypy schema_registry` + +### Publish Fails +- **First time publishing**: Ensure Trusted Publishing is set up on PyPI +- **Version conflict**: Update version in `pyproject.toml` (PyPI doesn't allow re-uploading same version) +- **Environment protection**: Check if environment has required reviewers + +### Trusted Publishing Not Working +- Verify the publisher settings on PyPI match exactly: + - Repository owner + - Repository name + - Workflow filename + - Environment name +- Check that the workflow has `id-token: write` permission + +## Best Practices + +1. **Test before publishing**: Always test on TestPyPI first +2. **Use releases**: Prefer GitHub Releases over manual dispatch for production +3. **Semantic versioning**: Follow [semver](https://semver.org/) for version numbers +4. **Update CHANGELOG**: Update `CHANGELOG.md` before each release +5. **Review changes**: Check the built package with `twine check dist/*` locally before pushing + +## Resources + +- [GitHub Actions Documentation](https://docs.github.com/en/actions) +- [PyPI Trusted Publishers](https://docs.pypi.org/trusted-publishers/) +- [Python Packaging Guide](https://packaging.python.org/) +- [Semantic Versioning](https://semver.org/) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..52c6da7 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,56 @@ +name: CI + +on: + push: + branches: [ main, develop ] + pull_request: + branches: [ main, develop ] + workflow_dispatch: + +jobs: + test: + runs-on: ${{ matrix.os }} + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, macos-latest, windows-latest] + python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"] + + steps: + - uses: actions/checkout@v4 + + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install -e ".[dev]" + + - name: Lint with flake8 + run: | + # stop the build if there are Python syntax errors or undefined names + flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics + # exit-zero treats all errors as warnings + flake8 . --count --exit-zero --max-complexity=10 --statistics + + - name: Check code formatting with black + run: | + black --check . + + - name: Type check with mypy + run: | + mypy schema_registry + + - name: Test with pytest + run: | + pytest tests/ -v --cov=schema_registry --cov-report=xml --cov-report=term + + - name: Upload coverage to Codecov + uses: codecov/codecov-action@v4 + if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11' + with: + file: ./coverage.xml + fail_ci_if_error: false diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml new file mode 100644 index 0000000..4fd8e2c --- /dev/null +++ b/.github/workflows/publish.yml @@ -0,0 +1,83 @@ +name: Publish to PyPI + +on: + release: + types: [published] + workflow_dispatch: + inputs: + publish_to_pypi: + description: 'Publish to production PyPI (instead of TestPyPI)' + required: true + type: boolean + default: false + +jobs: + build: + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.11' + + - name: Install build dependencies + run: | + python -m pip install --upgrade pip + pip install build twine + + - name: Build package + run: python -m build + + - name: Check package + run: twine check dist/* + + - name: Upload build artifacts + uses: actions/upload-artifact@v4 + with: + name: dist + path: dist/ + + publish-test: + needs: build + runs-on: ubuntu-latest + if: github.event_name == 'workflow_dispatch' && !inputs.publish_to_pypi + environment: + name: testpypi + url: https://test.pypi.org/p/python-schema-manager + permissions: + id-token: write + + steps: + - name: Download build artifacts + uses: actions/download-artifact@v4 + with: + name: dist + path: dist/ + + - name: Publish to TestPyPI + uses: pypa/gh-action-pypi-publish@release/v1 + with: + repository-url: https://test.pypi.org/legacy/ + + publish-prod: + needs: build + runs-on: ubuntu-latest + if: github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && inputs.publish_to_pypi) + environment: + name: pypi + url: https://pypi.org/p/python-schema-manager + permissions: + id-token: write + + steps: + - name: Download build artifacts + uses: actions/download-artifact@v4 + with: + name: dist + path: dist/ + + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 From db98deb0679faf2f24c89d1f225e06906b2b558d Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 30 Oct 2025 17:01:33 +0000 Subject: [PATCH 3/3] Add explicit permissions to GitHub Actions workflows for security Co-authored-by: hapticPaper <677759+hapticPaper@users.noreply.github.com> --- .github/workflows/ci.yml | 3 +++ .github/workflows/publish.yml | 3 +++ 2 files changed, 6 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 52c6da7..9654ff5 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -7,6 +7,9 @@ on: branches: [ main, develop ] workflow_dispatch: +permissions: + contents: read + jobs: test: runs-on: ${{ matrix.os }} diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 4fd8e2c..ec7b33c 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -11,6 +11,9 @@ on: type: boolean default: false +permissions: + contents: read + jobs: build: runs-on: ubuntu-latest