Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions docs/docs/development/building.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,9 @@ You can run the gateway with:
```bash
make serve # production-mode (Gunicorn) on http://localhost:4444
make dev # hot-reload (Uvicorn) on http://localhost:8000
make run # wrapper over uvicorn; pass --reload to enable auto-reload
./run.sh --reload # equivalent of 'make run' with explicit flags
make run # executes ./run.sh with your current .env settings
RELOAD=true make run # enable auto-reload via run.sh (same as ./run.sh --reload)
./run.sh --help # view all supported flags
```

Use `make dev` during development for auto-reload on port 8000.
Expand All @@ -59,7 +60,7 @@ Use `make dev` during development for auto-reload on port 8000.

## 🔄 Live Reload Tips

Ensure `RELOAD=true` and `DEV_MODE=true` are set in your `.env` during development.
When relying on `run.sh`, set `RELOAD=true` (or pass `--reload`) and `DEV_MODE=true` in your `.env` so settings match.

Also set:

Expand Down
4 changes: 2 additions & 2 deletions docs/docs/development/developer-onboarding.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
???+ check "Python tooling"
- [ ] `pip install --upgrade pip`
- [ ] `uv` and `uvx` installed - [install uv](https://github.com/astral-sh/uv)
- [ ] `.venv` created with `make venv install install-dev`
- [ ] `.venv` recreated with `make install-dev` (installs runtime + dev extras)

???+ check "Additional tools"
- [ ] `helm` installed for Kubernetes deployments ([Helm install docs](https://helm.sh/docs/intro/install/))
Expand All @@ -43,7 +43,7 @@

???+ check "Local setup"
- [ ] `make check-env` (validates .env is complete)
- [ ] `make venv install install-dev serve`
- [ ] `make install-dev serve`
- [ ] `make smoketest` runs and passes

???+ check "Container builds"
Expand Down
14 changes: 4 additions & 10 deletions docs/docs/development/developer-workstation.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,16 +100,10 @@ This guide helps you to set up your local environment for contributing to the Mo
### Set Up and Serve Documentation

```bash
# Create and activate virtual environment
make venv
source .venv/bin/activate # Linux/macOS
.venv\Scripts\activate # Windows

# Install dependencies
make install

# Serve documentation locally
make serve
# Build docs in an isolated environment
cd docs
make venv # first run only; installs MkDocs + plugins
make serve # http://127.0.0.1:8000 with live reload
```

## Signing commits
Expand Down
81 changes: 38 additions & 43 deletions docs/docs/development/doctest-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,28 +24,25 @@ Doctest is a Python testing framework that extracts interactive examples from do

## Coverage Status

### Current Coverage

| Module Category | Status | Coverage |
|----------------|--------|----------|
| **Transport Modules** | ✅ Complete | 100% |
| **Utility Functions** | ✅ Complete | 100% |
| **Validation Modules** | ✅ Complete | 100% |
| **Configuration** | ✅ Complete | 100% |
| **Service Classes** | 🔄 In Progress | ~60% |
| **Complex Classes** | 🔄 In Progress | ~40% |

### Modules with Full Coverage

- `mcpgateway/transports/base.py` - Base transport interface
- `mcpgateway/transports/stdio_transport.py` - Standard I/O transport
- `mcpgateway/transports/sse_transport.py` - Server-Sent Events transport
- `mcpgateway/transports/websocket_transport.py` - WebSocket transport
- `mcpgateway/transports/streamablehttp_transport.py` - Streamable HTTP transport
- `mcpgateway/transports/__init__.py` - Transport module exports
- `mcpgateway/utils/create_slug.py` - Slug generation utilities
- `mcpgateway/validation/jsonrpc.py` - JSON-RPC validation
- `mcpgateway/config.py` - Configuration management
### Current Focus

| Area | Status | Notes |
| ---- | ------ | ----- |
| Core transports & utilities | ✅ Doctest examples live directly in the modules (e.g. `mcpgateway/transports/*`, `mcpgateway/config.py`, `mcpgateway/wrapper.py`) |
| Service layer | 🔄 Many high-traffic services include doctests, but coverage is being expanded as modules are touched |
| Validators & schemas | ✅ JSON-RPC validation, slug helpers, and schema models ship with doctest-backed examples |
| Remaining modules | 🚧 Add doctests opportunistically when new behaviour is introduced |

### Key modules with doctests today

The following modules already contain runnable doctest examples you can reference when adding new ones:

- `mcpgateway/transports/base.py`, `stdio_transport.py`, `sse_transport.py`, `streamablehttp_transport.py`
- `mcpgateway/cache/session_registry.py` (initialisation handshake and SSE helpers)
- `mcpgateway/config.py` and supporting validators
- `mcpgateway/utils/create_jwt_token.py`
- `mcpgateway/wrapper.py` (URL conversion, logging toggles)
- `mcpgateway/validation/jsonrpc.py`

## Running Doctests

Expand Down Expand Up @@ -81,9 +78,17 @@ Doctests are automatically run in the GitHub Actions pipeline:

```yaml
# .github/workflows/pytest.yml
- name: Run doctests
- name: "📊 Doctest coverage with threshold"
run: |
pytest --doctest-modules mcpgateway/ \
--cov=mcpgateway \
--cov-report=term \
--cov-report=json:doctest-coverage.json \
--cov-fail-under=40 \
--tb=short
- name: "📊 Doctest coverage validation"
run: |
pytest --doctest-modules mcpgateway/ -v
python -m pytest --doctest-modules mcpgateway/ --tb=no -q
```

## Doctest Standards
Expand Down Expand Up @@ -157,41 +162,31 @@ def send_message(self, message: Dict[str, Any]) -> None:

## Pre-commit Integration

Doctests are integrated into the pre-commit workflow:
The default `.pre-commit-config.yaml` ships with a doctest hook commented out. Enable it locally by uncommenting the block (or copying the snippet below) if you want doctests to run on every commit:

```yaml
# .pre-commit-config.yaml
- repo: local
hooks:
- id: doctest
name: Doctest
entry: pytest --doctest-modules mcpgateway/
entry: pytest --doctest-modules mcpgateway/ --tb=short
language: system
types: [python]
```

This ensures that:
- All doctests pass before commits are allowed
- Documentation examples are always verified
- Code quality is maintained automatically
When enabled, the hook blocks commits until doctests pass—handy if you're touching modules with extensive inline examples.

## Coverage Metrics

### Current Statistics

- **Total Functions/Methods**: ~200
- **Functions with Doctests**: ~150
- **Coverage Percentage**: ~75%
- **Test Examples**: ~500+
- `make doctest-coverage` writes an HTML report to `htmlcov-doctest/` and an XML summary to `coverage-doctest.xml`.
- GitHub Actions currently enforces a doctest coverage floor of **40%** via `--cov-fail-under=40`.
- Use `coverage json -o doctest-coverage.json` (already produced in CI) or `coverage report` locally to inspect specific modules.

### Coverage Goals

- **Phase 1**: ✅ Infrastructure setup (100%)
- **Phase 2**: ✅ Utility modules (100%)
- **Phase 3**: ✅ Configuration and schemas (100%)
- **Phase 4**: ✅ Service classes (100%)
- **Phase 5**: ✅ Transport modules (100%)
- **Phase 6**: 🔄 Documentation integration (100%)
1. Keep transports, config validators, and request/response helpers covered with runnable examples.
2. Add doctests alongside new service-layer logic instead of backfilling everything at once.
3. Promote tricky bug fixes into doctest examples so regressions surface quickly.

## Contributing Guidelines

Expand Down
4 changes: 2 additions & 2 deletions docs/docs/development/documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,9 +145,9 @@ The `build` target produces a fully-static site (used by CI for docs previews an

## 📤 Publishing (CI)

Docs are tested, but not deployed automatically by GitHub Actions on every push to `main`. The workflow runs `cd docs && make build`.
We do not currently run a dedicated docs-build workflow in CI. Build locally with `make build` (or the `make doctest`/`make lint` suite from the repo root) before opening a PR that touches docs-heavy changes.

Publishing is done manually by repo maintainers with `make deploy` which publishes the generated site to **GitHub Pages**.
Publishing to GitHub Pages remains a manual maintainer task via `make deploy`.

---

Expand Down
34 changes: 15 additions & 19 deletions docs/docs/development/github.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ gh pr checkout 29
### 5.1 Local build (SQLite + self-signed HTTPS)

```bash
make venv install install-dev serve-ssl
make install-dev serve-ssl
```

* Sets up a Python virtualenv
Expand All @@ -202,34 +202,35 @@ make venv install install-dev serve-ssl
### 5.2 Container build (PostgreSQL + Redis)

```bash
make compose-up
make docker-prod # build the lite runtime image locally
make compose-up # start the Docker Compose stack (PostgreSQL + Redis)
```

* Spins up the full Docker Compose stack
* Uses PostgreSQL for persistence and Redis for queueing
* Rebuilds images so you catch Docker-specific issues
* Rebuilds/uses your freshly built image so you catch Docker-specific issues

### 5.3 Gateway JWT (local API access)

Quickly confirm that authentication works and the gateway is healthy:

```bash
export MCPGATEWAY_BEARER_TOKEN=$(python3 -m mcpgateway.utils.create_jwt_token -u admin@example.com --secret my-test-key)
curl -s -k -H "Authorization: Bearer $MCPGATEWAY_BEARER_TOKEN" https://localhost:4444/health
curl -s -H "Authorization: Bearer $MCPGATEWAY_BEARER_TOKEN" http://localhost:4444/health
```

Expected output:

```json
{"status": "ok"}
{"status": "healthy"}
```

If you see anything other than `{"status":"ok"}`, investigate before approving the PR.
If you see anything other than `{"status":"healthy"}`, investigate before approving the PR.

Quickly confirm that the MCP Gateway is configured with the correct database, and it is reachable:

```bash
curl -s -k -H "Authorization: Bearer $MCPGATEWAY_BEARER_TOKEN" https://localhost:4444/version | jq
curl -s -H "Authorization: Bearer $MCPGATEWAY_BEARER_TOKEN" http://localhost:4444/version | jq
```

Then proceed to register an MCP Server under Gateways using the UI, ensuring that Tools work, creating a Virtual Server and testing that from UI, API and a MCP Client.
Expand Down Expand Up @@ -311,19 +312,14 @@ Use the UI method only if reviewers are done-every push re-triggers CI.
Before requesting review, confirm that **all** required status checks on the PR page are green ✅ ("All checks have passed"). You should now see something like:

```text
Bandit / bandit (pull_request) ✅ Successful in 21s
Build Python Package / build-package (3.10) ✅ Successful in 12s
Code scanning results / Bandit ✅ No new alerts in code changed by this pull request
Code scanning results / Dockle ✅ No new alerts in code changed by this pull request
Code scanning results / Hadolint ✅ No new alerts in code changed by this pull request
Code scanning results / Trivy ✅ No new alerts in code changed by this pull request
CodeQL Advanced / CodeQL (javascript-typescript)✅ Successful in 1m
CodeQL Advanced / CodeQL (python) ✅ Successful in 1m
Secure Docker Build / docker-image ✅ Successful in ~4m
Build Python Package / python-package (3.11) ✅ Successful in ~2m
Tests & Coverage / pytest ✅ Successful in ~3m
Lint & Static Analysis / lint ✅ Successful in ~2m
Bandit / bandit ✅ Successful in ~1m
CodeQL ✅ Successful in ~1m
Dependency Review / dependency-review ✅ Successful in seconds
DCO ✅ Passed
Dependency Review / dependency-review ✅ Successful in 4s
Secure Docker Build / build-scan-sign ✅ Successful in 4m
Travis CI - Branch ✅ Build Passed
Travis CI - Pull Request ✅ Build Passed
```

If anything is red or still running, wait or push a **fix in the same PR** until every line is green. Ensure that a CODEOWNER is assigned to review the PR.
Expand Down
6 changes: 3 additions & 3 deletions docs/docs/development/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,9 +103,9 @@ echo $MCPGATEWAY_BEARER_TOKEN
Then test:

```bash
curl -k -sX GET \
curl -sX GET \
-H "Authorization: Bearer $MCPGATEWAY_BEARER_TOKEN" \
https://localhost:4444/tools | jq
http://localhost:4444/tools | jq
```

---
Expand All @@ -126,7 +126,7 @@ Key configs include:
| ------------------- | ---------------------------- |
| `DATABASE_URL` | Database connection |
| `JWT_SECRET_KEY` | Signing key for JWTs |
| `DEV_MODE=true` | Enables hot reload and debug |
| `DEV_MODE=true` | Enables relaxed development defaults (set together with `RELOAD=true` if you rely on `run.sh`) |
| `CACHE_TYPE=memory` | Options: memory, redis, none |

---
Expand Down
Loading
Loading