Repository navigation
Contract and Schema Testing
The test design for the API contract: the OpenAPI document checked in both directions, schema-driven fuzzing, breaking-change detection, executed examples and the parity contract with the ViaFrei MCP server. Part of the Testing Strategy. Requirements: TEST-003, TEST-004, TEST-005, TEST-006, TEST-009 on Spec: Conformance and Acceptance; TST-015, TST-016 on Spec: Test Strategy; VER-001, VER-002, VER-006 on Spec: Compatibility and Versioning. The shared components are sketched on Spec: OpenAPI Skeleton.
| Contract | Between | Checked by |
|---|---|---|
| The OpenAPI 3.1 document | the API and its clients | two-way conformance, fuzzing, breaking-change diff, executed examples |
| The common envelope and problem details | every operation and every client | schema rules in the document, contract assertions (ENV, ERR) |
| The read contract | the platform's storage and the API | mapping and integration tests, readiness refusal on an incompatible version (VER-009) |
| Fact parity | the API and the MCP server | the parity harness |
flowchart LR
D[OpenAPI document] --> V[Validator]
D --> NG[Negative request generator]
D --> EX[Example requests]
HT[Handler, integration and end-to-end tests] -->|every response| V
NG -->|one violation each| SVC[Service under test]
EX --> SVC
SVC -->|responses| V
V --> R[Conformance report<br/>operation by status coverage]
Text description of the diagram: the document feeds a validator, a negative request generator and the set of example requests; every response produced by any test, and every response to generated or example requests, passes through the validator, which writes a conformance report with the coverage of operation by status.
Every response received in a handler, integration or end-to-end test is validated against the document (TEST-003):
| Test condition | Expected |
|---|---|
| Status code is declared for the operation | otherwise fail |
Every response header is declared, with its schema (rate-limit headers, ETag, Cache-Control, Retry-After, deprecation signals) |
otherwise fail |
| Body is valid against the schema for that status and media type | otherwise fail |
Data responses have exactly data, meta, links (ENV-001) with meta.requestId, generatedAt, language (ENV-002) and non-empty meta.attribution and meta.freshness (ENV-003) |
otherwise fail |
Error responses are application/problem+json with the members of ERR-001 and a type from the catalogue on [[Spec: Errors |
Spec-Errors]] |
additionalProperties is false on response schemas, so an extra member (possibly a storage name) fails |
otherwise fail |
From the document, the generator derives one invalid request per constraint of every parameter and body member: wrong type, wrong format, below minimum, above maximum, too short, too long, pattern mismatch, value outside the enumeration, missing required, unknown parameter, repeated parameter. Each must be answered with 400 invalid-parameter, listing that violation in errors[] (ERR-005), and must not reach storage. Requests combining several violations must list all of them.
Every example request in the document is sent to the service on the ci fixture set and must be answered with a documented status and a valid body. Boundary values (exact minimum and maximum) must be accepted.
The harness prints a table of operation × documented status and marks each pair that has at least one validated response. The endpoint Definition of Done (TEST-010) needs every pair covered. An empty or unparsable document is a failure, not an empty table (TST-011).
| Item | Design |
|---|---|
| Tool category | schema-driven property-based testing of a running instance (for example Schemathesis), plus native fuzz targets per parser |
| Target | only a CI-started instance on the ci fixture set, with test keys |
| Checks | no 5xx other than the documented 503 causes; every response conforms (section 2.1); no response later than 2 s plus 50 ms; no response reveals a storage name or stack trace (ERR-004) |
| Stateful links | operations that return cursors are chained, so cursor reuse with other filters is exercised (ENV-023) |
| Budgets (design) | a short run on every pull request (issue #12); a long run nightly (issue #16) |
| Findings | a reproducer is saved and becomes a unit or handler test in the fixing pull request |
The generated document is compared with:
- the document on main, on every pull request (issue #5);
- the last released document, on every pull request and on release (issue #15).
A removed operation, member or enum value, a parameter made required, a narrowed range or a changed type fails within a major (VER-002). Additions pass. The rule set is the one on Spec: Compatibility and Versioning section 2. A planted removal proves the check can fail.
The document itself is linted (Spectral with the ruleset on Spec: API Lifecycle and Governance): every operation has a summary, a description, examples, an attribution note, declared problem responses, an operation id in the house style, and x-mcp-counterpart where one exists. The served document must be byte-identical to the committed one, minus disabled operations (FR-SVC-005).
Every example in the document and in the guide is executed against the fixture instance and validated (Docs method, issue #4). A failing example fails the documentation build; examples are never edited by hand to match output.
The API must answer the same facts as the MCP server for the same question (CON-007, FR-GEN-002, QAS-FUNC-01). The harness is built in issue #18.
sequenceDiagram
participant P as MCP side
participant B as Parity bundle
participant H as Parity harness
participant A as API under test
P->>B: seed rows, questions, expected facts for platform version N
H->>B: verify checksum and version
H->>A: load seed into a throwaway database
H->>A: ask the matching endpoint for each question
A-->>H: response
H->>H: project the response to facts and compare
Text description of the diagram: the MCP side produces a bundle of seed rows, questions and expected facts for a platform version; the harness verifies the bundle, loads the seed into a throwaway database, asks the API each question, projects the answer to facts and compares them with the expected facts.
| Projection rule | Detail |
|---|---|
| Identifiers | normalised to the public id form |
| Names | exact, Unicode NFC |
| Positions | WGS 84, rounded to five decimals |
| Prices | decimals, no float conversion |
| States and lamps | codes, not labels |
| Instants | UTC |
| Source families | the set of families that contributed |
| Condition and purpose notes | verbatim, original language |
The comparison is exact on facts and ignores presentation (ordering only where the operation declares a sort, wording, envelope members). The inventory check fails when an operation marked x-mcp-counterpart has no parity case (TEST-009). The bundle is a file pinned by the version sync; the harness needs no network.
| Requirement | Level | Given | When | Then |
|---|---|---|---|---|
| ENV-003 | Contract | the ci set with an area that has no charging station |
GET charging options near that area |
200, data empty, meta.attribution lists every family consulted |
| ERR-005 | Contract | — | a request with radius_m out of range and an unknown parameter |
400 invalid-parameter with two entries in errors[]
|
| VER-002 | Contract | a branch that removes an enum value | the breaking-change check runs | the pull request is red, naming the value |
| TEST-009 | Parity | the bundle for the pinned platform version | the harness runs the departures questions | every fact equal |
ViaFrei API: a public, keyed, read-only REST API to live German mobility data. Design phase; nothing on these pages is a measurement of a running service. Data on viafrei.de and in every API answer is attributed to its publishers under their licences.
About
Design
- Domain Model
- API Design
- Architecture Overview
- Security and Access Keys
- Rate Limits and Priority
- Performance and Caching
- Configuration
- Observability
Data Model
Technical specification
-
Technical Specification
- Interface
- Scope, Conventions and References
- Protocol
- Common Envelope
- Resource Model
- Endpoints, Places
- Endpoints, Roads
- Endpoints, Energy and Parking
- Endpoints, Public Transport and Accessibility
- Endpoints, Weather Risk, Driving Rules and Composed Features
- Service Endpoints
- Errors
- Authentication and Authorisation
- Rate Limiting, Quotas and Priority
- Data Licensing and Attribution
- Compatibility and Versioning
- OpenAPI Skeleton
- Quality
- Delivery and operations
- Interface
Testing
- Testing Strategy
- Testing and Quality (design summary)
Engineering
Technology choice
- Technology Evaluation
- Architecture: Go
- Architecture: Rust
- Architecture: TypeScript
- Stack Comparison
- Benchmark Plan
Reference