Repository navigation
Spec Conformance and Acceptance
Part of the Technical Specification. Requirement prefix: TEST. Defines when an implementation conforms to this specification and how each verification method in the Spec: Traceability Matrix is carried out. The layering and quality bars are in Spec: Test Strategy; the stack-selection measurements are in Benchmark Plan. How each method is carried out in practice is elaborated by the testing document set, starting at Testing Strategy; the per-milestone entry and exit criteria are on Test Plan, and the security acceptance cases of TEST-008 on Security Test Plan.
An implementation conforms to version N of this specification when every MUST and MUST NOT requirement on the Spec pages is met and verified by its listed method, and every SHOULD that is not met is recorded with its reason in the release notes. MAY items are optional. A conformance claim names the specification revision (the wiki commit) it was verified against.
flowchart LR
R[Requirement row] --> M[Verification method]
M --> T[Test or review artefact]
T --> E[Release evidence]
E --> C[Conformance claim per release]
| Method | What it means | Where it runs |
|---|---|---|
| Unit | Pure function or component test with an injected clock and no I/O | every pull request |
| Handler | The HTTP handler against a fixture-backed storage stub, asserting status, headers and body | every pull request |
| Contract | Responses validated against the published OpenAPI document; the document diffed against the previous release | every pull request |
| Mapping | Fixture rows from the read contract mapped to API entities, compared with golden files | every pull request |
| Parity | The same fixture question asked of the API and of the MCP server; the facts compared | every pull request |
| Load | Load and soak runs on a test stack, never on production | nightly and before a release |
| Security | Automated security tests (authentication, authorisation, injection, headers, dynamic scan) | every pull request and nightly |
| Config | A configuration key set to a non-default value and the change in behaviour observed | every pull request |
| Lint | A static rule over code, OpenAPI document or repository | every pull request |
| Docs | A documentation artefact exists and its examples execute | every release |
| Review | A recorded human review with a named reviewer | as stated |
| Ops | An operational control observed in a running environment, with evidence | release and periodically |
| ID | Requirement | Verification |
|---|---|---|
| TEST-001 | Every requirement row MUST name a verification method from section 2, and every MUST row with an automated method MUST have at least one test that carries the requirement ID in its name or metadata. | Lint |
| TEST-002 | CI MUST produce a report listing every requirement ID with its tests and their result; an automated MUST row with no passing test fails the release. | Lint |
| TEST-003 | Contract tests MUST validate every response of every handler test against the OpenAPI document, including problem responses and headers. | Contract |
| TEST-004 | A breaking-change check MUST compare the generated OpenAPI document with the last released one and fail on a removed operation, member, enum value or a tightened parameter within a major (VER-002). | Contract |
| TEST-005 | Property-based tests MUST cover parameter parsing (coordinates, bbox, polyline, instants, lists) and cursor round trips. | Unit |
| TEST-006 | Fuzzing MUST run against the request parsers and the query POST body decoder; a crash or a hang over the deadline fails. |
Security |
| TEST-007 | Performance acceptance MUST use the method of [[Benchmark Plan | Benchmark-Plan]]: a fixed, production-shaped synthetic data set, an open-model load generator, warm-up excluded, percentiles from full histograms; the targets are those of NFR-PERF and NFR-CAP. |
| TEST-008 | Security acceptance MUST include: OWASP API Top 10 test cases per operation (broken object and function authorisation, resource consumption, mass assignment of query parameters), a dynamic scan of the sandbox, a header check, and the licence-gate tests of SEC-030 to SEC-032. | Security |
| TEST-009 | The parity suite MUST cover every operation that has an MCP counterpart (x-mcp-counterpart). |
Parity |
| TEST-010 | Endpoint Definition of Done. An operation is done only when all of these hold: the approved OpenAPI fragment (LCY-001); handler tests for every status code it can return; contract validation; mapping tests over the read-contract fixtures; a parity test where a counterpart exists; licence-gate tests for every source family it reads; a configuration test for each of its limits; latency within its class in the nightly load run; documentation with executed examples; a requirement row on its Spec page; an owner in the code-owners file; and an entry in the operation inventory. | Review |
| TEST-011 | Release acceptance MUST record the conformance claim, the requirement report (TEST-002), the load and security results, and the open SHOULD deviations. | Review |
| TEST-012 | A requirement that cannot be verified by its method MUST be rewritten or downgraded; an unverifiable MUST is a specification defect. | Review |
| TEST-013 | Every threat row (THR) and every OWASP API Security Top 10 (2023) risk MUST map to at least one automated test case, listed on [[Security Test Plan | Security-Test-Plan]]; a threat or risk without a test fails the release acceptance. |
ViaFrei API: a public, keyed, read-only REST API to live German mobility data. Design phase; nothing on these pages is a measurement of a running service. Data on viafrei.de and in every API answer is attributed to its publishers under their licences.
About
Design
- Domain Model
- API Design
- Architecture Overview
- Security and Access Keys
- Rate Limits and Priority
- Performance and Caching
- Configuration
- Observability
Data Model
Technical specification
-
Technical Specification
- Interface
- Scope, Conventions and References
- Protocol
- Common Envelope
- Resource Model
- Endpoints, Places
- Endpoints, Roads
- Endpoints, Energy and Parking
- Endpoints, Public Transport and Accessibility
- Endpoints, Weather Risk, Driving Rules and Composed Features
- Service Endpoints
- Errors
- Authentication and Authorisation
- Rate Limiting, Quotas and Priority
- Data Licensing and Attribution
- Compatibility and Versioning
- OpenAPI Skeleton
- Quality
- Delivery and operations
- Interface
Testing
- Testing Strategy
- Testing and Quality (design summary)
Engineering
Technology choice
- Technology Evaluation
- Architecture: Go
- Architecture: Rust
- Architecture: TypeScript
- Stack Comparison
- Benchmark Plan
Reference