Repository navigation
Spec Operations
mavrovde edited this page Nov 1, 2026
·
1 revision
Part of the Technical Specification. Requirement prefix: OPS. Method: the Google SRE practice of SLOs, error budgets, symptom-based alerting, runbooks and blameless postmortems. Targets come from Spec: Non-Functional Requirements.
| SLI | Definition | SLO (design target) | Window |
|---|---|---|---|
| Availability | good = valid requests not answered 5xx / all valid requests |
99.5 % | rolling 30 days |
| Latency | good = valid requests within their class's p99 target / all valid requests | 99 % | rolling 30 days |
| Licence integrity | licence-gate post-check violations | 0 | always |
| Freshness reporting | responses whose meta.freshness is complete |
100 % | rolling 30 days |
| Budget remaining | Action |
|---|---|
| > 50 % | Normal delivery |
| 25–50 % | Releases continue; reliability work prioritised in planning |
| 0–25 % | Only fixes and reliability work are released; a review of the burn |
| Exhausted | Feature releases frozen until the budget recovers or the owner decides otherwise in writing; postmortem required |
Alert on symptoms that hurt clients, using multi-window, multi-burn-rate rules.
| Alert | Condition (design) | Severity |
|---|---|---|
| Fast burn | availability burn rate ≥ 14.4 over 1 h and over 5 min | page |
| Slow burn | burn rate ≥ 6 over 6 h and over 30 min | page |
| Gradual burn | burn rate ≥ 1 over 3 days | ticket |
| Licence-gate violation | count > 0 | page |
| Readiness failing |
/readyz failing for 5 min |
page |
| Deadline overrun | any request observed above 2 s + margin | ticket |
| Storage near its role limits | permits saturated for 10 min | ticket |
| Usage anomaly | one key above 5× its 7-day hourly median for 1 h | ticket |
| Certificate expiry | under 14 days | ticket |
| Unknown storage codes | rising for 1 h | ticket |
# Runbook: <alert name>
- Alert: <expression and thresholds>
- Severity: page | ticket
- Requirement ids: <for example NFR-AVAIL-001>
- Impact: what clients experience
- Dashboards: <links>
- Diagnosis: ordered checks, each with the command or query and the expected result
- Mitigation: ordered actions, safest first (roll back, shed, disable an operation, raise a limit)
- Escalation: who, when
- Verification: how to confirm recovery
- Follow-up: postmortem needed? issue templateflowchart LR
D[Detect: alert or report] --> T[Triage and declare severity]
T --> M[Mitigate first]
M --> C[Communicate on the status page]
C --> R[Resolve]
R --> P[Postmortem within 5 working days]
P --> A[Action items tracked to done]
| Severity | Definition | Status page update | Postmortem |
|---|---|---|---|
| SEV1 | API unavailable or a licence or security breach | within 30 min, then hourly | required |
| SEV2 | Major degradation of an endpoint class or a tier | within 1 h | required |
| SEV3 | Minor degradation, workaround exists | as needed | optional |
# Postmortem: <title>
- Date, duration, severity, authors
- Summary: what happened, in three sentences
- Impact: clients, endpoints, tiers, error budget consumed
- Timeline (UTC): detection, key decisions, mitigation, resolution
- Root cause: the chain of causes, not a person
- What went well, what went badly, where we were lucky
- Action items: each with owner, issue link, due date, and the requirement or test it addsStated generically because the team is small and the hours are not yet decided:
- An on-call rota covers the hours the per-tier SLA promises (Spec: Developer Experience); outside them, alerts queue.
- Acknowledge a page within 15 minutes inside covered hours.
- The person on call may roll back, shed load, disable an operation or revoke a key without further approval.
- Handover notes at the end of each rotation.
| ID | Requirement | Verification |
|---|---|---|
| OPS-001 | SLIs MUST be computed from the service's own metrics, excluding 4xx and 429. |
Ops |
| OPS-002 | The error budget policy MUST be applied as written; exceptions are recorded with the owner's decision. | Review |
| OPS-003 | Alerts MUST be symptom-based with multi-window burn rates; cause-based signals are tickets, not pages. | Review |
| OPS-004 | Every alert MUST link to a runbook in the template above. | Review |
| OPS-005 | Incidents MUST be declared with a severity, communicated on the status page within the stated time, and SEV1 and SEV2 MUST get a blameless postmortem within 5 working days. | Ops |
| OPS-006 | Postmortem action items MUST be tracked as issues with owners and due dates. | Ops |
| OPS-007 | Backups of the access store MUST run daily, be encrypted, and a restore MUST be tested at least quarterly; recovery point objective 24 h, recovery time objective 4 h (design targets). | Ops |
| OPS-008 | Capacity MUST be reviewed monthly against NFR-CAP; a forecast breach within 90 days opens an issue. | Ops |
| OPS-009 | A load test MUST NEVER be pointed at production. | Review |
| OPS-010 | The operator MUST be able to disable an operation, a source family or a key within 5 minutes without a deployment. | Config |
| OPS-011 | TLS certificates MUST renew automatically and be monitored for expiry. | Ops |
| OPS-012 | Per-key usage anomalies (section 3) MUST raise a ticket, as a signal of a leaked key. | Ops |
| OPS-013 | Time on every host and container MUST be synchronised (NTP), because quota resets, cursors and freshness depend on it. | Ops |
ViaFrei API: a public, keyed, read-only REST API to live German mobility data. Design phase; nothing on these pages is a measurement of a running service. Data on viafrei.de and in every API answer is attributed to its publishers under their licences.
About
Design
- Domain Model
- API Design
- Architecture Overview
- Security and Access Keys
- Rate Limits and Priority
- Performance and Caching
- Configuration
- Observability
Data Model
Technical specification
-
Technical Specification
- Interface
- Scope, Conventions and References
- Protocol
- Common Envelope
- Resource Model
- Endpoints, Places
- Endpoints, Roads
- Endpoints, Energy and Parking
- Endpoints, Public Transport and Accessibility
- Endpoints, Weather Risk, Driving Rules and Composed Features
- Service Endpoints
- Errors
- Authentication and Authorisation
- Rate Limiting, Quotas and Priority
- Data Licensing and Attribution
- Compatibility and Versioning
- OpenAPI Skeleton
- Quality
- Delivery and operations
- Interface
Testing
- Testing Strategy
- Testing and Quality (design summary)
Engineering
Technology choice
- Technology Evaluation
- Architecture: Go
- Architecture: Rust
- Architecture: TypeScript
- Stack Comparison
- Benchmark Plan
Reference