Skip to content

Test Data Management

mavrovde edited this page Nov 1, 2026 · 2 revisions

Test Data Management

How test data for the ViaFrei API is produced, used, versioned and kept lawful. Part of the Testing Strategy; requirements TST-009 and TST-014 on Spec: Test Strategy. The generator is built in issue #13.

1. Rules

Rule Why
All test data is synthetic: produced by the fixture generator from a seed reproducible results; no licence or privacy question
It is production-shaped: the read-contract columns, types, value ranges, distributions and volumes of a real Germany-wide data set tests and benchmarks see realistic plans and sizes
It is fictional: names and identifiers are generated, never copied from a real list or from production no third-party rights, no personal data
It contains no secret and no real key; test keys are generated per run with the vf_test_ prefix SSC-017
It is versioned with the code that reads it: the generator version, seed and profile are recorded in a manifest a failing test can be replayed exactly
No test copies data from production or from a live data source, not even "just once to get the shape" CON-009, TST-010

2. The generator

flowchart LR
  S[Seed and profile] --> G[Fixture generator]
  R[Read-contract stub<br/>column types and constraints] --> G
  L[Licence class catalogue<br/>fictional families] --> G
  G --> D[Seed rows per view]
  G --> M[Manifest<br/>counts, edge cases, checksum]
  D --> T[(Throwaway PostGIS)]
  M --> C[Coverage check]
Loading

Text description of the diagram: a seed and a profile, the read-contract stub and the catalogue of fictional source families go into the generator, which writes seed rows and a manifest; the rows are loaded into a throwaway PostGIS database, and the manifest feeds a coverage check.

Profiles

Profile Used by Size (design target) Load time (design target)
tiny unit, handler and most integration tests tens of rows per view well under 1 s
ci integration, contract, parity, end to end, fuzzing thousands of rows per view, every edge case a few seconds
benchmark the [[Benchmark Plan Benchmark-Plan]], the [[Performance Test Plan Performance-Test-Plan]]

Germany-wide shapes

Domain What the generator produces
Places places in all 16 Länder, with administrative codes in the right format, population classes, postcodes in valid ranges, addresses and points of interest; duplicates of one name in different Länder to exercise 422 ambiguous-place
Roads a motorway network with numbered roads, junctions and kilometre posts; situations and roadworks distributed over daily time partitions, with a skewed distribution (busy corridors, quiet roads)
Energy fuel stations with prices as decimals, charging stations with connectors and availability states, hydrogen stations
Parking facilities with capacity and occupancy over time
Public transport stops, departures with planned and real-time instants, disruptions, station facilities with states
Weather warnings with areas, severities and validity windows
Rules curated rule entries with review dates

Positions are generated inside the Germany bounding polygon, clustered around generated settlements so that "near" and "along" queries have realistic densities.

Edge cases planted in every ci and benchmark set

  • Both daylight-saving transitions of the year, UTC midnight and Berlin midnight, month and year ends (TST-005).
  • Names with umlauts, ß, hyphens and long compound words; the maximum lengths of every text column.
  • Points on the outer edge of the bounding box and on Land borders.
  • An empty area, an empty corridor, a motorway with no situations.
  • A stale family for each freshness category (NFR-FRESH), so that data_stale and lamp unknown are reachable.
  • An unknown code value in each coded column (ENV-051, QAS-MAINT-02).
  • A planted forbidden row for the licence post-check test (SEC-032), present only in the profile variant that test loads.
  • Prices with the maximum decimal precision; zero and very high occupancies.

Source families and licence classes

The generator does not reuse real publishers or their terms. It creates fictional source families that together cover every licence class the licence gate knows, so the gate matrix is complete without any real licence text:

Licence class Fictional family example Exercises
Attribution only family-alpha LIC-001, LIC-002
Share-alike family-beta LIC-005
Restricted purpose family-gamma LIC-006, SEC-030
Terms acceptance required family-delta SEC-034
Not permitted for any plan family-epsilon default deny

Real attribution texts belong to the production register; tests of the real register check its shape (non-empty, every family has one) and are run against the register file, not against fixture data.

Manifest

Every generated set carries a manifest with: generator version, seed, profile, row count per view, per Land and per family, the list of edge cases present, the licence of every input, and a checksum of the output. The coverage check fails if a Land, a family, a licence class or a named edge case is missing. The generator fails closed: an unknown profile, an empty output or a family with zero rows is an error, and the success line prints the counts written (TST-011).

Implementation in 0.2.0

The generator lives in internal/fixtures, and cmd/fixturegen runs it (docs/test-data.md). Where the first version differs from the design above:

Design 0.2.0
Germany bounding polygon, real Land borders The bounding box of Germany, cut into sixteen synthetic Land cells (a 4 x 4 grid) with official-format codes and generated names.
Five families, alpha to epsilon Seven families. zeta (zero) and eta (official) are added so that every stored licence category has rows. epsilon stores an ordinary licence with an empty list of use classes, which no plan matches.
Edge cases only in ci and benchmark Every profile, tiny included, plants all 18 named edge cases. The manifest counts each one.
An unknown code in each coded column One unmapped value (x_unmapped) in the main coded column of each domain: event kind, open state, availability, occupancy status, call status, facility status and hazard.
Addresses and points of interest Not yet. The 0.1 read-contract stub has no such tables. They are added together with the tables.
Maximum lengths of every text column Not yet. The long compound name is the one long-text case.
Planted forbidden row (SEC-032) Arrives with the licence post-check.
One generator module per domain page One file with one function per domain.

The success line prints the rows, tables, Länder, families and edge cases written, plus the checksum. seed.sql carries no transaction statements, so a test can load it inside a transaction and roll it back.

3. Licences of sample data

  • All generator inputs (syllable lists, distributions, templates) are our own work and are published under CC0 1.0.
  • If a test ever needs a sample that is not our own (for example a public reference geometry), its licence, source and attribution are recorded in the manifest and the sample must be licence-compatible with publication in a public repository. Data under share-alike or non-commercial terms is not used as a fixture.
  • The fixture files are not data products and are not offered as such.

4. Use in tests

Level Data
Unit values built in the test itself, or the tiny profile through a stub
Handler fixture-backed stub stores from the tiny profile
Integration a throwaway PostGIS from the read-contract stub, loaded with tiny or ci
Contract and fuzzing ci
Parity the parity bundle's own seed rows, pinned per platform version ([[Contract and Schema Testing
Performance and chaos benchmark

Each test that loads a set prints its seed on failure, so the failing run can be reproduced (TST-004). Tests never modify a shared set; each run starts from a fresh database.

5. Keys and secrets in tests

  • Test keys are generated per run, use the vf_test_ prefix and are discarded with the environment.
  • The pepper and every other secret in tests are generated per run.
  • Secret scanning runs on the repository including fixtures; a fixture that matches a secret pattern fails the pull request.

6. Data model reference

The shapes the generator produces follow the logical Data Model: one generator module per domain page (Places, Roads, Energy, Parking, Transit, Accessibility, Weather, Provenance), with value ranges, formats and nullability taken from the Data Dictionary. Mapping tests compare the generated rows with the API shapes defined on Data Model Mapping. A change to a data-model page that adds an entity or a field adds it to the generator and its coverage check in the same change. The public field names of responses come from the Domain Model, never from storage (DEV-044); the generator writes the read-contract shape described on Architecture Overview.

Home

About

Design

Data Model

Technical specification

Testing

Engineering

Technology choice

Reference

Clone this wiki locally