Repository navigation
Golden Test Cases
A golden test case (a "fixture") is a made-up household plus the result experts expect for each program. OpenBridge runs every one of them on every change, so a rule cannot quietly drift. This page explains the format, where they live, and how to run and add them.
If you do not write code, you can still make one: see Record a test case.
data/states/<code>/fixtures/<kebab-case-scenario>.json
For example data/states/oh/fixtures/family-with-children.json. Use a short, plain file name that
describes the household.
Each file is one JSON object, checked by the schema in
scripts/lib/loaders.ts:
{
"name": "Washington single adult with low wages",
"profile": {
"members": [{ "id": "self", "relationship": "self", "age": 30 }],
"incomeSources": [
{ "type": "wages", "amountCents": 100000, "frequency": "monthly", "earned": true }
],
"county": "King",
"stateCode": "WA"
},
"expected": { "snap": "likely_eligible", "medicaid": "likely_eligible" },
"expectedMembers": { "medicaid": { "self": "likely_eligible" } },
"expectedBenefits": { "snap": 11040 }
}-
name: a plain description of the household. -
profile: the household's answers, in the same shape as theHouseholdProfiletype inpackages/shared/src/types.ts. You can leave out values the engine works out itself, such as total gross income. -
expected: the expected outcome per program:eligible,likely_eligible,possibly_eligible,ineligible, orunknown. -
expectedMembers(optional): expected outcomes per person, for programs that judge each person (Medicaid and CHIP, WIC). -
expectedBenefits(optional): an expected benefit estimate in whole cents.
Use obviously made-up data: ids like self, spouse, kid1, and round amounts. Never use a real
case, even with the name removed.
Each state keeps test cases covering:
- a clear pass and a clear fail;
- a threshold boundary, just under and just over;
- a near-threshold
possibly_eligible; - categorical or adjunctive eligibility (for example, already receiving cash assistance);
- an elderly or disabled household;
- a mixed-status household;
- for states without Medicaid expansion, children eligible while parents are not.
Every change to an evaluator or an adapter adds or updates at least one test case.
From the repository root:
npm run test:fixtures # every golden test case
npm run validate:adapters # adapter checks, then the test cases againBoth run inside npm run verify and in CI. Every test case is evaluated as if today were
2024-06-01 (FIXTURE_AS_OF in
scripts/lib/fixtures.ts), so age-based results do
not change over time. A failure prints the state, the test case name, and what was expected
versus what the engine returned.
Never edit an
expectedvalue just to make a test pass. If the engine and the test case disagree, work out which one is wrong, explain why in the pull request, and get a maintainer to agree.
The app's /review page turns a made-up screening into a test case file without any coding:
- Screen a made-up household at
/screen. - Open
/review. Each program and person is pre-filled with what OpenBridge says now, worked out as of 2024-06-01 so it matches the test runner. - Change any result you think is wrong, give the test case a name, and confirm the household is made up.
- Download the file. It has exactly the format above. A benefit estimate is kept only where you agreed with the engine's outcome.
- Send it in: attach it to an "Eligibility rule correction" issue, or open a pull request that
adds it to
data/states/<code>/fixtures/.
The code is in
packages/web-app/src/features/review.
OpenBridge is a prototype. Its rules are drafts that experts have not verified yet, and results are
estimates, not decisions. It is not legal advice. Need help applying for benefits? Call 211 or visit
211.org. This wiki is edited in the wiki/ folder of the repository.
Questions: openbridge@starlingtek.com.
Start here
Use OpenBridge
- Using OpenBridge
- Understanding your results
- Guide for navigators
- Privacy and your data
- Run it on your computer
- Host it for others
- Use it with an AI assistant
Help without code
- Ways to help
- Review rules as an expert
- Record a test case
- Report a rule problem
- Translate OpenBridge
- Test accessibility
- Community rule toolkit
Rules and policy (with code)
- How the rules engine works
- State adapters
- Add or change a rule
- Add a new state
- Golden test cases
- Yearly data updates
Developers