Testing Strategy

Testing-Strategy

fiscalclock's results are checked by several independent methods whose expected values never come from fiscalclock itself, and this page lists what runs today and what is planned.

Last reviewed: 31 Oct 2026

The rules behind this page are in AGENTS.md ("Correctness workflow"). Every check marked built runs in CI on every pull request; the workflows are ci.yml and docs.yml.

Summary

Method	Status	What it catches
Conformance vectors	Built	Wrong results for cited or independently computed cases, in any language
Generator drift check	Built	Vectors edited by hand or out of step with their generators
Differential test, 1900 to 2200	Built	Boundary arithmetic that disagrees with a day-by-day walk
Property tests	Built	Gaps, overlaps, broken round trips, wrong year lengths
Spec rule tests	Built	A rule stated in the spec's prose that changes without notice
Cross-binding byte identity	Built	Any binding or platform returning different JSON
Timestamp rejection tests	Built	A binding accepting a timestamp as a date
Claim verification script	Built	A date-arithmetic claim in the documents that does not hold
Fuzzing	Planned, Gate C	Panics, hangs or unbounded allocation on malformed input
Benchmarks	Planned, 0.2	An unmeasured performance claim
Second implementation	Planned, 1.0	Errors shared by fiscalclock and its own generators

Built in 0.1.0

Conformance vectors

conformance/vectors/ holds 112 JSON files with 4,070 cases. Each file has a definition, inputs and expected outputs, and states where its expected values come from. The Rust, Python and JavaScript suites load the same files.

Group	Files	Source of expected values
oracle	62	Generators in conformance/generators/, written with the Python standard library and not importing fiscalclock. The ISO vector uses date.isocalendar() and date.fromisocalendar().
cited	2	A published source quoted verbatim: NRF's list of 53-week years and Apple's Form 10-Q for the quarter ended 30 December 2023.
spec	4	Worked examples from the spec and README.
errors	44	Definitions and arguments that must fail with a given code and path; every validation code in the catalogue has at least one.

Expected values are never produced by running fiscalclock, and an existing expected value changes only with a linked issue that explains why. The schema for a vector file is vector.schema.json.

Every generated vector is reproducible: the verification job in docs.yml runs check.py, which regenerates the vectors and fails if the output differs from the committed files.

Oracles

An oracle is an independent source of expected results.

Oracle	Status	How it is used
ISO 8601 week dates (Python date.isocalendar())	Built	iso-week-year.json locates 28 December and 4 January for every year from 1600 to 2399; check C15 compares year bounds for the same 800 years.
NRF published calendar	Built	nrf-53-week-years.json, citing NRF.
SEC filings	Partly built	Apple only. Vishay, Costco, Kroger and Care.com vectors are planned for Gate C.
Independent Python generator	Built	fiscal_oracle.py recomputes week-based and fixed-date calendars without sharing code with the core.
pandas FY5253, fiscalyear, broadcast calendar	Planned	Named as oracles in the research proposal; not yet wired in.
ThreeTen-Extra AccountingChronology	Planned, 1.0	See "Second implementation" below.

Contradiction D5: the proposal's summary says "Five oracles", but its oracle table lists seven (it adds fiscalyear and the broadcast calendar). Contradiction D8: the gap analysis says every claim was checked against a primary source, but some of its sources are Wikipedia and a forum post. The Evidence page classifies each source.

Differential test against a reference implementation

differential.rs compares the core with a deliberately slow reference in tests/reference/ that walks the calendar one day at a time. For every generated definition, under each of the three year-numbering rules (start, end, majority), both must accept or refuse the rule alike, and where both accept, locate must agree for every date from 1900-01-01 to 2200-12-31 (109,938 days). The test asserts that at least 600 definitions are generated. The release build checks all of them; debug builds check every eighth, to keep cargo test quick.

The reference stays slow and obvious and shares no code with the core's boundary arithmetic (AGENTS.md, correctness rule 4).

Property tests

properties.rs uses proptest with 512 cases per property over random valid definitions and dates from 1900 to 2200.

• Tiling and nesting: located periods contain the date and nest inside each other; each level's children exactly cover their parent; consecutive years are contiguous.
• Round trips: shift(n) then shift(-n) returns the same period, for n from -120 to 120; parse(format(p)) returns p.
• Week-based rules only: every year has 364 or 371 days, has_extra_week is true exactly for 371-day years, and any 400 consecutive years hold exactly 71 long years.

The 364/371 and 400-year properties hold only for rule-only week-based definitions, because 400 Gregorian years are exactly 20,871 weeks (checks C01 and C02). They will not apply to definitions with exceptions or table years when those arrive in 0.2; such definitions will be tested for tiling and agreement with their listed boundaries instead.

The proposal also plans the same properties in Hypothesis (Python) and fast-check (JavaScript). In 0.1.0 the property tests run in Rust only; the bindings are covered by the conformance vectors and byte identity.

Spec rule tests and other Rust tests

• spec_rules.rs: one test per rule that the spec states in prose and no vector pins, so a change to the rule fails here as well as in review.
• validation.rs: error codes and paths for invalid definitions.
• json_api.rs: argument errors, including INVALID_DATE for strings with a time or zone suffix.
• known_values.rs: NRF and Apple years, the ISO Thursday rule for every day from 1900 to 2200, and range edges.

Cross-binding byte identity

Each language's conformance runner writes the raw JSON of every case to a transcript, and the identical-output job compares 14 transcripts (every operating system and runtime in the CI matrix) with Rust's, byte for byte. See Architecture and ADR-0019.

Timestamp rejection

test_api.py proves that Python rejects datetime.datetime and pandas Timestamp, and api.test.js proves that JavaScript rejects Date and the Temporal timestamp types, each with TIMESTAMP_REJECTED. The release smoke tests repeat the Python check on the built wheel. See Semantics.

Claim verification script

research/verification/verify_claims.py reproduces every date-arithmetic claim in the research documents (checks C01 to C18, plus the correction C12a), using the Python standard library only. The verification job compares its output with expected_output.txt byte for byte. These checks test claims in prose; the vectors test the library.

Planned

Method	Planned for	Plan
Fuzzing	Gate C (0.2)	The definition parser and label parser fuzzed with cargo-fuzz for at least 24 CPU hours in total, with no panic, hang or unbounded allocation. Today the test suite feeds malformed input to every operation and checks that it never panics (core README), but that is not fuzzing.
Benchmarks	0.2	Single-date latency and million-date throughput per binding, so the claim that the bulk path is faster is measured rather than assumed.
Proration properties	Gate C (0.2)	Shares sum to exactly 1; allocated amounts sum to the input, in every binding.
Cited filer vectors	Gate C (0.2)	Vishay, Costco, Kroger and Care.com, each quoting its filing; Care.com expects RULE_CONFLICT with the affected years.
Restatement and equivalent-day vectors	Gate D (0.3)	NRF's published restatement; both equivalent-day modes.
Schema validation of every vector	0.2 and Gate E	Every vector's definition validates against the JSON Schema generated from the Rust types.
Second implementation	1.0	ThreeTen-Extra's AccountingChronology run as a differential test in CI.