Attach the case a failing performance number came from - #20
Merged
Conversation
The numeric suites assert golden distances at 1% relative tolerance. When one moves, the failure reports two doubles and nothing about the row that produced them, so it can't say whether the model moved or the expectation did. PerformanceCase captures the conditions, configuration, runway, and NOTAM a performance model was built from, and records them as a CSV row — on failure only, so a green run carries nothing — next to the value computed and the value wanted. The contamination suite, the tabular edge cases, and the regression offscale suite assert through it. Two tests attach a different subject. The DataTable interpolation tests attach the table they queried, since there the table is what's under test. And validateRegressionPredictions attaches the training rows whose predictions fell outside the 95% CI: it checks 3,000-odd rows behind a single failure-rate expectation and discarded every one of them. SF50 TOLDTests comes back because generateTakeoffReport is internal to the app target, and the one unit-test bundle links only the framework. The target was dropped as empty in 41fd636, but the shared scheme kept a TestableReference to its blueprint identifier; restoring it under that identifier makes the dangling reference resolve again, and adding it to the shared unit test plan is what makes CI run it. Its single test renders a takeoff TLR and attaches the HTML, so a report regression is a document you can open rather than a diff of escaped markup. Shared unit tests 386/386 on iPhone 17 Pro. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EdB27epdLAZ1x23zkSJ8LT
RISCfuture
added a commit
that referenced
this pull request
Sep 3, 2026
The numeric suites assert golden distances at 1% relative tolerance. When one moves, the failure reports two doubles and nothing about the row that produced them, so it can't say whether the model moved or the expectation did. PerformanceCase captures the conditions, configuration, runway, and NOTAM a performance model was built from, and records them as a CSV row — on failure only, so a green run carries nothing — next to the value computed and the value wanted. Every failure site that has a model in scope reports through it: the contamination suite, the tabular and regression golden suites for both aircraft types, the tabular edge cases, the RwyCC suite, and the offscale suite. Guards that covered two models at once are split so each model fails on its own, naming which one went offscale rather than reporting that one of them did. The regression suites' tuple switches over two and three models collapse the same way — one switch per model, each matching the .value / .valueWithUncertainty pair — which also retires the combinatorial case lists they spelled out. validateRegressionPredictions takes an aircraftType, because a model does not retain the type it was built for and the helper is where its failures surface. Two tests attach a different subject. The DataTable interpolation tests attach the table they queried, since there the table is what's under test. And validateRegressionPredictions attaches the training rows whose predictions fell outside the 95% CI: it checks 3,000-odd rows behind a single failure-rate expectation and discarded every one of them. ValueArithmeticTests is left alone. It exercises Value's arithmetic over bare doubles, with no model and no table to name. SF50 TOLDTests comes back because generateTakeoffReport is internal to the app target, and the one unit-test bundle links only the framework. The target was dropped as empty in 41fd636, but the shared scheme kept a TestableReference to its blueprint identifier; restoring it under that identifier makes the dangling reference resolve again, and adding it to the shared unit test plan is what makes CI run it. Its single test renders a takeoff TLR and attaches the HTML, so a report regression is a document you can open rather than a diff of escaped markup. Shared unit tests 386/386 on iPhone 17 Pro. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XgsEB3UyjyDLFn5duGQsSn
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The numeric suites assert golden distances at 1% relative tolerance. When one moves, the failure reports two doubles and nothing about the row that produced them — it can't say whether the model moved or the expectation did.
Part A — CSV attachments on the numeric suites
SF50 SharedTests/PerformanceCaseAttachment.swiftaddsPerformanceCase, built straight from aBasePerformanceModel(PerformanceCase(for: model, aircraftType: .g1)), since the base class already publishes theconditions,configuration,runway, andnotamit was constructed with. It records those as a CSV row on failure only, next to the value computed and the value wanted:Callers:
ContaminationTests(26 sites),RegressionModelOffscaleTests(20Issue.recordarms),TabularModelEdgeCaseTests(3 loops).Two tests attach a different subject:
DataTableTests—sparseDataInterpolation3D,csvParsing,minMaxValuesattach the table they queried, since there the table is what's under test. The rest of that suite operates on raw[[Double]]below the domain layer and is untouched.RegressionTestHelpers.validateRegressionPredictions— attaches the training rows whose predictions fell outside the 95% CI. It checks thousands of rows behind a single 20% failure-rate expectation and discarded every one of them.Scope note
The backlog wording asked for safety factor and VREF columns. Neither is reachable from these suites:
safetyFactorlives only in the app target'sPerformanceInput, and these tests asserttakeoffRunFt/takeoffDistanceFt/landingRunFt/landingDistanceFt. The CSV carries the inputs that are actually in scope rather than plumbing added to satisfy the wording.Part B — HTML attachment from the app target
generateTakeoffReportis internal toSF50 TOLD, andSF50 SharedTestslinks only the framework, so no existing bundle could reach it.SF50 TOLDTestsis restored. It was dropped as empty in 41fd636, but the shared scheme kept aTestableReferenceto blueprint identifier0163BC782DDEE9D4008FEE8F, which existed nowhere in the project. The target is restored under that identifier, so the dangling reference resolves without touching the scheme; it is added toSF50 Shared Unit Tests.xctestplan, which is what makes CI run it. Its single test renders a takeoff TLR and attaches the HTML — a report regression becomes a document you can open rather than a diff of escaped markup.Verification
swiftlint --strictandswift-format lint --strictclean on every changed file. (swiftlint --strictreports 5legacy_swiftui_aspect_ratioerrors repo-wide, all inSF50 TOLD/Views/*files this branch does not touch — they are onmaintoo.)Shared unit tests 386/386 on iPhone 17 Pro, iOS 26.
Attachments were proven to land by temporarily breaking expectations and exporting the result bundle:
ContaminationTests.waterContamination_increasesLandingRun_tabularG1performance-case_0_….csv, 301 bytes, header + the input row aboveDataTableTests.minMaxValuesdata-table_0_….csv, the three-row tablevalidateRegressionPredictionsthreshold forcedtakeoffGroundRun missed rows_0_….csv, weight/altitude/temperature/expected/predicted/uncertaintyTakeoffReportTeststakeoff-report_0_….html, 12,371 bytes, a full<!DOCTYPE html>TLRAll expectations were restored and the suite re-run green before committing.
🤖 Generated with Claude Code
https://claude.ai/code/session_01EdB27epdLAZ1x23zkSJ8LT