Releases: atulkapoor/fde-framework
Release list
0.1.34
- The emitted harness never raises on what the pipeline returned
(issue #2, reported with a clean reproduction). Single-field outputs
put the exam on the decision path, and a scaffold pipeline that hands
back its intermediate envelope -- a dict that is not a label -- reached
in knownas an unhashable value and the harness died with a
TypeError instead of scoring the case wrong. Predictions are now
reduced through one total function: a string is itself, anything else
is a single marker that is never a known label and never an
abstention, so the golden layer scores 0.0% and the report says
250 EUR -> <not a label>. Pinned by a test that builds the reported
shape and runs its harness. - The site's sitemap listed a page that did not exist and carried a
stale date; both fixed. PyPI and llms.txt now link the site.
0.1.33
- Stop conditions alone are not an outcome contract. Recording them
created the contract file, which changed the eighth gate's reason from
"no outcome contract was recorded" to "the contract lacks owner, ...",
and a waiver granted against the first reason lapsed -- the banking
record dropped from production to discovery the moment its stop
conditions went on. The gate now reads a file with only stop
conditions as no contract, pinned by test. - The bench and the README carry the banking record after its stop
drill: stopped once by its own condition at the default margin, cleared
at the shipped 1.0-nat margin, 90.2% on the answered.
0.1.32
Stop is a legitimate outcome, and an engagement that cannot say what
would stop it cannot be stopped by evidence.
- Stop conditions.
fde stop-whenrecords, on the outcome contract,
what evidence would stop the engagement --answered_accuracy < 0.88,
abstain_rate > 0.25,adoption < 0.4-- over figures the record
measures: the scorecard's out-of-sample rows, the field journal, the
outcomes recorded in the field. Each is judged against what was
measured; a figure not on the record leaves its condition unjudged and
says so. A triggered condition makes STOP the engagement's stage, on
the record with the trigger and the threshold, until the condition is
restated with a reason, the build is changed and scored again, or the
case is captured. - Sessions carry their date. Every interview, frame and samples
session now records the day it was taken, andfde debtages a fact a
person said where a measurement was possible from that day. - The diagram is drawn from the code.
assets/how-it-fits.pyreads
the gate count off the gates module and the registry counts off
framework/, so the image cannot drift from the repository again.
0.1.31
fde debtno longer lists a standing waiver as blocking production. A
waiver is accepted risk with a name and a date on it and ships in
RISKS.md; the stage proceeds past it. It stays on the debt list because
it stands in for a condition, and it ages. The banking demo showed the
contradiction:fde stagesaid production whilefde debtsaid two
items blocked it.
0.1.30
A second outside reading, this time of the source, agreed with the
README's own verdict -- built and demonstrated, not proven -- and named
the gap that is not another feature: the causal chain from a business
outcome to an architecture, and the assumptions an engagement quietly
rests on. This release makes both legible on the record.
- The eighth gate: an outcome contract.
fde outcome-contract
records who owns the number the system exists to move, what it is,
its value today, its target, how it is measured and over what window.
Nothing builds until that is on the record or waived with a reason that
ships in RISKS.md; a target nobody set is refused rather than filled
in. The adoption stage reads the contracted metric back from what was
measured in the field. - Decision debt.
fde debtlists everything the engagement rests on
that nobody has settled: a gate still failing, a waiver standing in for
a condition, a fact the framework guessed or a person merely said where
a measurement was possible, two people disagreeing, an attestation with
nobody's name on it, a role never asked, an incident open, a component
nothing serves -- each with an owner, what it blocks, and its age. fde nextsays what hangs on the question. When the next move is
an interview question, the command prints the evidence already on
record for that dimension, every candidate answer tried as the
framework's own guess, and the decisions that turn on it.- The claim ladder. The README's status is now six claims, each
marked verified, partial or unverified with the evidence and where it
stops, so the project cannot be over-read in either direction. - The example walkthroughs and their pinned transcripts carry the eighth
gate.
0.1.29
The operating loop from 0.1.28 has now run in public: on the banking
demo, two field streams through the delivered build, a drift incident
opened and closed on the record, a value document. This release adds
the layers around it that could be built honestly without a client.
- The stakeholder map.
fde stakeholdersreads the engagement's
people off the record: which of the five roles has been heard (every
session carries the role and, when given, the name), who signed what,
which roles were never asked, and what is on the record with nobody's
name on it.fde stakeholder addnames the people who have not spoken
yet. It is a map, not a contact list. - Names on the record.
--byondata-access,security-review,
waive,deployed,outcomeandincident closerecords who signed;
the entry is honest state without it, and the map points at it. - Exports as the connector.
fde importturns a client export --
csv, tsv, jsonl, json, the client's column names -- into the pairs the
intake reads, and reports what it kept, skipped, dropped as duplicate
and counts as verified. Nothing is verified unless the caller says
which column and value means a person checked it. Live connectors are
not here: an export is what a client can hand over inside their own
boundary, and the only interface testable without their credentials. - A bench, not a benchmark.
fde benchreads the same figures off
every engagement record side by side -- stage, the out-of-sample rows,
the gap, incidents, days to pilot -- and writesBENCH.md, which says
how many rows it has. The repository's ownBENCH.mdis the four
public demos. - The history page.
fde historyprints every dated entry on the
record in order, the undated ones above it, one line each.
0.1.28
An outside reading of the repository put it plainly: a strong engineering
framework, not yet an operating system for the engagement, because the
record stopped at the build. This release carries the record through
deployment, the field, and the client's figures. Nothing in it is
declared by the tool; each fact is computed from evidence or attested by
a named person.
- The stage is computed, never declared.
fde stagereads the record
against per-stage criteria -- discovery, validation, prototype, pilot,
production, adoption, retrospective -- and appends each transition to
lifecycle.jsonlwith its evidence. Production needs a deployment on
record and no open incident; adoption needs a figure measured in the
field; retrospective needs a case.fde outcomesprints what the trail
shows: transitions, days to pilot, loop rounds, reversals, incidents. - The field is read against the exam.
fde driftreads the deployed
service's journal -- theansweredlines it already writes -- and
compares abstention, decision mix, errors and margins with the golden
set and the last scorecard. Past a threshold it opens an incident on
the record and exits 1; an open incident pulls production back to
pilot untilfde incident closerecords what was done. Fewer than
thirty events is a sample of nothing, said as such. - Value in the client's own figures.
fde valuewritesVALUE.md
from the recorded baseline and the holdout row: automated share,
accuracy on the automated, residual human work, errors added and
removed, hours and money, build and run cost, payback, a Wilson
interval on the accuracy. Every line carries its basis -- measured,
stated, assumed, derived -- and the document names the stated and
assumed lines before any total. - Attestations, not inferences.
fde deployedandfde outcome
record where the system runs and what the client measured; the
lifecycle reads them and nothing infers them from a build. - The README says what the operating loop still is not: a stakeholder
graph, enterprise connectors, a benchmark of engagements. Those need
engagements that have not happened yet.
0.1.27
The ninth pass reproduced every number 0.1.26 printed and showed what
they hid: the scorecard measured self-consistency, not fitness, and the
banking deliverable routed a greeting to the commonest queue with a
0.02-nat margin. Each finding is a check before it is a fix.
- The scorecard has fitness rows. A generalisation-gap row (golden
in-sample minus holdout, capped at twenty points); an external-exam row
(--external <jsonl>, a second out-of-sample set nobody at the
engagement chose -- the row a component that memorises the golden and
holdout files cannot pass); a "beats the baseline error rate" row read
from the engagement's recorded baseline; a valid request through the
edge, and whether the answer says why; readiness judged where nothing
external is needed; a regression row against the last card; floors on
the edge and adversarial layers. The card says which rows are fitness
and which are self-consistency, and that the service was booted on the
measuring machine, not in the unit. - The baseline abstains and explains. Below a top-two margin
(ABSTAIN_MARGIN, default 0.5 nats) the labelled-decision baseline
answersunknownand says it abstained; the harness reports the
abstained share and the accuracy on what was answered; every routed
answer carries the top labels with scores, the margin and the tokens
that carried it, in the response and in the journal. A build with no
model seam no longer turns every valid request into a 500 when a model
endpoint sits in its environment. A scaffold answers 501 by name, never
a bare 500. - A label is stripped only where it is dictated. Quoted, braced, or
after an instruction cue. Stripping every occurrence cost two points on
real customers who were stating the intent in their own words. - The loop's fence covers the tests and the contract. An agent that
deleted the lint test and hollowed outapp/contract.pyonce reported
green with no violation. The shipped baseline's holdout is measured
before any round, and a round that lands below it is refused: it
traded generalisation for the exam. - The decision shape survives one rare label. Nine in ten pairs
carrying a repeated value is a label set; a singleton intent is learned
from, never held out. A definition that says "stated" is marked stated. - The banking run, again, under these rules. Rebuilt on 0.1.27 the
shipped baseline abstains on 16% of the holdout and answers 87.9% of the
rest correctly, a tenth of a point under the bank's recorded 88%
first-pass accuracy; the implement loop, fenced and guarded, took the
holdout from 73.7% to 77.5% in two rounds. At a one-nat margin the
finished deliverable answers 90.2% of what it routes correctly and hands
18% to a person: the engagement's bar is met in assist mode and the card
says so, with the vendor's own test split as the external exam beside
it. It is not an autonomous router, and nothing on the card claims it is.
0.1.26
Production grade, measured, and a first industry use case run end to end.
fde scorecard <project> [--holdout <jsonl>]. One command runs
what a deliverable can prove about itself and writesSCORECARD.md
beside it: its own tests, lint, every exam layer against its majority
rate with the in-sample layer marked, the harness's own verdict, judge
calibration, the holdout with the acceptance protocol's sample floor
and the digest on record, the exam record, the edge probed by booting
the service (identity, forged result, forged identity, malformed body,
readiness), the risk register, the environment, the training path.
The verdict is a count of rows -- "16 of 17 measured properties hold"
-- and a property the build cannot measure is n/a, never a pass.- The out-of-sample CI lane. A second job scores the engagement's
holdout where it lives, on a self-hosted runner that holds the file at
the pathHOLDOUT_PATHnames; until it is configured the golden score
is the only one CI sees, and the README says so. - A text decision with a labelled history is labelled-decision at any
labelled count. Classical ML avoids free text: routed there, a text
decision met a component that asked for numeric features. The label
set may run to five hundred. - What routing a retail bank's support intents found. A one-field
output was read as a decision only up to five distinct values, so
seventy-seven intents built with no reasoning component and an exam at
0.0%; a label set is now recognised by repetition. "Routed to",
"triage" and "by intent" read as a decision. A label joined by
underscores is stripped the way it tokenises (an injection naming
card_payment_fee_chargedwas followed until it was). The
role-scoped-authority template failed lint, and no acceptance shape had
ever emitted it: a routing shape joins the suite (five shapes now). - What the implement loop found on the same run. The loop scored the
holdout against the golden bar, and the golden score is in-sample
wherever the baseline is fitted on it: an implementation that had
raised the holdout by three points was refused as "memorised". The
holdout is now scored against the harness's own gate (the majority
rate, the exclusive half-right floor) with the golden bar stripped. And
the loop's check was the harness alone, so a round could leave a lint
error behind a green exam: the deliverable's own tests -- now including
lint wherever ruff is installed -- run first, as the floor beneath the
harness. Under the corrected gate the loop finished green in two rounds,
andfde scorecardreports 17 of 17 measured properties holding: golden
95.4% in-sample, holdout 81.3% on 3,036 cases never shipped, 80.2% on
the vendor's own 3,079-case test split, every adversarial probe passed.
0.1.25
The eighth pass gave the decision shape its first sign-off, with
conditions, and refused the fine-tuning path on named ones. It also
found that two 0.1.24 fixes were shaped to the one demo. Each is a
check before it is a fix.
- A label written out in full is not evidence; its words may be.
Dropping every word of every label from the vocabulary cost a
refund/escalate/reply corpus a third of its accuracy, because there the
label word is the cue. The full label phrase is stripped from the text
instead; single words stay evidence. The docstring says what a
bag-of-words baseline cannot defend: content that repeats a label's
strongest cues. - Probe bases are typical cases, one per label, at the median length.
Built on the two shortest inputs, every probe on the demo sat on a case
the baseline misreads, and "0 injections followed" read as a pass while
measuring nothing. The bases ship in the edge layer (so each is scored
un-steered) and leave golden. When every base is still misread, the
harness says no probe was scorable rather than printing a takers count;
a steered probe answered with a third label counts as wrong under
mutation, not as a misread. - The merge path trained one epoch and saved zeros. Merging inside
the loop left the optimizer holding the old adapter tensors; the saved
adapter's B matrices were exactly zero under a versioned name. The best
adapter is now merged from disk onto a fresh base after training, and a
real-weights pin (tests/test_finetune_real.py, run where torch is
installed) proves the merge path trains what the plain path trains. - The recipe learned from the run. Padded batches with the padding
masked out of attention and loss, linear warm-up and cosine decay,
gradient clipping at 1.0, adapted modules chosen per architecture by
peft unless overridden (a hard-coded Llama list refused every other
family), the optimizer-step count and schedule on the record. - The comparison says what it can and cannot claim. A form score --
the share of answers that open the way the verified answers open --
beside the judge's score, in every layer and in the comparison record;
quotable: falsewith the reason when the holdout has fewer than
thirty cases or the judge is uncalibrated; a delta on too few cases is
refused unless asked for as a smoke test. The floor in force
(--min-verified) and the environment that trained (device, dtype,
library versions, base revision) are on the record. Evidence is
attached with the pipeline's own defaultk. - Boot refusals that were missing. A fine-tune build without
FINETUNED_MODELrefuses to boot instead of answering 503 to every
request; a labelled-decision build withoutevals/manifest.jsonrefuses
rather than serve an unverifiable fit; the holdout path notes a file
that is not the recorded one and an uncalibrated judge.