v1.0-paper — Frozen Benchmark Results (Paper Reference)

This tag freezes the exact Fair Code benchmark results cited in the research paper currently in peer review. It exists so the paper's tables and the repository never drift apart.

🔒 Paper freeze — active

The benchmark results are frozen at this point and must not change until the paper is published. Any drift would make the paper's tables disagree with the repo. Full policy: CLAUDE.md.

• Tag: v1.0-paper
• Frozen commit: bbef2ba

What's frozen

• paper/results-frozen/ — the paper's evidence (never modified, for any reason)
• results/ — results_fairness.csv, results_performance.csv, summary.csv, figures/
• The faircode/ analysis core and every audit.yaml manifest + dataset CSV
• Reproducibility parameters: random_state = 42, test_size = 0.2 (stratified), DemographicParity, the six fairness metrics, and the bootstrap/permutation counts

No new audits during the freeze

The paper covers exactly seven domains, so new audits will not be merged into main until the paper is published. Contributions are still welcome — audits are parked on a branch or held with a post-paper label and merged after publication. This is a timing hold, not a rejection. Explainers, docs, and website content are unaffected.

When the freeze lifts

On publication: the benchmark may be re-run, parked audits merge, v3.0.0 is cut, and the paper's citation + DOI are added to README.md and CITATION.cff. paper/results-frozen/ stays permanently untouched as the historical record.

v2.0.0 — Cross-Domain Fairness Benchmark Harness

Fair Code v2.0.0 — Cross-Domain Fairness Benchmark Harness

Fair Code moves from seven bespoke bias audits to a benchmark harness that runs one uniform, reproducible pipeline over all of them — the core of what a research paper built on this repo would cite. Nothing about the existing audits (unfair.py / fair.py, the website, the explainers) changed or broke; this release is additive.

🎯 Cross-Domain Benchmark Harness

• audit.yaml manifests — every audit folder now carries a declarative manifest naming its label column, protected attributes, proxy features, and core (fair) feature set. Schema: faircode/MANIFEST_SPEC.md.
• Five mitigation strategies (S0–S4), run uniformly across all seven domains:
  • baseline → unawareness → unawareness_proxy_removal (the existing fair.py method) → in_processing (Fairlearn ExponentiatedGradient) → post_processing (Fairlearn ThresholdOptimizer)
• Three model families (logistic regression, random forest, gradient boosting) × six fairness metrics (demographic parity, disparate impact ratio, equal opportunity, equalized odds, predictive parity, accuracy equality) — each with a bootstrap CI and permutation-test p-value — plus accuracy/AUC/F1 and intersectional gaps.
• New CLI: faircode benchmark (optional faircode[benchmark] extra). Writes results_fairness.csv, results_performance.csv, summary.csv, and 300-dpi figures to results/.

🔁 Reproducibility & Paper Freeze

• All randomness pinned to random_state: 42 across every model, split, bootstrap, and permutation — verified, not assumed.
• requirements-lock.txt — exact environment (Python 3.13.2, scikit-learn 1.8.0, fairlearn 0.14.0, pandas 3.0.2).
• scripts/freeze_paper_results.py — snapshots results/ into paper/results-frozen/ with a MANIFEST.md recording the git commit, package versions, and the exact set of audits included, so a citation points at a defined, reproducible snapshot instead of "whatever was in the repo that week."

✅ Tests & CI

• 50 new tests (108 total, ~10s) covering all six fairness metrics against hand-computed examples, manifest validation, S0–S4 strategy correctness, and a genuine end-to-end harness run.
• New benchmark-harness CI job runs the suite plus a real CLI smoke test on one small audit. The full seven-domain sweep stays out of CI by design — it's run locally and the output committed.

🐛 Fixed

• ThresholdOptimizer (S4) was calibrating decision thresholds on the same rows its base model was trained on, producing a near-zero gap in-sample but +0.22 out-of-sample on German Credit Lending. Fixed with a proper fit/calibration split — residual gap now a defensible +0.07.
• All 14 fair.png / unfair.png screenshots regenerated from fresh runs — the old images predated the bootstrap CI / permutation-test output and only showed the bare headline gap. Every underlying number was independently re-verified against the live site (all matched exactly).

📈 The headline result (COMPAS)

Demographic parity gap across the five strategies: 86% → 18% → 12% → 1.4% → 2.4%. The two Fairlearn-based strategies land within a couple points of simple proxy removal — evidence for a residual fairness floor, not a limitation of any one method.

Full changelog: CHANGELOG.md § 2.0.0

Fair Code v1.3.3 — Explainers, Intersectional Foundations & AI Discoverability

v1.3.3 - Explainers, Intersectional Foundations & AI Discoverability

Covers everything since v1.3.0: three new explainers, and a pass making the site properly indexable and correctly attributed by search engines and AI assistants.

Added

• Explainer: What Is Unsupervised Learning? (contributed by @AnayDhawan) - k-means on the Benefits Denial dataset with sex, race, and native.country excluded still recovers a strong sex split (one cluster 89.3% male) and a real race split, showing clustering can sort people along demographic lines with no labels and no protected attribute in the feature set
• Explainer: What Is Model Drift? (contributed by @AnayDhawan) - re-measuring the German Credit Lending age fairness gap across five rolling windows shows it swinging from 4.3% to 15.1%, versus a single 6.39% audit-time snapshot; ships population_stability_index() and page_hinkley_test() for rolling-window monitoring
• Explainer: What Is Selection Bias? - the German Credit Lending dataset's 700/300 good/bad split contains zero rejected applicants, because a turned-down applicant never generates a repayment outcome to record; includes a from-scratch Berkson's-paradox simulation as detection code
• llms.txt at the site root - a plain-text index of audits, tools, and explainers for AI assistants and crawlers to read directly, following the llmstxt.org convention
• Person schema and author attribution (name "Yash Kewlani") added across the homepage and all 29 explainer pages via JSON-LD and <meta name="author">, giving AI grounding an unambiguous, machine-readable source instead of guessing from the GitHub handle

Fixed

• Sitemap/canonical host mismatch. Every canonical tag, og:url, and JSON-LD url field, plus sitemap.xml and robots.txt's Sitemap: line, referenced the bare apex domain (thefaircode.xyz), which 308-redirects to www.thefaircode.xyz - the host that actually serves the site. Google Search Console rejected the sitemap ("Sitemap could not be read") because none of its listed URLs matched the host it was fetched from. All 33 affected files now consistently use www.thefaircode.xyz

Changed

• robots.txt, canonical URLs, and homepage JSON-LD structured data (WebSite/Organization schema) added for search and AI crawlability; all 29 explainer pages made crawlable (canonical tags, og:type=article, per-page DefinedTerm JSON-LD)
• METRICS.md: new 2026-W30 snapshot - stars 27 → 38, forks 8 → 14, contributors 7 → 9, code audits 6 → 7, social reach ~10K → ~16K; Watching (7) tracked for the first time
• ROADMAP.md: traction table refreshed, Phase 4 status updated to 9 external contributors
• README.md: Traction table refreshed to match; Star History section removed
• CITATION.cff: version 1.3.0 → 1.3.3

Full Changelog: v1.3.0...v1.3.3

v1.3.0 — Intersectional Bias Analysis: auditing two protected attributes at once

Intersectional bias — seeing the harm a single-axis audit can't

Every Fair Code audit so far measured fairness one protected attribute at a
time: sex, then race, then age, each against everyone else. That averaging is
exactly what hides the group at the intersection of two attributes. This release
adds intersectional auditing — crossing two protected attributes to check the
doubly-disadvantaged group directly — grounded in Crenshaw's (1989) original
argument that a harm landing on, say, minority women is invisible to a race-only
or sex-only analysis.

🔀 New: intersectional_report (faircode.significance)

Give it an outcome and two boolean masks (each marking the disadvantaged side of
one attribute) and it splits the population into the four quadrants of the two
attributes, then compares the doubly-disadvantaged cell against the
baseline cell with the same bootstrap CI + permutation p-value the
single-axis audits already use. It also reports:

• each attribute's marginal gap (what it would report on its own), so the
  single-axis and intersectional views sit side by side
• a superadditive flag — true when the compounded gap exceeds the sum of
  the two marginals (the intersection is worse than the parts)
• per-quadrant rates and sizes, so a thin doubly-disadvantaged cell is visible
  before the small-sample warning even fires

Pure numpy/pandas — no new dependencies.

📉 Wired into the three multi-attribute audits

Each now prints an intersectional block after its existing single-attribute
output, in both unfair.py and fair.py:

• Insurance Denial — Age × Sex (Young × Female)
• Benefits Denial — Sex × Race (Female × Minority)
• Healthcare Readmission — Sex × Race (Female × Minority)

COMPAS, AI Fair Recruitment, and German Credit Lending track a single protected
attribute each and are unchanged — there is nothing to cross.

📓 New notebook: 07_intersectional_bias_audit.ipynb

Walks through what intersectionality means and why marginal fairness gaps can
hide a worse compounded one, then runs intersectional_report on the biased vs.
mitigated models for all three pairs. The finding worth reading: proxy removal
closes the intersectional gap roughly in step with the marginals for Benefits
and Healthcare — but for Insurance the marginal age and sex gaps shrink while the
compounded gap for young women does not, tipping the mitigated model into
superadditive territory. Removing proxies fixed the single-axis numbers without
fixing the intersection.

✅ Tests & docs

• Three new significance tests (additive, superadditive, small-cell warning) —
  50 passing
• Methodology note in README.md; Phase 5 roadmap item checked off;
  CONTRIBUTING.md audit template now asks 2+-attribute audits to report at
  least one intersectional pair

Note: intersectional auditing does not replace the marginal audit — the
doubly-disadvantaged cell is the smallest quadrant, so its gap carries the widest
CI and a non-significant result means "not enough evidence," not "no bias." It
catches the harm the marginal audit is structurally unable to see.

Full changelog: v1.2.5...v1.3.0

v1.2.5 — Open Dataset Profiler: drift comparison, column mapping & reference baselines

Open Dataset Profiler — a major capability wave

This release rounds out the Open Dataset Profiler with six enhancements. All of
them run through one shared spec (faircode/SPEC.md), so the faircode CLI and
the in-browser profiler produce identical numbers.

⚖ Compare two datasets for representation drift (#60)

faircode compare train.csv prod.csv — or drop an A/B pair in the web profiler —
to see how each demographic dimension shifted. Reports the Population Stability
Index (PSI), Total Variation Distance, per-group share shifts, and
appeared/disappeared groups, flagging significant drift and score drops.

🗺 Manual column mapping (#62)

When auto-detection misses an oddly-named column (gndr, patient_region_code),
override it: faircode profile data.csv --map gndr=sex, or editable per-column
dropdowns on the web that re-run the audit instantly.

📊 Reference-population baseline (#56)

Score a dataset against an external population (e.g. Census age×sex), not just
internal balance: faircode profile data.csv --reference census.csv or a web
upload. Surfaces per-group expected-vs-actual deltas and flags
under-representation relative to who the model will actually serve.

✕ Choosable intersection pair (#58)

Cross any two demographic columns, not just the first two detected:
faircode profile data.csv --cross race,age or two dropdowns in the web UI.

🔗 Chi-squared proxy hints (#61)

faircode profile data.csv --proxy-hints flags strongly-associated column pairs
— a "this may be a proxy for a protected attribute" signal — with p-values and
Cramér's V. Opt-in via the optional scipy extra; informational, never scored.

🎚 Tunable thresholds (#63)

--min-share, --intersection-floor, --imbalance-flag, and --missing-flag
let auditors set their own bars without editing source.

Install: pip install -e ".[proxy]" for the chi-squared extra.
Full changelog: see CHANGELOG.md → [1.2.5].

v1.1.0 — Healthcare Readmission Audit + 4 Explainers

Fair Code's second tagged release. Adds the sixth bias audit — clinical AI — plus four new explainers covering foundational AI concepts.
What's included
1 bias audit

Healthcare Readmission (Diabetes 130-US Hospitals, 101,766 records) — Race: 25% reduction, Age: 68% reduction, Gender: +100% ↑ (gap worsened — proxy removal backfired. Payer code was suppressing the gender signal while also encoding it. Remove it, and the suppression disappears. This is proxy entanglement — and pre-processing mitigation can't fix it. That's a future audit.)

4 explainers

What Happens Inside a Neural Network
Why AI Hallucinates
What Is Reinforcement Learning
What Is Proxy Entanglement

6 Jupyter notebooks — full walkthrough of every audit with visualisations and proxy detection
Live website — fair-code-five.vercel.app
CI pipeline — all audit scripts run automatically on every push and PR

What's Changed

• Update matplotlib requirement from >=3.10.8 to >=3.10.9 by @dependabot[bot] in #37
• Update pandas requirement from >=3.0.2 to >=3.0.3 by @dependabot[bot] in #38
• Update numpy requirement from >=2.4.4 to >=2.4.6 by @dependabot[bot] in #39
• Update scikit-learn requirement from >=1.8.0 to >=1.9.0 by @dependabot[bot] in #41
• Refactor dataset structure in README.md by @yakew7 in #42
• Explainer: Why AI Hallucinates Added by @Shreyash0712 in #43
• Update README with new document and formatting changes by @yakew7 in #44
• Update CHANGELOG.md for version 1.1.0 by @yakew7 in #45
• Fix: First Interaction Trigger Change, Audits Workflow Enhancement and Version Upgrade by @Shreyash0712 in #46
• Delete .github/workflows/update-changelog.yml by @yakew7 in #47
• Create reinforcement-learning.md by @evanjain-dot in #48
• Create explainer for Proxy Entanglement by @yakew7 in #49
• Add proxy-entanglement.md to contributing guidelines by @yakew7 in #50

New Contributors

• @dependabot[bot] made their first contribution in #37
• @yakew7 made their first contribution in #42
• @Shreyash0712 made their first contribution in #43

Full Changelog: v1.0.0...v1.1.0

v1.0.0 — First Stable Release

Fair Code's first tagged release. The project has been live and open to contributors since May 2026. This tag marks the point where the core scope is complete: 5 audits, 13 explainers, full notebook walkthroughs, a live website, and a CI pipeline.

What's included

5 bias audits

• Criminal Justice (COMPAS, 70,000+ records) -71% gap reduction
• Hiring (AI Fair Recruitment, ~6,500 records) - 97.3% gap reduction
• Lending (German Credit, 1,000 records) - 73.6% gap reduction
• Healthcare (Insurance Denial, 1,340 records) - 60–72% gap reduction
• Welfare Eligibility (UCI Adult Census, 48,842 records) - 46–88% gap reduction

13 explainers
Proxy Variables, Equalized Odds, Sampling Bias, SHAP Values, Disparate Impact, Disparate Treatment, Fairness Metric Conflicts, Calibration, Demographic Parity, Feedback Loop Bias, Label Bias, Individual Fairness, Counterfactual Fairness

5 Jupyter notebooks - full walkthrough of every audit with visualisations and proxy detection

Live website - fair-code-five.vercel.app

CI pipeline - all audit scripts run automatically on every push and PR