Skip to content

feat: add confidence-aware privacy re-answers and weighted leakage metric#88

Merged
lipikaramaswamy merged 4 commits into
mainfrom
lipikaramaswamy/feat/72-weighted-privacy-reanswers-74-metrics
Apr 2, 2026
Merged

feat: add confidence-aware privacy re-answers and weighted leakage metric#88
lipikaramaswamy merged 4 commits into
mainfrom
lipikaramaswamy/feat/72-weighted-privacy-reanswers-74-metrics

Conversation

@lipikaramaswamy
Copy link
Copy Markdown
Collaborator

Summary

  • Added strict privacy re-answer fields (confidence, reason) and updated evaluator prompt/parser handling.
  • Updated leakage computation to be confidence-aware (leakage_mass is now weighted by confidence for leaked items).
  • Updated repair context formatting to include explicit leakage confidence and reason signals.
  • Added and exposed normalized weighted_leakage_rate in rewrite outputs and preview display.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Refactoring

Testing

  • make test passes locally
  • make check passes locally (format + lint + typecheck + lock-check)
  • Added/updated tests for changes

Documentation

Related Issues

Closes #72
Closes #74

…leakage signal

Expose a normalized weighted leakage metric in rewrite outputs and preview display so users can compare leakage severity across records without changing existing leakage_mass-based repair/review thresholds.
@lipikaramaswamy lipikaramaswamy requested a review from a team as a code owner April 2, 2026 06:56
@lipikaramaswamy lipikaramaswamy changed the title Lipikaramaswamy/feat/72 weighted privacy reanswers 74 metrics feat: add confidence-aware privacy re-answers and weighted leakage metric Apr 2, 2026
@lipikaramaswamy lipikaramaswamy merged commit 414b3fc into main Apr 2, 2026
9 checks passed
@lipikaramaswamy lipikaramaswamy deleted the lipikaramaswamy/feat/72-weighted-privacy-reanswers-74-metrics branch April 2, 2026 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(evaluate): add normalized leakage_rate metric feat(evaluate): weighted privacy re-answers and reason-aware repair

2 participants