Skip to content

[codex] add semantic similarity rating#1

Draft
lfiaschi wants to merge 2 commits into
mainfrom
codex/add-ssr
Draft

[codex] add semantic similarity rating#1
lfiaschi wants to merge 2 commits into
mainfrom
codex/add-ssr

Conversation

@lfiaschi

Copy link
Copy Markdown
Owner

Summary

Adds semantic-similarity rating (SSR) as an opt-in rating layer for AudienceKit rather than as a dataset adapter or model backend.

  • Adds SSRAnchorSet, SSRResult, SemanticSimilarityRater, and SentenceTransformerEmbeddings.
  • Adds SyntheticPanel.run_ssr_survey(...), which elicits free-text Likert answers, scores them into PMFs, and preserves both text and structured columns.
  • Keeps embeddings injectable so hosted embeddings, local models, and tests can use the same scoring abstraction.
  • Adds id_column, weight_column, and respondent metadata handling to SyntheticPanel for non-GSS panels.
  • Documents SSR usage, custom anchors, optional ssr extra, and the standalone PyMC Labs SSR reference implementation.

Validation

  • git diff origin/main..HEAD --check
  • uv run --extra dev python -m pytest tests -q (28 passed)
  • uv run python -m compileall audiencekit scripts
  • uv build
  • Read-only subagent review was run before PR creation; validation and custom-dataset metadata findings were fixed in 9948a61.

Notes

Direct Likert prompting remains available through run_survey(...). SSR is opt-in through run_ssr_survey(...) and should use anchors that match the Likert question being scored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant