Skip to content

v1.0.0

Choose a tag to compare

View all tags
@rvong65 rvong65 released this 28 Jun 22:33
v1.0.0
6878a5b

Summary

Initial public MVP: ethical decision simulator for auditing how LLMs reason through moral dilemmas — Explore single-model review, Compare multi-model audit with semantic agreement metrics, ethical lens stress tests, Compare-only JSON export, crisis guardrails, Streamlit UI, and GitHub Actions CI.

Live demo: https://ethical-decision-simulator.streamlit.app/

Features

Ethics pipeline

  • Structured JSON verdicts (Pydantic) — decision, reasoning steps, creative alternatives, detected frameworks, confidence
  • Explore — single-model walkthrough with ethical lens selection (neutral, utilitarian, deontological, care ethics); on-screen results only
  • Compare — up to 3 models side-by-side with semantic decision agreement (primary) and lexical agreement (secondary)
  • Framework stress test — batch run all four lenses per model to probe framework adherence
  • Consistency test — rephrase variants on curated scenarios to measure framing sensitivity
  • Creative alternatives first — models list escape hatches before forced tradeoff analysis
  • Lens-adherence mismatch warnings when forced lens ≠ detected framework
  • Compare JSON export (ethical_comparison_*.json) — full metrics, verdicts, consistency data, and run metadata

Safety & responsible AI

  • Crisis keyword detector with 988 / Crisis Text Line disclaimer banner
  • Simulated mode — labeled mock fallback on rate limits, auth failure, or missing API key (never silent)
  • Educational framing — not moral guidance, legal advice, therapy, or crisis service
  • Session-only state — no project database; Compare export is user-initiated download only
  • Privacy & data expander on Home + README data-flow table

Streamlit app

  • Custom indigo sidebar theme; Home / Explore / Compare navigation
  • 12 curated moral dilemmas plus custom dilemma input
  • In-app glossary — ethical lenses, Compare modes, metric definitions
  • Provider status badge (live vs simulated); friendly LLM errors (429, auth, timeout)
  • Hybrid LLM — Groq (Llama 3.3 70B, GPT-OSS 20B, Llama 3.1 8B) default; optional Ollama (gemma3:4b) for local inference

Quality & docs

  • 46 offline pytest tests + 3 optional Groq API smoke tests
  • GitHub Actions CI — offline tests + Docker build/health on every push/PR (no API keys required); optional API smoke on main
  • README with architecture overview, quick start, privacy, and safety considerations
  • docs/architecture.md, CHANGELOG, Dockerfile / docker-compose, brand assets
  • Streamlit Cloud deployment

Quick start

git clone https://github.com/rvong65/ethical-decision-simulator.git
cd ethical-decision-simulator
pip install -r requirements.txt
cp .streamlit/secrets.toml.example .streamlit/secrets.toml   # add GROQ_API_KEY
streamlit run app.py

Try it: Home → Explore or Compare → pick a curated scenario in the sidebar → Run analysis or Run comparison.

Docker:

docker compose up --build 
# open http://localhost:8501

Note

This release marks the MVP baseline.

Assets 2
Loading