Skip to content

v1.0.0

Latest
Latest

Choose a tag to compare

View all tags
@rvong65 rvong65 released this 21 Jul 17:07
v1.0.0
facb5bf

Summary

Initial public MVP: ethical agent auditor for reviewing AI agent decisions in real time — hybrid rule pre-checks, Chroma RAG, five-principle LLM scoring, Examples + EDS dilemma pairing, labeled Simulated mode, Streamlit UI, Docker, and GitHub Actions CI.

Live demo: https://ethical-agent-auditor.streamlit.app/

Features

Audit pipeline

  • Hybrid evaluation — deterministic rule flags (PII, red flags, deceptive language, high-stakes without transparency) injected into Groq LLM scoring
  • Five principles — harm minimization, fairness, transparency, privacy, safety with per-principle evidence and suggestions
  • Structured AuditReport (Pydantic) — overall score, risk band, rule flags, recommendations, live vs mock source
  • Chroma RAG — pre-built index over principles, agent examples, and EDS dilemmas (data/chroma/ committed for Cloud/Docker)
  • Shared execute_audit path — UI and pytest exercise the same Run Audit pipeline
  • Audit JSON export — user-initiated download after a run

Safety & responsible AI

  • Simulated mode — labeled mock fallback on rate limits, auth failure, or missing API key (never silent)
  • Educational framing — not legal advice, compliance certification, therapy, or crisis support
  • Session-only state — no project database; exports are user-initiated only
  • Privacy & data expander on Home + README data-flow table
  • Optional LangSmith — @traceable on audit_agent_action for operator observability (not required for the public demo)

Streamlit app

  • Custom amber + teal theme (complements Ethical Decision Simulator indigo); Home / Audit / Examples navigation
  • Curated agent-action examples with one-click load into Audit
  • EDS dilemma pairing — bundle dilemmas from Ethical Decision Simulator for complementary audits
  • Provider status badge (Groq configured); friendly LLM errors
  • Hybrid LLM — Groq GPT-OSS 20B (default) / GPT-OSS 120B; optional Ollama (gemma3:4b) for local inference

Quality & docs

  • 36 offline pytest tests + live Groq API integration + LangSmith smoke
  • GitHub Actions CI — offline tests + Docker build/health on every push/PR (no API keys required); optional API / LangSmith smoke on main
  • README with architecture overview, quick start, privacy, screenshots (Audit, Examples, LangSmith), and safety considerations
  • docs/architecture.md, CHANGELOG, Dockerfile / docker-compose, brand assets
  • Streamlit Cloud deployment

Quick start

git clone https://github.com/rvong65/ethical-agent-auditor.git
cd ethical-agent-auditor
pip install -r requirements.txt
cp .streamlit/secrets.toml.example .streamlit/secrets.toml   # add GROQ_API_KEY
streamlit run app.py

Try it: Home → Examples → load a scenario → Audit → Run Audit.

Docker:

docker compose up --build
# open http://localhost:8501

Note

This release marks the MVP baseline.

Assets 2
Loading