Skip to content

Content Moderation

Jump to bottom
mdeguzis edited this page Jun 6, 2026 · 2 revisions

Content Moderation

User-submitted report text is scanned automatically on two layers:

  1. Wordlist (naughty-words) - offline multilingual filter, runs first on every row
  2. OpenAI Moderation API - semantic fallback for anything the wordlist misses (requires OPENAI_API_KEY secret in the proton-pulse-data repo; falls back to wordlist-only if absent)

The scan runs every 4 hours via GitHub Actions (content-moderation.yml) with a 5-hour lookback window, and does a full 25-hour sweep daily at 02:00 UTC. Flagged reports are hidden from public views automatically. Report authors see a "Flagged" badge on their profile page with a plain-language explanation and a link to the Discord server for disputes.

To trigger a manual scan:

gh workflow run content-moderation.yml --repo mdeguzis/proton-pulse-data \
  -f dry_run=true -f lookback_hours=720

Admin Roles

The public.admins table controls who can access the admin panel at /admin.html. There are two roles.

Moderator (default)

This is the default role when a user is added to the admins table. Moderators can:

  • View and manage flagged reports (hide, unhide, unflag)
  • View and manage the banned users list (add, remove bans)
  • View and search banned phrases

Moderators cannot add or remove other admins, and cannot change anyone's role.

Super Admin

Super admins have all moderator permissions plus:

  • Add new admins (assign moderator or super_admin role)
  • Change an existing admin's role
  • Remove admins from the table
  • Add and remove banned phrases

Only super admins can modify the admins table. The RLS policies on public.admins enforce this at the database level via the is_current_user_super_admin() SECURITY DEFINER function.

How Roles Are Enforced

Role checks happen in two places:

Database (RLS policies)

  • is_current_user_admin() - SECURITY DEFINER function, returns true if the calling user's auth.uid() is present in public.admins regardless of role. Used to gate read access to the admins, banned_users, and banned_phrases tables.
  • is_current_user_super_admin() - same pattern but also checks role = 'super_admin'. Used to gate INSERT/UPDATE/DELETE on the admins table.

These functions bypass RLS internally to avoid infinite recursion (PostgreSQL 42P17).

Frontend (admin panel)

admin.html checks whether the signed-in user has a row in public.admins before revealing the panel. If the check fails (not in the table), a "not authorized" message is shown instead.

The admin nav link in the top bar (topbar.js) does the same check and stays hidden for non-admins.

Adding an Admin

Only a super admin can do this through the admin panel. The panel's "Admins" tab has a form with fields for the user's UUID (from auth.users), their Steam username, and the role to assign.

To find a user's UUID: look it up in the Supabase dashboard under Authentication > Users, or have the user check their profile page on proton-pulse.com.

Banning a User

When a user is added to public.banned_users, a trigger (trg_hide_configs_on_ban) automatically sets is_hidden = true on all their user_configs rows. This removes their reports from public views immediately without needing a separate moderation step.

Banned users are matched by proton_pulse_user_id (for signed-in users) or client_id (for anonymous submissions).

Clone this wiki locally