chatstream-moderate

Real-time chat moderation for Eventyay conference streams, hosted on Toolforge.

Incoming chat messages are held in a moderation queue. Moderators approve, reject, or highlight messages before they appear in the output feeds. Multiple moderators can work the same queue simultaneously — competing decisions are resolved by most-restrictive-wins.

---

How it works

1. Eventyay posts channel.message and event.reaction events to /webhook/channel/<id> via HMAC-signed HTTP
2. Messages enter the moderation queue (unless the sender is blacklisted, matches a blocked pattern, or is whitelisted)
3. Moderators log in with their Wikimedia account and work the queue: approve / reject / highlight / ban / allow
4. Approved and highlighted messages are published via an RSS feed consumed by display tools

---

Local development

Prerequisites

• UV for Python dependency management
• Python 3.11+ (3.13 recommended; matches Toolforge webservice)

Setup

git clone https://github.com/lgelauff/chatstream-moderate
cd chatstream-moderate
uv sync

Create a .env file (never commit this):

OAUTH_CLIENT_ID=your_client_id
OAUTH_CLIENT_SECRET=your_client_secret
OAUTH_REDIRECT_URI=http://127.0.0.1:5000/oauth-callback
SECRET_KEY=any-random-string-for-local-dev

Running

FLASK_DEBUG=1 uv run python app.py

Visit http://127.0.0.1:5000 (not localhost — AirPlay Receiver can intercept port 5000 on macOS).

Dev login (bypass OAuth)

With FLASK_DEBUG=1, skip OAuth entirely:

http://127.0.0.1:5000/dev-login?username=YourWikimediaName

Superadmin access

Edit the SUPERADMIN_USERS list in app.py:

SUPERADMIN_USERS: list[str] = ["YourWikimediaName"]

Simulating a message stream

1. Log in as superadmin
2. Visit /admin/ → activate the simulation channel
3. Open the simulation channel's queue — a floating panel lets you start/stop a message stream at up to 240 msg/min

Running the tests

Integration tests use Flask's test client and an in-memory SQLite database — no running server needed:

.venv/bin/python tests/test_webhook.py
.venv/bin/python tests/test_multiuser.py

See tests/README.md for what each suite covers.

Database

SQLite is used automatically for local dev (instance/dev.db). No setup needed.

---

Configuration

All secrets are read from /etc/passwords/<name> (Toolforge Kubernetes secrets) with environment variable fallback. For local dev, set them via .env.

Secret name	Env var	Description
oauth-client-id	OAUTH_CLIENT_ID	Wikimedia OAuth consumer key
oauth-client-secret	OAUTH_CLIENT_SECRET	Wikimedia OAuth consumer secret
oauth-redirect-uri	OAUTH_REDIRECT_URI	OAuth callback URL
secret-key	SECRET_KEY	Flask session secret
db-host	DB_HOST	MariaDB host (default: tools.db.svc.wikimedia.cloud)
db-user	DB_USER	MariaDB user
db-password	DB_PASSWORD	MariaDB password
db-name	DB_NAME	MariaDB database name

---

Deployment (Toolforge)

This is a one-time setup. For routine updates see Updating.

1. Register the tool

Register at https://toolsadmin.wikimedia.org with tool name chatstream-moderate.

2. Register OAuth consumer

Register at https://meta.wikimedia.org/wiki/Special:OAuthConsumerRegistration with:

• Callback URL: https://chatstream-moderate.toolforge.org/oauth-callback
• Grant: User identity verification only (confidential client, authorization code only)

Public consumers require admin approval — plan for several days wait. Owner-only consumers are active immediately.

3. SSH into Toolforge

ssh USERID@login.toolforge.org
become chatstream-moderate

Reference: ../wikimedia-coding-agent-lessons/toolforge/lessons.md has ground-truth Toolforge deployment notes.

4. Clone the repository

git clone https://github.com/lgelauff/chatstream-moderate ~/chatstream-moderate

5. Create the database

Read your credentials:

cat ~/replica.my.cnf

Connect and create the database — the name must start with your tools prefix:

mariadb --defaults-file=$HOME/replica.my.cnf -h tools.db.svc.wikimedia.cloud

CREATE DATABASE `s12345__chatstream`
  CHARACTER SET utf8mb4
  COLLATE utf8mb4_unicode_ci;
EXIT;

6. Set secrets

Use single quotes to avoid shell interpretation:

toolforge envvars create OAUTH_CLIENT_ID     'YOUR_CLIENT_ID'
toolforge envvars create OAUTH_CLIENT_SECRET 'YOUR_CLIENT_SECRET'
toolforge envvars create OAUTH_REDIRECT_URI  'https://chatstream-moderate.toolforge.org/oauth-callback'
toolforge envvars create SECRET_KEY          "$(python3 -c 'import secrets; print(secrets.token_hex(32))')"
toolforge envvars create DB_USER             's12345'
toolforge envvars create DB_PASSWORD         'YOUR_DB_PASSWORD'
toolforge envvars create DB_NAME             's12345__chatstream'

toolforge envvars list masks values after creation — keep a local record.

7. Set up the web service directory

~/www/python must be a real directory, not a symlink:

mkdir -p ~/www/python
ln -s ~/chatstream-moderate ~/www/python/src

Open a webservice shell to create the venv — must be done inside the container, not on the bastion. Bastion and webservice both run Python 3.13. Never run pip from the bastion against the webservice venv.

toolforge webservice --backend=kubernetes python3.13 shell

Inside the shell, create the venv and install packages. Use get-pip.py piped directly — ensurepip and python3 -m venv (without --without-pip) hang due to subprocess restrictions in the shell pod:

python3 -m venv ~/www/python/venv --without-pip
curl -sS https://bootstrap.pypa.io/get-pip.py | ~/www/python/venv/bin/python3
~/www/python/venv/bin/python3 -m pip install -e ~/chatstream-moderate
exit

8. Start the web service

Run from your home directory:

cd ~
toolforge webservice --backend=kubernetes python3.13 start

Check logs:

toolforge webservice logs

lseek: Illegal seek lines in the logs are harmless uWSGI noise — filter with grep -v lseek.

The database schema is created automatically on first startup.

Updating

For code changes (no new dependencies):

bash ~/chatstream-moderate/deploy.sh

For dependency changes (new packages added to pyproject.toml), you must reinstall inside the webservice shell:

toolforge webservice --backend=kubernetes python3.13 shell
~/www/python/venv/bin/python3 -m pip install -e ~/chatstream-moderate
exit
cd ~
toolforge webservice --backend=kubernetes python3.13 restart

Troubleshooting

WARNING: Ignoring invalid distribution ~ip — corrupted pip leftover from a failed install. Fix from the bastion:

rm -rf ~/www/python/venv/lib/python3.13/site-packages/~ip*

OAuth invalid_scope error — the Wikimedia OAuth 2.0 scope must be basic, not openid. Check app.py.

---

Project structure

chatstream-moderate/
  app.py              — Flask app factory, OAuth flow, SUPERADMIN_USERS config
  wsgi.py             — WSGI entry point
  uwsgi.ini           — uWSGI config (buffer-size for long OAuth codes)
  deploy.sh           — Toolforge deploy script
  pyproject.toml      — Dependencies (managed with UV)
  src/
    models.py         — SQLAlchemy models
    webhook.py        — Webhook receiver, message intake, blacklist/whitelist checks
    queue_bp.py       — Moderation queue UI and API actions
    admin_bp.py       — Channel admin and superadmin management
    display_bp.py     — RSS feed output
    auth.py           — Auth helpers, role checks
    utils.py          — Levenshtein, token generation
  templates/
    base.html         — Header, flash messages
    queue.html        — Moderation queue (JSON polling, keyboard shortcuts)
    queue/log.html    — Moderation decision log
    admin/            — Channel settings, blacklist, whitelist, simulation
  static/
    css/app.css       — Light wiki-polis theme
    js/user-picker.js — Wikimedia username autocomplete (meta.wikimedia.org API)
  tests/
    test_webhook.py   — 20 webhook intake integration tests
    test_multiuser.py — 16 multi-moderator decision tests
    README.md         — Test suite documentation

---

Webhook payload format

{
  "message_id":     "uuid",
  "channel":        "eventyay-channel-id",
  "timestamp":      "ISO8601",
  "screen_name":    "display name",
  "message":        "text content",
  "message_type":   "text | emoji | qa",
  "sender_id":      "eventyay-user-id | null",
  "centralauth_id": "wikimedia-centralauth-id | null",
  "profile_img":    "url | null",
  "user_language":  "BCP47 | null",
  "meta":           {}
}

Authentication: X-Eventyay-Signature: sha256=<hmac-hex> over the raw request body. Shared secret is per-channel.