From 1f932bba4da05013870ff8f50b8f23b0a0955851 Mon Sep 17 00:00:00 2001 From: Lodewijk <32471480+lgelauff@users.noreply.github.com> Date: Sat, 18 Apr 2026 16:57:52 +0200 Subject: [PATCH 01/10] docs: add debugging section to deployment.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Documents common Toolforge debugging steps discovered during beta deployment of the image→filerevision migration: - uwsgi log location (/data/project/montage-beta/uwsgi.log) - pip/Python must run inside toolforge webservice shell, not bastion - restart command syntax - how to inspect the SQLite DB without the sqlite3 CLI Co-Authored-By: Claude Sonnet 4.6 --- deployment.md | 45 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/deployment.md b/deployment.md index d83c4713..171b932e 100644 --- a/deployment.md +++ b/deployment.md @@ -123,4 +123,47 @@ toolforge webservice python3.9 restart ##### 9. Testing of deployment * Visit /meta to see the API. Example: https://montage-beta.toolforge.org/meta/ -* In the top section, you should see that the service was restarted in the last few seconds/minutes. \ No newline at end of file +* In the top section, you should see that the service was restarted in the last few seconds/minutes. + + +--- + + +## Debugging + +##### Viewing logs + +The uwsgi log is at: +```bash +tail -50 /data/project/montage-beta/uwsgi.log +``` + +Note: there is no `logs/` directory inside `src/` — the log is one level up. + +##### Running Python / pip commands + +Always run `pip install` and Python diagnostics inside the webservice shell, not the bastion shell. The two environments use different venvs: + +```bash +toolforge webservice python3.9 shell +# venv is activated automatically +pip install -r ~/www/python/src/requirements.txt +python3 -c "import montage.app" +exit +``` + +Running `pip` on the bastion shell installs to a different venv and will not affect the running service. + +##### Restarting the service + +```bash +toolforge webservice python3.9 restart +``` + +##### Inspecting the SQLite database (beta only) + +There is no `sqlite3` CLI on Toolforge. Use Python instead: + +```bash +python3 -c 'import sqlite3; c=sqlite3.connect("/data/project/montage-beta/www/python/src/tmp_montage.db"); print(c.execute("SELECT COUNT(*) FROM entries").fetchone())' +``` \ No newline at end of file From 85d0be4942544db6b8d4ad20312ab8e61c9873e6 Mon Sep 17 00:00:00 2001 From: Lodewijk <32471480+lgelauff@users.noreply.github.com> Date: Sun, 19 Apr 2026 15:56:09 +0200 Subject: [PATCH 02/10] docs: fix MariaDB setup steps in deployment.md - Step 5: use correct `mariadb` command with explicit -h flag and utf8mb4 charset - Step 6: fix db_url format (mysql+pymysql driver, tools.db.svc.wikimedia.cloud host, charset=utf8mb4) - Step 8 (new): add schema initialisation step (create_schema.py for fresh installs, migrate_prod_db.sql for upgrades) - Debugging: add MariaDB connection section, note that -h flag is required (no local socket on bastion) - Debugging: rename SQLite section to clarify it is legacy/dev only Observed during #514 (beta MariaDB setup). Co-Authored-By: Claude Sonnet 4.6 --- deployment.md | 47 ++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 40 insertions(+), 7 deletions(-) diff --git a/deployment.md b/deployment.md index 171b932e..5f318c37 100644 --- a/deployment.md +++ b/deployment.md @@ -33,9 +33,16 @@ exit This will build the vue prod bundle and put in backend's `template` and `static` directory. ##### 5. Create your database -* Get the user name of database (`cat ~/replica.my.cnf`) -* Open up MariaDB with `sql local` -* Create a [Toolforge user database](https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#User_databases) (`create database __;`), and remember the name for the config +* Get the username and password from `cat ~/replica.my.cnf` +* Connect to MariaDB: + ```bash + mariadb --defaults-file=~/replica.my.cnf -h tools.db.svc.wikimedia.cloud + ``` +* Create a [Toolforge user database](https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#User_databases) with `utf8mb4` charset, and remember the name for the config: + ```sql + CREATE DATABASE `__` DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci; + EXIT; + ``` ##### 6. Set up the montage config * Make a copy of `config.default.yaml` for your environment @@ -43,7 +50,7 @@ This will build the vue prod bundle and put in backend's `template` and `static` * Add the `oauth_consumer_token` and `oauth_secret_token` * Add a `cookie_secret: ` * Add the `db_url` with your user database name, and the password from `~/.replica.my.cnf` - * The format is: `mysql://:@tools.labsdb/?charset=utf8` + * The format is: `mysql+pymysql://:@tools.db.svc.wikimedia.cloud/?charset=utf8mb4` * Add `api_log_path: /data/project//logs/montage_api.log` * Add `replay_log_path: /data/project//logs/montage_replay.log` * Add `labs_db: True` @@ -61,12 +68,24 @@ pip install -r $HOME/www/python/src/requirements.txt exit ``` -##### 8. Start the backend service +##### 8. Initialise the database schema +```bash +cd $HOME/www/python/src +source $HOME/www/python/venv/bin/activate +python3 montage/create_schema.py +``` + +If this is an upgrade of an existing deployment (not a fresh install), run the migration SQL instead: +```bash +mariadb --defaults-file=~/replica.my.cnf -h tools.db.svc.wikimedia.cloud < tools/migrate_prod_db.sql +``` + +##### 9. Start the backend service ```bash toolforge webservice python3.9 start ``` -##### 9. Testing of deployment +##### 10. Testing of deployment * Visit /meta to see the API. Example: https://montage-beta.toolforge.org/meta/ * In the top section, you should see that the service was restarted in the last few seconds/minutes. @@ -160,7 +179,21 @@ Running `pip` on the bastion shell installs to a different venv and will not aff toolforge webservice python3.9 restart ``` -##### Inspecting the SQLite database (beta only) +##### Inspecting the MariaDB database + +Always pass `-h tools.db.svc.wikimedia.cloud` explicitly — there is no local socket on the Toolforge bastion: + +```bash +mariadb --defaults-file=~/replica.my.cnf -h tools.db.svc.wikimedia.cloud +``` + +Example queries: +```sql +SELECT COUNT(*) FROM entries; +DESCRIBE entries; +``` + +##### Inspecting the SQLite database (legacy / dev only) There is no `sqlite3` CLI on Toolforge. Use Python instead: From 40b157dc4dbe45bad0078f0716114ac5a31cf69b Mon Sep 17 00:00:00 2001 From: Lodewijk <32471480+lgelauff@users.noreply.github.com> Date: Sun, 19 Apr 2026 15:56:38 +0200 Subject: [PATCH 03/10] docs: note that beta SQLite file may coexist with MariaDB setup Co-Authored-By: Claude Sonnet 4.6 --- deployment.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/deployment.md b/deployment.md index 5f318c37..9d670edb 100644 --- a/deployment.md +++ b/deployment.md @@ -195,6 +195,8 @@ DESCRIBE entries; ##### Inspecting the SQLite database (legacy / dev only) +Note: montage-beta originally used SQLite and the file (`tmp_montage.db`) may still exist alongside the MariaDB setup. It is no longer used by the running service once the config switches to `mysql+pymysql://`. + There is no `sqlite3` CLI on Toolforge. Use Python instead: ```bash From 12c56bbcd4ed03786a8448cf86ac13ad274800d3 Mon Sep 17 00:00:00 2001 From: Lodewijk <32471480+lgelauff@users.noreply.github.com> Date: Tue, 21 Apr 2026 10:15:45 +0200 Subject: [PATCH 04/10] docs(deployment): update Python version from 3.9 to 3.11 All Toolforge deployments now use python3.11. The 3.9 references were stale and would cause a venv mismatch for anyone following the docs from scratch. Co-Authored-By: Claude Sonnet 4.6 --- deployment.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/deployment.md b/deployment.md index 9d670edb..ae273bda 100644 --- a/deployment.md +++ b/deployment.md @@ -60,7 +60,7 @@ This will build the vue prod bundle and put in backend's `template` and `static` ##### 7. Creating a virtual environment ```bash -toolforge webservice python3.9 shell +toolforge webservice python3.11 shell python3 -m venv $HOME/www/python/venv source $HOME/www/python/venv/bin/activate pip install --upgrade pip wheel @@ -82,7 +82,7 @@ mariadb --defaults-file=~/replica.my.cnf -h tools.db.svc.wikimedia.cloud Date: Tue, 21 Apr 2026 10:16:23 +0200 Subject: [PATCH 05/10] docs(deployment): update Node version from 18 to 20 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit node18 is no longer available on Toolforge. node20 works as a drop-in replacement — no issues observed in practice on montage-dev. Co-Authored-By: Claude Sonnet 4.6 --- deployment.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/deployment.md b/deployment.md index ae273bda..9e8f5089 100644 --- a/deployment.md +++ b/deployment.md @@ -24,7 +24,7 @@ git clone https://github.com/hatnote/montage.git src ##### 4. Make the frontend build ```bash -toolforge webservice node18 shell -m 2G +toolforge webservice node20 shell -m 2G cd $HOME/www/python/src/frontend npm install npm run toolforge:build @@ -119,7 +119,7 @@ git pull ##### 4. Make the frontend build ```bash -toolforge webservice node18 shell -m 2G +toolforge webservice node20 shell -m 2G cd $HOME/www/python/src/frontend npm install npm run toolforge:build From 5a55438d3ff27cf9e8e1a2e5238b4701daf293d5 Mon Sep 17 00:00:00 2001 From: Lodewijk <32471480+lgelauff@users.noreply.github.com> Date: Tue, 21 Apr 2026 11:05:28 +0200 Subject: [PATCH 06/10] docs(deployment): standardize log paths to logs/ subdirectory All three deployments (montage, montage-beta, montage-dev) now use /data/project//logs/ for log files. Added mkdir step to from-scratch instructions and corrected the debugging section note. Co-Authored-By: Claude Sonnet 4.6 --- deployment.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/deployment.md b/deployment.md index 9e8f5089..67a9f565 100644 --- a/deployment.md +++ b/deployment.md @@ -51,6 +51,7 @@ This will build the vue prod bundle and put in backend's `template` and `static` * Add a `cookie_secret: ` * Add the `db_url` with your user database name, and the password from `~/.replica.my.cnf` * The format is: `mysql+pymysql://:@tools.db.svc.wikimedia.cloud/?charset=utf8mb4` +* Create the log directory: `mkdir -p /data/project//logs` * Add `api_log_path: /data/project//logs/montage_api.log` * Add `replay_log_path: /data/project//logs/montage_replay.log` * Add `labs_db: True` @@ -157,7 +158,7 @@ The uwsgi log is at: tail -50 /data/project/montage-beta/uwsgi.log ``` -Note: there is no `logs/` directory inside `src/` — the log is one level up. +Note: the log directory is `/data/project//logs/`, not inside `src/`. ##### Running Python / pip commands From aa44c1214d71fdba2ad7e087b589990e73370373 Mon Sep 17 00:00:00 2001 From: Lodewijk <32471480+lgelauff@users.noreply.github.com> Date: Tue, 21 Apr 2026 11:48:18 +0200 Subject: [PATCH 07/10] docs(dev): clarify local login flow in debug mode The login button triggers OAuth which doesn't work locally. Developers need to navigate to /complete_login directly to establish their session. Verified working locally (redirects to localhost:5173, sets cookie). Co-Authored-By: Claude Sonnet 4.6 --- dev.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dev.md b/dev.md index 6a84a027..b749dbcb 100644 --- a/dev.md +++ b/dev.md @@ -58,7 +58,7 @@ Edit the `.env` file to match your development environment. By default, it's con Montage uses MediaWiki OAuth for authentication. There are two modes for local development: **Default (dev/debug mode) -- no setup required:** -The backend runs with `debug: True` (the default in `config.default.yaml`). In this mode, the OAuth handshake is bypassed entirely and you are automatically logged in as `Slaporte`. This is sufficient for most frontend and backend development. +The backend runs with `debug: True` (the default in `config.default.yaml`). In this mode, the OAuth handshake is bypassed entirely. To establish your session, navigate directly to `http://localhost:5001/complete_login` — this sets the session cookie and redirects you to the frontend. You will be logged in as `Slaporte`. Do not use the login button on the frontend; it triggers the OAuth flow which does not work locally. **Real OAuth (optional) -- for testing the actual login flow:** If you need to test the real OAuth login/logout flow, you need to register an OAuth consumer: From eb31a3f443ca98b65ddceabc4392c0f324b19ae8 Mon Sep 17 00:00:00 2001 From: Lodewijk <32471480+lgelauff@users.noreply.github.com> Date: Tue, 21 Apr 2026 11:49:23 +0200 Subject: [PATCH 08/10] docs(dev): replace hardcoded rdb.py line number with stable reference Line numbers drift as the file grows. Reference the MAINTAINERS constant by name instead. Co-Authored-By: Claude Sonnet 4.6 --- dev.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dev.md b/dev.md index b749dbcb..cec7c84a 100644 --- a/dev.md +++ b/dev.md @@ -101,7 +101,7 @@ log in to the local app in your browser, and then copy the value from the * (Optional) Add your username as the `superuser` in the config. (This will allow you to add `su_to=` to the backend, if you want to test submitting as another juror.) -* Add your username to the list of maintainers in [rdb.py line 113](https://github.com/hatnote/montage/blob/master/montage/rdb.py#L113). +* Add your username to the `MAINTAINERS` list near the top of `montage/rdb.py`. This will give your user top-level permissions in the full app, so you can view some logs (audit logs, active users), add/remove organizers, and get a coordinator view into all campaigns. From 707ee2942ae43dab3a82ab5003b89ae394cfb0cd Mon Sep 17 00:00:00 2001 From: Lodewijk <32471480+lgelauff@users.noreply.github.com> Date: Tue, 21 Apr 2026 12:57:34 +0200 Subject: [PATCH 09/10] docs: add CONTRIBUTING.md Covers contribution workflow, PR expectations, code quality rules, three-environment deployment model, and the requirement to open and discuss an issue before submitting a PR. Co-Authored-By: Claude Sonnet 4.6 --- CONTRIBUTING.md | 60 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..d0f9f7df --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,60 @@ +# Contributing to Montage + +Montage is a small project maintained by volunteers. Contributions are welcome — please read this before opening a PR. + +We follow the [Wikimedia technical spaces Code of Conduct](https://www.mediawiki.org/wiki/Code_of_Conduct). + +## Getting started + +Check the [open issues](https://github.com/hatnote/montage/issues) for something to pick up. Issues tagged [`good-first-issue`](https://github.com/hatnote/montage/issues?q=is%3Aissue+is%3Aopen+label%3Agood-first-issue) are a good starting point. + +Before writing any code, leave a comment on the issue to let maintainers know you're working on it and to confirm the approach. If no issue exists for what you want to do, open one first. **Do not open a PR without a corresponding issue that has received a response from a maintainer** — especially if you are a new contributor. This avoids wasted effort on both sides. + +See [dev.md](dev.md) for local setup instructions. + +## Before opening a PR + +Follow the setup instructions in [dev.md](dev.md), including installing pre-commit hooks and running tests locally. Make sure tests pass before opening a PR. + +## Pull request expectations + +- CI must pass +- One focused change per PR — link the issue it addresses (`Fixes #NNN`) +- If your PR adds or changes database columns, include the SQL migration and flag it in the PR description +- Do not commit screenshots, generated files, or large binaries + +## Code quality + +- Follow existing patterns in the file you're editing +- Backend: read `montage/rdb.py` before writing queries — column names are authoritative there. Note that `.filter()` and `.order_by()` return new objects — always reassign +- Backend: write code that works with both MySQL (production) and SQLite (tests) — no MySQL-only syntax +- Frontend: use Composition API with `