Skip to content
Merged
62 changes: 62 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Contributing to Montage

Montage is a small project maintained by volunteers. Contributions are welcome — please read this before opening a PR.

We follow the [Wikimedia technical spaces Code of Conduct](https://www.mediawiki.org/wiki/Code_of_Conduct).

## Getting started

Check the [open issues](https://github.com/hatnote/montage/issues) for something to pick up. Issues tagged [`good-first-issue`](https://github.com/hatnote/montage/issues?q=is%3Aissue+is%3Aopen+label%3Agood-first-issue) are a good starting point.

Before writing any code, leave a comment on the issue to let maintainers know you're working on it and to confirm the approach. If no issue exists for what you want to do, open one first. **Do not open a PR without a corresponding issue that has received a response from a maintainer** — especially if you are a new contributor. This avoids wasted effort on both sides.

See [dev.md](dev.md) for local setup instructions.

## Before opening a PR

Follow the setup instructions in [dev.md](dev.md), including installing pre-commit hooks and running tests locally. Make sure tests pass before opening a PR.

## Pull request expectations

- CI must pass
- One focused change per PR — link the issue it addresses (`Fixes #NNN`)
- If your PR adds or changes database columns, include the SQL migration and flag it in the PR description
- Do not commit screenshots, generated files, or large binaries

## Code quality

- Follow existing patterns in the file you're editing
- Backend: read `montage/rdb.py` before writing queries — column names are authoritative there. Note that `.filter()` and `.order_by()` return new objects — always reassign
- Backend: write code that works with both MySQL (production) and SQLite (tests) — no MySQL-only syntax
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that Docker support has gotten pretty good, I don't know if sqlite support is strictly necessary, tbh.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that also true locally? I like the simpler approach where possible, but would be nice indeed to split that into a separate PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah docker is just for local, so docker compose would be the best way to get production-like env locally now.

- Frontend: use Composition API with `<script setup>` — no Options API
- All user-facing strings must go in `frontend/src/i18n/en.json` and use `t('key')` — TranslateWiki handles other locales, do not edit them manually

## When to open a PR for review

Only open a PR for review when you consider it ready to merge. If your work is still in progress, open it as a **draft PR** — maintainers will not review draft PRs.

A PR is ready for review when:
- It does one thing and does it completely
- CI passes
- You have tested it yourself and are confident it works
- You would be comfortable if it were merged as-is

Do not open a PR to ask for direction or feedback on an approach — use the issue for that conversation first.

When you are a new contributor to this codebase (e.g. not part of the maintainer team) we kindly ask you to make a screen recording of the platform with human voice-over that reproduces the issue locally, and shows the change after the implemented changes. Please also make sure to be explicit about your testing approach and the logic used.

## Deployment

Only maintainers have access to the three Toolforge environments:

| Environment | Purpose |
|-------------|---------|
| montage-dev | Flexible test environment — maintainers can deploy any branch here |
| montage-beta | Tracks `master` — used by volunteers to catch bugs before production |
| montage-prod | Stable production — promoted manually when beta is stable |

Contributors do not need Toolforge access. Push your branch to GitHub and open a PR against `master`. A maintainer can then deploy your branch to montage-dev to test it — either in isolation or combined with other PRs to check interaction effects.

## Questions

Leave a comment on the relevant GitHub issue, or reach out on [Commons talk:Montage](https://commons.wikimedia.org/wiki/Commons_talk:Montage).
105 changes: 92 additions & 13 deletions deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ git clone https://github.com/hatnote/montage.git src

##### 4. Make the frontend build
```bash
toolforge webservice node18 shell -m 2G
toolforge webservice node20 shell -m 2G
cd $HOME/www/python/src/frontend
npm install
npm run toolforge:build
Expand All @@ -33,17 +33,25 @@ exit
This will build the vue prod bundle and put in backend's `template` and `static` directory.

##### 5. Create your database
* Get the user name of database (`cat ~/replica.my.cnf`)
* Open up MariaDB with `sql local`
* Create a [Toolforge user database](https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#User_databases) (`create database <user>__<db name>;`), and remember the name for the config
* Get the username and password from `cat ~/replica.my.cnf`
* Connect to MariaDB:
```bash
mariadb --defaults-file=~/replica.my.cnf -h tools.db.svc.wikimedia.cloud
```
* Create a [Toolforge user database](https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#User_databases) with `utf8mb4` charset, and remember the name for the config:
```sql
CREATE DATABASE `<user>__<db name>` DEFAULT CHARACTER SET utf8mb4 DEFAULT COLLATE utf8mb4_unicode_ci;
EXIT;
```

##### 6. Set up the montage config
* Make a copy of `config.default.yaml` for your environment
* You may need to update `USER_ENV_MAP` in `montage/utils.py` if you need to add a new environment
* Add the `oauth_consumer_token` and `oauth_secret_token`
* Add a `cookie_secret: <your random secret>`
* Add the `db_url` with your user database name, and the password from `~/.replica.my.cnf`
* The format is: `mysql://<user>:<password>@tools.labsdb/<db name>?charset=utf8`
* The format is: `mysql+pymysql://<user>:<password>@tools.db.svc.wikimedia.cloud/<db name>?charset=utf8mb4`
* Create the log directory: `mkdir -p /data/project/<project>/logs`
* Add `api_log_path: /data/project/<project>/logs/montage_api.log`
* Add `replay_log_path: /data/project/<project>/logs/montage_replay.log`
* Add `labs_db: True`
Expand All @@ -53,20 +61,32 @@ This will build the vue prod bundle and put in backend's `template` and `static`

##### 7. Creating a virtual environment
```bash
toolforge webservice python3.9 shell
toolforge webservice python3.11 shell
python3 -m venv $HOME/www/python/venv
source $HOME/www/python/venv/bin/activate
pip install --upgrade pip wheel
pip install -r $HOME/www/python/src/requirements.txt
exit
```

##### 8. Start the backend service
##### 8. Initialise the database schema
```bash
toolforge webservice python3.9 start
cd $HOME/www/python/src
source $HOME/www/python/venv/bin/activate
python3 montage/create_schema.py
```

##### 9. Testing of deployment
If this is an upgrade of an existing deployment (not a fresh install), run the migration SQL instead:
```bash
mariadb --defaults-file=~/replica.my.cnf -h tools.db.svc.wikimedia.cloud <db name> < tools/migrate_prod_db.sql
```

##### 9. Start the backend service
```bash
toolforge webservice python3.11 start
```

##### 10. Testing of deployment
* Visit /meta to see the API. Example: https://montage-beta.toolforge.org/meta/
* In the top section, you should see that the service was restarted in the last few seconds/minutes.

Expand Down Expand Up @@ -100,7 +120,7 @@ git pull

##### 4. Make the frontend build
```bash
toolforge webservice node18 shell -m 2G
toolforge webservice node20 shell -m 2G
cd $HOME/www/python/src/frontend
npm install
npm run toolforge:build
Expand All @@ -110,17 +130,76 @@ exit
##### 5. (Optional) Install python packages
If you added new python packages in changes then you have to install them in pod.
```bash
toolforge webservice python3.9 shell
toolforge webservice python3.11 shell
source $HOME/www/python/venv/bin/activate
pip install -r $HOME/www/python/src/requirements.txt
exit
```

##### 8. Restart the backend service
```bash
toolforge webservice python3.9 restart
toolforge webservice python3.11 restart
```

##### 9. Testing of deployment
* Visit /meta to see the API. Example: https://montage-beta.toolforge.org/meta/
* In the top section, you should see that the service was restarted in the last few seconds/minutes.
* In the top section, you should see that the service was restarted in the last few seconds/minutes.


---


## Debugging

##### Viewing logs

The uwsgi log is at:
```bash
tail -50 /data/project/montage-beta/uwsgi.log
```

Note: the log directory is `/data/project/<project>/logs/`, not inside `src/`.

##### Running Python / pip commands

Always run `pip install` and Python diagnostics inside the webservice shell, not the bastion shell. The two environments use different venvs:

```bash
toolforge webservice python3.11 shell
# venv is activated automatically
pip install -r ~/www/python/src/requirements.txt
python3 -c "import montage.app"
exit
```

Running `pip` on the bastion shell installs to a different venv and will not affect the running service.

##### Restarting the service

```bash
toolforge webservice python3.11 restart
```

##### Inspecting the MariaDB database

Always pass `-h tools.db.svc.wikimedia.cloud` explicitly — there is no local socket on the Toolforge bastion:

```bash
mariadb --defaults-file=~/replica.my.cnf -h tools.db.svc.wikimedia.cloud <db name>
```

Example queries:
```sql
SELECT COUNT(*) FROM entries;
DESCRIBE entries;
```

##### Inspecting the SQLite database (legacy / dev only)

Note: montage-beta originally used SQLite and the file (`tmp_montage.db`) may still exist alongside the MariaDB setup. It is no longer used by the running service once the config switches to `mysql+pymysql://`.

There is no `sqlite3` CLI on Toolforge. Use Python instead:

```bash
python3 -c 'import sqlite3; c=sqlite3.connect("/data/project/montage-beta/www/python/src/tmp_montage.db"); print(c.execute("SELECT COUNT(*) FROM entries").fetchone())'
Comment on lines +199 to +204
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we don't need sqlite, we won't need this caveat :)

```
4 changes: 2 additions & 2 deletions dev.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ Edit the `.env` file to match your development environment. By default, it's con
Montage uses MediaWiki OAuth for authentication. There are two modes for local development:

**Default (dev/debug mode) -- no setup required:**
The backend runs with `debug: True` (the default in `config.default.yaml`). In this mode, the OAuth handshake is bypassed entirely and you are automatically logged in as `Slaporte`. This is sufficient for most frontend and backend development.
The backend runs with `debug: True` (the default in `config.default.yaml`). In this mode, the OAuth handshake is bypassed entirely. To establish your session, navigate directly to `http://localhost:5001/complete_login` — this sets the session cookie and redirects you to the frontend. You will be logged in as `Slaporte`. Do not use the login button on the frontend; it triggers the OAuth flow which does not work locally.

**Real OAuth (optional) -- for testing the actual login flow:**
If you need to test the real OAuth login/logout flow, you need to register an OAuth consumer:
Expand Down Expand Up @@ -101,7 +101,7 @@ log in to the local app in your browser, and then copy the value from the
* (Optional) Add your username as the `superuser` in the config. (This will allow you to
add `su_to=<any user>` to the backend, if you want to test submitting as another
juror.)
* Add your username to the list of maintainers in [rdb.py line 113](https://github.com/hatnote/montage/blob/master/montage/rdb.py#L113).
* Add your username to the `MAINTAINERS` list near the top of `montage/rdb.py`.
This will give your user top-level permissions in the full app, so you can view
some logs (audit logs, active users), add/remove organizers, and get a
coordinator view into all campaigns.
Expand Down
Loading