Add a GH action to update the translation issues when .po files change

.github/workflows/refresh-translation-issues.yml

@@ -0,0 +1,164 @@
+name: Refresh translation issues
+
+# Updates the GitHub issues used to track the work in the translations.
+# Each language has a parent issue with a table with the overall stats, plus
+# one sub-issue per catalog (`.po` file).
+#
+# When `.po` files are merged, the stats are recalculated and the issues are
+# rewritten. It only rewrites the body of the issue if there are changes.
+#
+# It uses labels, titles and a hidden HTML comment in the body to find the
+# right issue to update.
+#
+# It does not open new issues for a new language, this step has to be done
+# manually by a maintainer.
+#
+# In case of issues that need a maintainer to act on, it posts a comment to the issue
+# with the Translation Maintenance Log issue (label `po-refresh-tracker`), which
+# will appear in the `#translations` Slack channel.
+on:
+  push:
+    branches: [main]
+    paths:
+      - "locales/**/*.po"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  issues: write
+
+# A second push arriving while one is running should cancel the first one.
+concurrency:
+  group: refresh-translation-issues
+  cancel-in-progress: true
+
+env:
+  # Label to look for the Translation Maintenance Log issue (post to `#translations` Slack channel).
+  TRACKER_LABEL: po-refresh-tracker
+
+jobs:
+  refresh:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      # install `babel` to read the catalogs (`.po` files)
+      - name: Install the catalog reader
+        run: python -m pip install babel
+
+      - name: Collect the issues that track each language
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const fs = require('fs');
+            const { owner, repo } = context.repo;
+
+            // The locale directories are the list of languages, and `lang-XX`
+            // is the label convention label-translations.yml uses.
+            const locales = fs.readdirSync('locales', { withFileTypes: true })
+              .filter(entry => entry.isDirectory())
+              .map(entry => entry.name)
+              .sort();
+
+            const found = {};
+            for (const locale of locales) {
+              const label = `lang-${locale.toUpperCase()}`;
+              // A label no issue carries comes back empty so a language without issues
+              // needs no special case.
+              const issues = await github.paginate(github.rest.issues.listForRepo, {
+                owner, repo, labels: label, state: 'open', per_page: 100,
+              });
+              // Drops PRs with language labels.
+              found[locale] = issues
+                .filter(issue => !issue.pull_request)
+                .map(issue => ({
+                  number: issue.number,
+                  title: issue.title,
+                  body: issue.body || '',
+                }));
+              core.info(
+                `${locale}: ${found[locale].length} open issue(s) labelled ${label}.`,
+              );
+            }
+            fs.writeFileSync('issues.json', JSON.stringify(found));
+
+      # All checks happen in Python so logic can be tested with pytest.
+      - name: Work out which issues need rewriting
+        id: render
+        run: |
+          python scripts/translation/refresh_translation_issues.py \
+            issues.json bodies.json report.md
+          if [ -s report.md ]; then
+            echo "report=true" >> "$GITHUB_OUTPUT"
+            cat report.md
+          else
+            echo "report=false" >> "$GITHUB_OUTPUT"
+          fi
+
+      - name: Rewrite them
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const fs = require('fs');
+            const { owner, repo } = context.repo;
+            const updates = JSON.parse(fs.readFileSync('bodies.json', 'utf8'));
+
+            if (updates.length === 0) {
+              core.info('Every translation issue is already updated.');
+              return;
+            }
+            for (const { number, body } of updates) {
+              // Only update the body so maintainers can change labels and titles.
+              await github.rest.issues.update(
+                { owner, repo, issue_number: number, body },
+              );
+              core.info(`Rewrote #${number}.`);
+            }
+            await core.summary
+              .addRaw(`Rewrote ${updates.length} translation issue(s).`)
+              .write();
+
+      - name: Report anything that needs a maintainer
+        if: steps.render.outputs.report == 'true'
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const fs = require('fs');
+            const label = process.env.TRACKER_LABEL;
+            const { owner, repo } = context.repo;
+            const report = fs.readFileSync('report.md', 'utf8');
+
+            const found = await github.paginate(github.rest.issues.listForRepo, {
+              owner, repo, labels: label, state: 'all', per_page: 100,
+            });
+            // This endpoint counts pull requests as issues, so drop those.
+            const issues = found.filter(i => !i.pull_request);
+
+            if (issues.length === 0) {
+              // If the Translation Maintenance Log issue is not found (nothing with the expected label)
+              // create a warning rather than a failure.
+              core.warning(
+                `No issue carries the \`${label}\` label, so there is nowhere to ` +
+                `report this. Create the translation tracking issue and apply that label.` +
+                `\n\n${report}`,
+              );
+              return;
+            }
+            if (issues.length > 1) {
+              // If there are multiple issues with the translation maintenance label, use the lowest-numbered one and warn.
+              core.warning(
+                `${issues.length} issues carry \`${label}\`: ` +
+                `${issues.map(i => '#' + i.number).join(', ')}. ` +
+                `Using the lowest-numbered one.`,
+              );
+            }
+            const issue = issues.sort((a, b) => a.number - b.number)[0];
+
+            await github.rest.issues.createComment(
+              { owner, repo, issue_number: issue.number, body: report },
+            );
+            core.info(`Commented on #${issue.number}.`);

noxfile.py

@@ -437,9 +437,11 @@ def test_translation_scripts(session):
     """
     Run the unit tests for the translation helper scripts.
 
-    Only pytest is installed since it's the only thing the scripts under test need.
+    Only pytest and babel are installed: babel because stats.py reads the catalogs
+    with it. Installing the project would pull in sphinx and plotly to run tests
+    that never build anything.
     """
-    session.install("pytest")
+    session.install("pytest", "babel")
     session.run("pytest", str(TRANSLATION_SCRIPTS_DIR), *session.posargs)
 
 

pyproject.toml

@@ -4,9 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "python-package-guide"
-dynamic = [
-    "version"
-]
+dynamic = ["version"]
 dependencies = [
     "pydata-sphinx-theme==0.20.0",
     "myst-nb",
@@ -49,3 +47,4 @@ version.source = "vcs"
 [tool.codespell]
 ignore-words = "codespell-ignore.txt"
 skip = "./.git,./.nox,./_static,./_build,codespell-ignore.txt,*.svg"
+ignore-sic = true

scripts/translation/child-issue-template.md

@@ -0,0 +1,34 @@
+> _This issue is created automatically and should not be edited directly._
+
+This issue is for translating **[`{{FILENAME}}`]({{FILE_URL}})** into {{LANGUAGE}}. It is one part of the larger effort tracked in the main issue: {{MAIN_ISSUE_URL}}. Start there if you want the full picture and the setup steps.
+
+Thank you for helping! If this is your first open source contribution, you are in the right place.
+
+> **`index.po` comes first.** It holds the guide's landing page, and translating it is what lets us publish the {{LANGUAGE}} guide on the site. If it still has untranslated strings, it is the most useful file to pick.
+
+If you are working in one of our development sprints at a conference, someone from the pyOpenSci team will be available to help you get set up.
+
+### Claim your lines
+
+To keep our work from overlapping, claim a range of lines before you start:
+
+1. Read the comments below to see which lines are already taken.
+2. Add a comment with the lines you will translate. You can copy this:
+   > I'm working on lines 1–100.
+3. When your part is ready, open a Pull Request and link back to this issue.
+
+You can see line numbers when you open the file on GitHub. If you are not sure how much you can take on, start with a small range. You can always claim more later.
+
+When you link this issue from your Pull Request, please write `Part of #{{ISSUE_NUMBER}}` rather than `Closes #{{ISSUE_NUMBER}}`. Several people are translating this file, so it should stay open until every line is done.
+
+### Resources
+
+- [Translation Guide](https://www.pyopensci.org/python-package-guide/TRANSLATING.html) — the full workflow and how to set up your local environment
+- [Editing the Translation Files](https://www.pyopensci.org/python-package-guide/TRANSLATING.html#editing-the-translation-files) — what a `.po` entry looks like, and tools that help you edit one
+- [Frequently Asked Questions (FAQ)](https://www.pyopensci.org/python-package-guide/TRANSLATING.html#frequently-asked-questions-faq) — common questions answered
+
+### This file, as of {{STATS_DATE}}
+
+**If you come across a string marked `fuzzy`**, it already has a {{LANGUAGE}} translation, but that translation needs a second look to confirm it is correct. Usually this is because the English text changed after the string was translated, though a string can be marked fuzzy for other reasons too. Compare the translation against the English text above it. If it still says the right thing, simply remove the line with the **fuzzy** tag. If it does not, rewrite it and then remove the tag. The Translation Guide explains this further in [What happens when a string has changed in the original English text](https://www.pyopensci.org/python-package-guide/TRANSLATING.html#what-happens-when-a-string-has-changed-in-the-original-english-text).
+
+{{STATS_TABLE}}

scripts/translation/parent-issue-template.md

@@ -0,0 +1,35 @@
+> _This issue is created automatically and should not be edited directly._
+
+We are translating the Python Package Guide into {{LANGUAGE}}, and we need new contributors. If you speak {{LANGUAGE}} and you are new to open source, this is a great place to start!
+
+### What you will be doing
+
+The guide is divided into sections. For each section, the English text is stored in a `.po` file inside `./locales/{{LOCALE}}/LC_MESSAGES`. Next to each English string, there is a space to write the {{LANGUAGE}} translation.
+
+### Getting started
+
+Read the [Translation Guide](https://www.pyopensci.org/python-package-guide/TRANSLATING.html) first. It explains the workflow and how to set up your local environment.
+
+New to open source? You can also work entirely from the GitHub website. Fork the repository into your account, make your changes on your copy, and open a Pull Request. Two parts of the Translation Guide are worth reading first: [Editing the Translation Files](https://www.pyopensci.org/python-package-guide/TRANSLATING.html#editing-the-translation-files), which shows what a `.po` entry looks like, and the [Frequently Asked Questions (FAQ)](https://www.pyopensci.org/python-package-guide/TRANSLATING.html#frequently-asked-questions-faq).
+
+If you are working in one of our development sprints at a conference, someone from the pyOpenSci team will be available to help you get set up.
+
+### Pick a file and claim your work
+
+Each file in the table below has its own issue. Click a file name to open it. There, leave a comment claiming a range of lines to work on, so your work does not overlap with anyone else's. Read the existing comments first to see which lines are already taken.
+
+Look at the untranslated column — a file with a smaller number there is an easier place to start.
+
+**Not sure where to start?** If `index.po` still has untranslated strings, start there. Finishing it is what lets us publish this language, so it is the most useful file to work on when you don't have a particular section in mind.
+
+### See an example
+
+{{EXAMPLE_SECTION}}
+
+### Translation status as of {{STATS_DATE}}
+
+The table shows the number of strings in each file.
+
+**If you come across a string marked `fuzzy`**, it already has a {{LANGUAGE}} translation, but that translation needs a second look to confirm it is correct. Usually this is because the English text changed after the string was translated, though a string can be marked fuzzy for other reasons too. Compare the translation against the English text above it. If it still says the right thing, simply remove the line with the **fuzzy** tag. If it does not, rewrite it and then remove the tag. The Translation Guide explains this further in [What happens when a string has changed in the original English text](https://www.pyopensci.org/python-package-guide/TRANSLATING.html#what-happens-when-a-string-has-changed-in-the-original-english-text).
+
+{{STATS_TABLE}}