Skip to content

repo sync #22063

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 16, 2022
Merged

repo sync #22063

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bd23217
Create translation-health-report.yml (#32486)
heiskr Nov 16, 2022
b1b220a
Test workflow for WIP languages and the translation pipeline (#32532)
rsese Nov 16, 2022
d2a39cb
Merge branch 'main' into repo-sync
Octomerger Nov 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 132 additions & 0 deletions .github/workflows/translation-health-report.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
name: Translation health report

# **What it does**: Provides errors and summary statistics on rendering translated content.
# **Why we have it**: To improve our translations by having clearer visibility.
# **Who does it impact**: Docs engineering, Microsoft translators.

on:
workflow_dispatch:
schedule:
- cron: '20 16 * * *' # Run every day at 16:20 UTC / 8:20 PST

permissions:
contents: read

jobs:
create-translation-health-report:
name: Create translation health report
if: github.repository == 'github/docs-internal'
runs-on: ubuntu-latest
# This sets a maximum execution time of 300 minutes (5 hours)
# to prevent the workflow from running longer than necessary.
timeout-minutes: 300
strategy:
fail-fast: false
max-parallel: 1
matrix:
include:
- language: es
language_dir: translations/es-ES
language_repo: github/docs-internal.es-es

- language: ja
language_dir: translations/ja-JP
language_repo: github/docs-internal.ja-jp

- language: pt
language_dir: translations/pt-BR
language_repo: github/docs-internal.pt-br

- language: cn
language_dir: translations/zh-CN
language_repo: github/docs-internal.zh-cn

# We'll be ready to add the following languages in a future effort.

# - language: ru
# language_dir: translations/ru-RU
# language_repo: github/docs-internal.ru-ru

# - language: ko
# language_dir: translations/ko-KR
# language_repo: github/docs-internal.ko-kr

# - language: fr
# language_dir: translations/fr-FR
# language_repo: github/docs-internal.fr-fr

# - language: de
# language_dir: translations/de-DE
# language_repo: github/docs-internal.de-de

steps:
- name: Checkout the docs-internal repo
uses: actions/checkout@dcd71f646680f2efd8db4afa5ad64fdcba30e748

- name: Remove all language translations
run: |
git rm -rf --quiet ${{ matrix.language_dir }}/content
git rm -rf --quiet ${{ matrix.language_dir }}/data

- name: Checkout the language-specific repo
uses: actions/checkout@dcd71f646680f2efd8db4afa5ad64fdcba30e748
with:
repository: ${{ matrix.language_repo }}
token: ${{ secrets.DOCUBOT_READORG_REPO_WORKFLOW_SCOPES }}
path: ${{ matrix.language_dir }}

- name: Get language SHA
run: |
gitref=$(cd ${{ matrix.language_dir }} && git rev-parse --short HEAD)
echo "gitref=$gitref" >> $GITHUB_ENV

- name: 'Setup node'
uses: actions/setup-node@17f8bd926464a1afa4c6a11669539e9c1ba77048
with:
node-version: '16.17.0'

- name: npm ci
run: npm ci

- name: Create translation health report
run: |
translation_health_report=$( \
node script/i18n/create-translation-health-report.js \
--language ${{ matrix.language }} \
--gitref ${{ env.gitref }} \
| jq -Rsa .
)
echo "translation_health_report=$translation_health_report" >> $GITHUB_ENV

- name: Log in to Azure
uses: azure/login@1f63701bf3e6892515f1b7ce2d2bf1708b46beaf
with:
creds: ${{ secrets.PROD_AZURE_CREDENTIALS }}

- name: Upload to Azure blob storage
uses: azure/CLI@61bb69d64d613b52663984bf12d6bac8fd7b3cc8
with:
inlineScript: |
az storage blob upload \
--name "${{ matrix.language }}-latest.json" \
--data $translation_health_report \
--container-name translation-health-reports
az storage blob upload \
--name "${{ matrix.language }}-$(date +%Y-%m-%d).json" \
--data $translation_health_report \
--container-name translation-health-reports

- name: Log out from Azure
if: always()
run: |
az logout

# Emit a notification for the first responder to triage if the workflow failed.
- name: Send Slack notification if workflow failed
uses: someimportantcompany/github-actions-slack-message@f8d28715e7b8a4717047d23f48c39827cacad340
if: failure()
with:
channel: ${{ secrets.DOCS_ALERTS_SLACK_CHANNEL_ID }}
bot-token: ${{ secrets.SLACK_DOCS_BOT_TOKEN }}
color: failure
text: 'The health report for ${{ matrix.language }} failed.'
173 changes: 173 additions & 0 deletions .github/workflows/wip-langs-create-translation-batch-pr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
name: WIP Languages Create translation Batch Pull Request

# **What it does**:
# - Creates one pull request per WIP language after running a series of automated checks,
# removing translations that are broken in any known way
# **Why we have it**:
# - To test the translation pipeline for WIP languages
# **Who does it impact**: Helps test how WIP languages will behave in CI

on:
workflow_dispatch:

permissions:
contents: write

jobs:
create-translation-batch-for-wip-langs:
name: Create translation batch for WIP languages
if: github.repository == 'github/docs-internal'
runs-on: ubuntu-latest
# A sync's average run time is ~3.2 hours.
# This sets a maximum execution time of 300 minutes (5 hours) to prevent the workflow from running longer than necessary.
timeout-minutes: 300
strategy:
fail-fast: false
max-parallel: 1
matrix:
include:
- language: ru
language_dir: translations/ru-RU
language_repo: github/docs-internal.ru-ru

- language: ko
language_dir: translations/ko-KR
language_repo: github/docs-internal.ko-kr

- language: fr
language_dir: translations/fr-FR
language_repo: github/docs-internal.fr-fr

- language: de
language_dir: translations/de-DE
language_repo: github/docs-internal.de-de

steps:
- name: Set branch name
id: set-branch
run: |
echo "::set-output name=BRANCH_NAME::msft-translation-batch-${{ matrix.language }}-$(date +%Y-%m-%d__%H-%M)"

- run: git config --global user.name "docubot"
- run: git config --global user.email "67483024+docubot@users.noreply.github.com"

- name: Checkout the docs-internal repo
uses: actions/checkout@dcd71f646680f2efd8db4afa5ad64fdcba30e748
with:
fetch-depth: 0
lfs: true

- name: Create a branch for the current language
run: git checkout -b ${{ steps.set-branch.outputs.BRANCH_NAME }}

- name: Remove unwanted git hooks
run: rm .git/hooks/post-checkout

- name: Remove all language translations
run: |
git rm -rf --quiet ${{ matrix.language_dir }}/content
git rm -rf --quiet ${{ matrix.language_dir }}/data

- name: Checkout the language-specific repo
uses: actions/checkout@dcd71f646680f2efd8db4afa5ad64fdcba30e748
with:
repository: ${{ matrix.language_repo }}
token: ${{ secrets.DOCUBOT_READORG_REPO_WORKFLOW_SCOPES }}
path: ${{ matrix.language_dir }}

- name: Remove .git from the language-specific repo
run: rm -rf ${{ matrix.language_dir }}/.git

- name: Commit translated files
run: |
git add ${{ matrix.language_dir }}
git commit -m "Add translations" || echo "Nothing to commit"

- name: 'Setup node'
uses: actions/setup-node@17f8bd926464a1afa4c6a11669539e9c1ba77048
with:
node-version: '16.17.0'

- run: npm ci

- name: Homogenize frontmatter
run: |
node script/i18n/homogenize-frontmatter.js
git add ${{ matrix.language_dir }} && git commit -m "Run script/i18n/homogenize-frontmatter.js" || echo "Nothing to commit"

- name: Fix translation errors
run: |
node script/i18n/fix-translation-errors.js
git add ${{ matrix.language_dir }} && git commit -m "Run script/i18n/fix-translation-errors.js" || echo "Nothing to commit"

- name: Check rendering
run: |
node script/i18n/lint-translation-files.js --check rendering | tee -a /tmp/batch.log | cat
git add ${{ matrix.language_dir }} && git commit -m "Run script/i18n/lint-translation-files.js --check rendering" || echo "Nothing to commit"

- name: Reset files with broken liquid tags
run: |
node script/i18n/msft-reset-files-with-broken-liquid-tags.js --language=${{ matrix.language }} | tee -a /tmp/batch.log | cat
git add ${{ matrix.language_dir }} && git commit -m "run script/i18n/msft-reset-files-with-broken-liquid-tags.js --language=${{ matrix.language }}" || echo "Nothing to commit"

- name: Check in CSV report
run: |
mkdir -p translations/log
csvFile=translations/log/msft-${{ matrix.language }}-resets.csv
script/i18n/msft-report-reset-files.js --report-type=csv --language=${{ matrix.language }} --log-file=/tmp/batch.log > $csvFile
git add -f $csvFile && git commit -m "Check in ${{ matrix.language }} CSV report" || echo "Nothing to commit"

- name: Write the reported files that were reset to /tmp/pr-body.txt
run: script/i18n/msft-report-reset-files.js --report-type=pull-request-body --language=${{ matrix.language }} --log-file=/tmp/batch.log --csv-path=${{ steps.set-branch.outputs.BRANCH_NAME }}/translations/log/msft-${{ matrix.language }}-resets.csv > /tmp/pr-body.txt

- name: Push filtered translations
run: git push origin ${{ steps.set-branch.outputs.BRANCH_NAME }}

- name: Close existing stale batches
uses: lee-dohm/close-matching-issues@e9e43aad2fa6f06a058cedfd8fb975fd93b56d8f
with:
token: ${{ secrets.OCTOMERGER_PAT_WITH_REPO_AND_WORKFLOW_SCOPE }}
query: 'type:pr label:translation-batch-${{ matrix.language }}'

- name: Create translation batch pull request
env:
GITHUB_TOKEN: ${{ secrets.DOCUBOT_REPO_PAT }}
TITLE: '[DO NOT MERGE - WIP Language test]: New translation batch for ${{ matrix.language }}'
BASE: 'main'
HEAD: ${{ steps.set-branch.outputs.BRANCH_NAME }}
LANGUAGE: ${{ matrix.language }}
BODY_FILE: '/tmp/pr-body.txt'
run: .github/actions-scripts/msft-create-translation-batch-pr.js

# - name: Approve PR
# if: github.ref_name == 'main'
# env:
# GITHUB_TOKEN: ${{ secrets.OCTOMERGER_PAT_WITH_REPO_AND_WORKFLOW_SCOPE }}
# run: gh pr review --approve || echo "Nothing to approve"

# - name: Set auto-merge
# if: github.ref_name == 'main'
# env:
# GITHUB_TOKEN: ${{ secrets.OCTOMERGER_PAT_WITH_REPO_AND_WORKFLOW_SCOPE }}
# run: gh pr merge ${{ steps.set-branch.outputs.BRANCH_NAME }} --auto --squash || echo "Nothing to merge"

# When the maximum execution time is reached for this job, Actions cancels the workflow run.
# This emits a notification for the first responder to triage.
# - name: Send Slack notification if workflow is cancelled
# uses: someimportantcompany/github-actions-slack-message@f8d28715e7b8a4717047d23f48c39827cacad340
# if: cancelled()
# with:
# channel: ${{ secrets.DOCS_ALERTS_SLACK_CHANNEL_ID }}
# bot-token: ${{ secrets.SLACK_DOCS_BOT_TOKEN }}🎉
# color: failure
# text: 'The new translation batch for ${{ matrix.language }} was cancelled.'

# Emit a notification for the first responder to triage if the workflow failed.
# - name: Send Slack notification if workflow failed
# uses: someimportantcompany/github-actions-slack-message@f8d28715e7b8a4717047d23f48c39827cacad340
# if: failure()
# with:
# channel: ${{ secrets.DOCS_ALERTS_SLACK_CHANNEL_ID }}
# bot-token: ${{ secrets.SLACK_DOCS_BOT_TOKEN }}
# color: failure
# text: 'The new translation batch for ${{ matrix.language }} failed.'
Loading