Skip to content

Ci/centralize external docs sync#814

Merged
ccamel merged 8 commits intomainfrom
ci/centralize-external-docs-sync
Mar 19, 2026
Merged

Ci/centralize external docs sync#814
ccamel merged 8 commits intomainfrom
ci/centralize-external-docs-sync

Conversation

@ccamel
Copy link
Member

@ccamel ccamel commented Mar 19, 2026

Changes the external documentation update model from a mix of upstream-triggered and notification-based mechanisms to an exclusively pull-based approach.

Summary by CodeRabbit

  • New Features

    • Automated sync workflow to import and version external documentation, with manual trigger and dry-run support.
  • Documentation

    • Added a “Documentation sources” section describing external doc composition and directory mappings.
  • Chores

    • Scheduled daily sync, automated signed-commit flow, new sync scripts, removed the prior update workflow, adjusted editorconfig and markdown lint ignores.

@ccamel ccamel self-assigned this Mar 19, 2026
@vercel
Copy link

vercel bot commented Mar 19, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment Mar 19, 2026 7:29pm

Request Review

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 19, 2026

Note

Reviews paused

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds an automated docs sync: a Node.js script and GitHub Actions workflow that pull versioned docs from external GitHub repositories, sync them into local Docusaurus version directories (including live refs), plus manifest/config, README notes, and removal of an older workflow.

Changes

Cohort / File(s) Summary
Sync script
​.github/scripts/sync-external-docs.mjs
New executable Node.js script: loads .github/sync-external-docs-sources.json, discovers remote tags (semver or regex families), downloads/extracts tarballs, caches snapshots, rsyncs docs paths into local sections, and invokes Docusaurus versioning; supports --dry-run and error handling.
Manifest / Sources
​.github/sync-external-docs-sources.json
New manifest with five external doc sources specifying repository, section, docs_path, live_ref, and tag selection (semver or custom tag_patterns with excluded_families).
GitHub Actions — new workflow
​.github/workflows/sync-external-docs.yml
New workflow (manual + daily cron) that checks out repo using secrets.OPS_TOKEN, imports GPG key, sets up Node.js/Yarn, runs the sync script, detects working-tree changes, and creates a signed commit when changes exist.
GitHub Actions — removed workflow
​.github/workflows/update-versioned-docs.yml
Removed previous workflow that handled manual versioned-docs updates and created PRs/commits; superseded by the automated sync workflow.
Repo metadata & docs
package.json, .editorconfig, README.md, ​.github/workflows/lint.yml
Added npm scripts sync:external-docs and sync:external-docs:dry-run; JS file editorconfig overrides; README section documenting documentation sources; extended markdown-lint ignore globs for predicate docs.

Sequence Diagram

sequenceDiagram
    participant GHA as GitHub Actions (Workflow)
    participant Script as Sync Script (Node.js)
    participant GHAPI as GitHub API
    participant FS as File System
    participant Docusaurus as Docusaurus CLI
    participant Git as Git

    GHA->>GHA: Checkout repo & setup env
    GHA->>Script: Execute sync-external-docs.mjs

    Script->>Script: Load manifest & init caches
    Script->>GHAPI: Fetch remote tags for each repo
    GHAPI-->>Script: Return tag refs
    Script->>Script: Parse tags into families/versions
    Script->>FS: Read local `<section>_versions.json`

    loop For each pending tag
        Script->>GHAPI: Download tarball for `repo@tag`
        GHAPI-->>Script: Tarball bytes
        Script->>FS: Write & extract tarball to temp cache
        FS-->>Script: Extraction path
        Script->>FS: Validate `docs_path` exists
        Script->>FS: rsync docs_path -> local section dir
        Script->>Docusaurus: Run `docusaurus docs:version:<section> <tag>` (unless dry-run)
        Docusaurus-->>Script: Versioning result
    end

    Script->>GHAPI: Materialize live_ref snapshot
    Script->>FS: Sync live docs_path -> live section
    Script-->>GHA: Exit (status)

    GHA->>Git: git status --porcelain
    Git-->>GHA: Changes present?
    alt Changes detected
        GHA->>Git: Create signed commit with bot identity
        Git-->>GHA: Commit pushed
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~40 minutes

Suggested reviewers

  • amimart

Poem

🐇 I hopped through tags and tarball streams,
I fetched the docs and stitched their seams,
I synced the versions, leaf by leaf,
Live refs tucked cozy, history brief,
Signed commit burrowed—hop, the portal beams!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Ci/centralize external docs sync' accurately captures the main change: introducing a centralized, automated CI-based system for synchronizing external documentation instead of the previous mixed approach.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/centralize-external-docs-sync
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Member

@bot-anik bot-anik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Device URL
desktop http://localhost:3000/

Device URL
mobile http://localhost:3000/

Not what you expected? Are your scores flaky? GitHub runners could be the cause.
Try running on Foo instead

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
.github/scripts/sync-external-docs.mjs (2)

11-14: Consider cleaning up temporary directory on exit.

The script creates a temp directory at tempRoot but never cleans it up. While the OS will eventually purge /tmp, explicit cleanup prevents disk accumulation during repeated local runs or long-lived CI runners.

♻️ Proposed cleanup handler
 const tempRoot = mkdtempSync(path.join(os.tmpdir(), 'docs-sync-'))
+
+process.on('exit', () => {
+  try {
+    fs.rm(tempRoot, { recursive: true, force: true })
+  } catch {
+    // Ignore cleanup errors
+  }
+})

Note: Since fs is imported as promises, you'd need to use rmSync from the sync API or handle this differently.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/scripts/sync-external-docs.mjs around lines 11 - 14, The temp
directory created by mkdtempSync and stored in tempRoot is never removed; add a
cleanup handler that removes tempRoot on process exit/signals (e.g., 'exit',
'SIGINT', 'SIGTERM', 'uncaughtException') to avoid accumulating temp files.
Implement it by calling a synchronous removal (fs.rmSync or fs.rmdirSync) or
safely awaiting fs.promises.rm in async handlers, ensure the handler checks that
tempRoot is set and exists and ignores ENOENT, and register it near where
tempRoot is created so mkdtempSync/tempRoot are the referenced symbols.

55-68: Family prefix matching with startsWith is fragile for multi-family scenarios.

The version.startsWith(family) filter at line 57 works correctly with current data, but is brittle if contract families are added with overlapping prefixes. For example, if families "axone-objectarium-v" and "axone-objectarium-v-utils-v" both existed, startsWith("axone-objectarium-v") would incorrectly match versions from both families. Currently only one contract family ("axone-gov-v") exists in tracked versions, so this doesn't manifest, but consider using more precise matching (e.g., splitting on the version part) to prevent issues when new contract families are added.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/scripts/sync-external-docs.mjs around lines 55 - 68, The filtering
using trackedVersions.filter(version => version.startsWith(family)) is fragile;
instead derive each tracked version's family the same way remoteTags does and
compare equality. Update the block (variables remoteTags, family,
trackedVersions, trackedForFamily) to compute trackedVersionFamily for each
tracked version (e.g., split or parse the version string the same way you
compute tag.family from remoteTags) and use trackedVersions.filter(version =>
trackedVersionFamily === family) so only exact family matches are selected
before building trackedSet and missingForFamily.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In @.github/scripts/sync-external-docs.mjs:
- Around line 11-14: The temp directory created by mkdtempSync and stored in
tempRoot is never removed; add a cleanup handler that removes tempRoot on
process exit/signals (e.g., 'exit', 'SIGINT', 'SIGTERM', 'uncaughtException') to
avoid accumulating temp files. Implement it by calling a synchronous removal
(fs.rmSync or fs.rmdirSync) or safely awaiting fs.promises.rm in async handlers,
ensure the handler checks that tempRoot is set and exists and ignores ENOENT,
and register it near where tempRoot is created so mkdtempSync/tempRoot are the
referenced symbols.
- Around line 55-68: The filtering using trackedVersions.filter(version =>
version.startsWith(family)) is fragile; instead derive each tracked version's
family the same way remoteTags does and compare equality. Update the block
(variables remoteTags, family, trackedVersions, trackedForFamily) to compute
trackedVersionFamily for each tracked version (e.g., split or parse the version
string the same way you compute tag.family from remoteTags) and use
trackedVersions.filter(version => trackedVersionFamily === family) so only exact
family matches are selected before building trackedSet and missingForFamily.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 4e2f6268-73b8-4e3f-a698-a03d76bf48dc

📥 Commits

Reviewing files that changed from the base of the PR and between c0c1c05 and 04b832e.

📒 Files selected for processing (5)
  • .github/scripts/sync-external-docs.mjs
  • .github/sync-external-docs-sources.json
  • .github/workflows/sync-external-docs.yml
  • .github/workflows/update-versioned-docs.yml
  • README.md
💤 Files with no reviewable changes (1)
  • .github/workflows/update-versioned-docs.yml

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (2)
.github/scripts/sync-external-docs.mjs (2)

261-271: Dry-run still performs downloads and extractions.

The tar exception on Line 262 means dry-run mode still downloads tarballs and extracts them—only rsync and docusaurus commands are skipped. This is reasonable for validation purposes but worth documenting if not already clear to users.

Consider adding a timeout to execFileSync for commands like rsync that could potentially hang.

📝 Proposed enhancement
   execFileSync(command, commandArgs, {
     cwd: workspace,
-    stdio: 'inherit'
+    stdio: 'inherit',
+    timeout: 300000 // 5 minutes for rsync operations
   })
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/scripts/sync-external-docs.mjs around lines 261 - 271, The run
function currently bypasses dry-run for only the 'tar' command (dryRun variable
and run(command, commandArgs)), causing downloads/extractions to still occur;
update run to either skip 'tar' as well when dryRun is true (so nothing is
executed) or explicitly document this behavior if intentional, and add a timeout
option to the execFileSync invocation to prevent long-running hangs for commands
like 'rsync' (modify the execFileSync call site to include { cwd: workspace,
stdio: 'inherit', timeout: <reasonable-ms> } or conditionally set timeouts for
specific commands).

106-109: Consider adding a timeout to prevent indefinite hangs.

The execFileSync call has no timeout, which could cause the script (and CI) to hang indefinitely if the git command stalls due to network issues.

⏱️ Proposed fix to add timeout
   const output = execFileSync('git', ['ls-remote', '--tags', '--refs', repositoryUrl], {
     cwd: workspace,
-    encoding: 'utf8'
+    encoding: 'utf8',
+    timeout: 60000 // 60 seconds
   })
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/scripts/sync-external-docs.mjs around lines 106 - 109, The
execFileSync call that populates the output variable can hang without a timeout;
update the execFileSync invocation (the call using execFileSync with
repositoryUrl and workspace) to include a sensible timeout option (e.g., 2–5
minutes, configurable via env var) and wrap the call in a try/catch so timeouts
throw a clear error and the script exits non‑zero with a helpful log message;
ensure you reference the same execFileSync call that assigns output and preserve
encoding and cwd options when adding the timeout.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/scripts/sync-external-docs.mjs:
- Around line 190-199: readTrackedVersions currently calls JSON.parse on the
file at versionsPath without handling invalid JSON; wrap the read/parse in a
try-catch in readTrackedVersions to catch JSON parsing errors and either return
an empty array or rethrow a new, clearer Error that includes versionsPath and
the original error message; ensure parsed is still validated with
Array.isArray(parsed) before returning and reference the readTrackedVersions
function, versionsPath variable and parsed value when adding the error handling.
- Line 14: The temp directory created by mkdtempSync and assigned to tempRoot is
never removed; update the main() flow to remove tempRoot when finished (or on
error) by adding cleanup logic that deletes the directory (e.g., using fs.rmSync
or fs.rmdirSync with recursive:true) in a finally block or at the end of main();
ensure references to tempRoot are used for the removal and that cleanup runs
even on exceptions so temporary archives are not left behind.

---

Nitpick comments:
In @.github/scripts/sync-external-docs.mjs:
- Around line 261-271: The run function currently bypasses dry-run for only the
'tar' command (dryRun variable and run(command, commandArgs)), causing
downloads/extractions to still occur; update run to either skip 'tar' as well
when dryRun is true (so nothing is executed) or explicitly document this
behavior if intentional, and add a timeout option to the execFileSync invocation
to prevent long-running hangs for commands like 'rsync' (modify the execFileSync
call site to include { cwd: workspace, stdio: 'inherit', timeout:
<reasonable-ms> } or conditionally set timeouts for specific commands).
- Around line 106-109: The execFileSync call that populates the output variable
can hang without a timeout; update the execFileSync invocation (the call using
execFileSync with repositoryUrl and workspace) to include a sensible timeout
option (e.g., 2–5 minutes, configurable via env var) and wrap the call in a
try/catch so timeouts throw a clear error and the script exits non‑zero with a
helpful log message; ensure you reference the same execFileSync call that
assigns output and preserve encoding and cwd options when adding the timeout.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 53e8d14b-43a9-44ab-a49b-b7a5b4d90c85

📥 Commits

Reviewing files that changed from the base of the PR and between 04b832e and 5f32ddb.

📒 Files selected for processing (5)
  • .editorconfig
  • .github/scripts/sync-external-docs.mjs
  • .github/sync-external-docs-sources.json
  • .github/workflows/sync-external-docs.yml
  • package.json
✅ Files skipped from review due to trivial changes (4)
  • .editorconfig
  • package.json
  • .github/sync-external-docs-sources.json
  • .github/workflows/sync-external-docs.yml

@ccamel
Copy link
Member Author

ccamel commented Mar 19, 2026

@coderabbitai pause

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 19, 2026

✅ Actions performed

Reviews paused.

@ccamel
Copy link
Member Author

ccamel commented Mar 19, 2026

@coderabbitai resolve

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 19, 2026

✅ Actions performed

Comments resolved. Auto-approval is disabled; enable reviews.request_changes_workflow to approve automatically.

@ccamel ccamel merged commit b26df74 into main Mar 19, 2026
18 checks passed
@ccamel ccamel deleted the ci/centralize-external-docs-sync branch March 19, 2026 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants