Skip to content

feat(sync): repo profile writer + Sync command#8

Merged
bit-incarnas merged 1 commit into
mainfrom
feature/repo-profile-writer
Apr 22, 2026
Merged

feat(sync): repo profile writer + Sync command#8
bit-incarnas merged 1 commit into
mainfrom
feature/repo-profile-writer

Conversation

@bit-incarnas
Copy link
Copy Markdown
Owner

Summary

First real sync path. Six commits in, we can now actually pull data out of GitHub into the vault.

  • src/vault/writer.ts: VaultWriter interface + ObsidianVaultWriter (defensive folder creation, atomic frontmatter via processFrontMatter) + InMemoryVaultWriter for tests.
  • src/sync/repo-profile-writer.ts: syncRepoProfile(owner, repo, options) — fetches repo metadata, tries README (tolerates 404), composes a contained vault path, writes body + frontmatter atomically.
  • New command: GitHub Data: Sync all repo profiles iterates the allowlist, writes a profile file per entry, persists lastSyncedAt, reports ok/failed totals.
  • Allowlist-enforced + name-validated before any API call (Security Invariant H3).
  • Path containment via joinInsideRoot — malformed owner/repo can't escape 02_AREAS/GitHub/Repos/ (Security Invariant C1).
  • README fenced as code block for v0.1 per Security Invariant L3 — defeats inline-HTML / Templater / Dataview execution from synced content while the full body sanitizer is being built.
  • Frontmatter via processFrontMatter — never raw string concat into YAML (Security Invariant C3).

Output shape (per repo)

File at 02_AREAS/GitHub/Repos/{owner}__{repo}/00_{repo}.md:

  • Frontmatter: type, repo, owner, name, description, language, topics, visibility, stars, forks, open_issues_plus_prs, default_branch, license, homepage, html_url, created, pushed, last_synced, schema_version, tags
  • Body: H1 + description quote + stats table + fenced README + NAV footer

Data egress update

Plugin runtime now has three possible outbound calls, all user-initiated:

  • GET /user — Test Connection button
  • GET /repos/{o}/{r} — Sync command
  • GET /repos/{o}/{r}/readme — Sync command

No background polls, no scheduled syncs, no auto-fetches. docs/data-egress.md updated.

Tests

129 passing across 7 suites (+8 from this slice). New: full happy path with README, missing README (404) tolerated, allowlist reject, malformed owner rejected before any API call, 403 mapping, re-sync overwrite, case-insensitive allowlist match, folder ensure ordering.

Out of scope

  • Issue/PR writers (next slice)
  • Persist blocks (issues/PRs, not repo profile)
  • Update modes (repo profile always overwrites)
  • Full body sanitizer for H1 (README currently fenced-as-code)
  • ETag conditional requests (sync is idempotent but not bandwidth-minimal yet)

🤖 Generated with Claude Code

First real sync path. Fetches per-repo metadata + README and writes a
vault-native markdown file for each allowlisted entry.

Architecture
- VaultWriter interface (src/vault/writer.ts) abstracts vault I/O the
  same way HttpFn abstracts HTTP. ObsidianVaultWriter is the default;
  InMemoryVaultWriter backs unit tests.
- syncRepoProfile(owner, repo, options) in src/sync/repo-profile-writer.ts
  validates name -> checks allowlist -> composes path via
  joinInsideRoot -> fetches metadata -> optional README -> ensures
  folders -> writes body -> sets frontmatter atomically via
  updateFrontmatter.
- New "GitHub Data: Sync all repo profiles" command iterates the
  allowlist, calls syncRepoProfile per entry, persists lastSyncedAt,
  and reports ok/failed totals via Notice.

Security layering (all enforced before any network call)
- H3: allowlist check fails closed even if owner/repo pass validation.
- C1: joinInsideRoot + sanitizePathSegment -- malformed inputs cannot
  escape 02_AREAS/GitHub/Repos. Containment check asserts final path
  stays inside the root.
- C3: frontmatter set via updateFrontmatter (-> processFrontMatter on
  the Obsidian side), never raw string concat into YAML.
- L3: README body fenced as a code block in v0.1. Defeats inline-HTML
  / Templater / Dataview execution against the synced content while
  the full middle-ground body sanitizer is being built.

Data egress
- Plugin runtime now has three possible outbound calls, all user-
  initiated: GET /user (Test connection), GET /repos/{o}/{r}, and
  GET /repos/{o}/{r}/readme. docs/data-egress.md updated. Still no
  background polls, scheduled syncs, or auto-fetches.

Tests (129 passing across 7 suites; +8 from this slice)
- src/sync/repo-profile-writer.test.ts covers: full-happy-path with
  README, missing README (404) tolerated, allowlist reject, malformed-
  owner reject before any API call, 403 mapping to structured error,
  re-sync overwrite, case-insensitive allowlist match, folder ensure.
- src/main.test.ts updated: both ping + sync commands registered.

Bundle size 121k (up from 113k -- sync writer + path utils + allowlist).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 22, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9e138fd2-681a-44b9-b546-fd3468e4caa0

📥 Commits

Reviewing files that changed from the base of the PR and between 8e92907 and aa4eb71.

📒 Files selected for processing (7)
  • CHANGELOG.md
  • docs/data-egress.md
  • src/main.test.ts
  • src/main.ts
  • src/sync/repo-profile-writer.test.ts
  • src/sync/repo-profile-writer.ts
  • src/vault/writer.ts

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Added command to sync all allowlisted repository profiles with GitHub data, including statistics and README content.
    • Automatically updates last synced timestamp in settings for each successful sync.
    • Displays progress and result notifications with success/failure counts.
  • Documentation

    • Updated data egress documentation to reflect new GitHub API endpoints for profile syncing.

Walkthrough

This PR introduces a new repo profile synchronization feature with vault-writing abstractions. It adds VaultWriter interface with ObsidianVaultWriter and InMemoryVaultWriter implementations, a syncRepoProfile function that validates and syncs individual repos to vault, and a syncAllRepoProfiles command that batches syncs across allowlisted repos with progress/result reporting.

Changes

Cohort / File(s) Summary
Documentation
CHANGELOG.md, docs/data-egress.md
Updated changelog with new vault-writing abstractions and sync features. Expanded data-egress documentation to detail GitHub API calls for repo profile sync (GET /repos/{owner}/{repo} and GET /repos/{owner}/{repo}/readme) and clarify all calls are user-initiated with no background polling.
Vault Abstraction
src/vault/writer.ts
Added VaultWriter interface defining folder/file/frontmatter operations. Implemented ObsidianVaultWriter using Obsidian APIs and InMemoryVaultWriter for testing with in-memory Map and Set backing stores.
Repo Profile Sync
src/sync/repo-profile-writer.ts, src/sync/repo-profile-writer.test.ts
Implemented syncRepoProfile function that enforces allowlist validation, fetches repo metadata and README from GitHub, composes markdown with stats table and fenced README, ensures vault folders exist, and updates frontmatter atomically. Added comprehensive test suite validating success/failure cases, allowlist enforcement, case-insensitive matching, and 404-tolerant README fetching.
Plugin Integration
src/main.ts, src/main.test.ts
Added syncAllRepoProfiles() command that iterates allowlisted repos, parses repo paths, skips invalid entries, calls syncRepoProfile for each, updates settings.lastSyncedAt, and reports aggregated results via Notice. Updated test to validate both "ping" and "sync-repo-profiles" command registration.

Sequence Diagram

sequenceDiagram
    actor User
    participant Plugin as GithubDataPlugin
    participant RepoSync as syncRepoProfile
    participant GitHub as GitHub API
    participant VaultWriter as VaultWriter
    participant Settings as Settings

    User->>Plugin: Trigger "Sync all repo profiles" command
    Plugin->>Settings: Get repoAllowlist & token
    Plugin->>Plugin: Validate token & allowlist
    
    loop For each allowlisted repo
        Plugin->>RepoSync: syncRepoProfile(owner, repo)
        RepoSync->>RepoSync: Validate allowlist membership
        RepoSync->>RepoSync: Validate owner/repo names
        RepoSync->>GitHub: GET /repos/{owner}/{repo}
        GitHub-->>RepoSync: Repo metadata
        RepoSync->>GitHub: GET /repos/{owner}/{repo}/readme
        alt README exists
            GitHub-->>RepoSync: README content
        else README missing
            GitHub-->>RepoSync: 404
            RepoSync->>RepoSync: Continue with empty README
        end
        RepoSync->>VaultWriter: ensureFolder(path)
        VaultWriter-->>RepoSync: Folder created
        RepoSync->>VaultWriter: writeFile(path, markdown)
        VaultWriter-->>RepoSync: File written
        RepoSync->>VaultWriter: updateFrontmatter(path, mutate)
        VaultWriter-->>RepoSync: Frontmatter updated
        RepoSync-->>Plugin: SyncResult { ok, path, syncedAt }
        Plugin->>Settings: Update lastSyncedAt for repo
    end
    
    Plugin->>Settings: Persist updated settings
    Plugin->>User: Show aggregated success/failure Notice
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 A rabbit's delight in the code that's been stitched!

New vaults and profiles all perfectly hitched,
Repos sync smoothly across GitHub's great land,
Allowlists keep chaos at bay—oh so grand!
With frontmatter whispers and tables so neat,
This feature's completion makes the sprint complete! 🌟

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/repo-profile-writer

Comment @coderabbitai help to get the list of available commands and usage tips.

@bit-incarnas bit-incarnas enabled auto-merge (squash) April 22, 2026 21:34
@bit-incarnas bit-incarnas merged commit b5042ec into main Apr 22, 2026
3 of 4 checks passed
@bit-incarnas bit-incarnas self-assigned this Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant