Skip to content

v1.10.0 — page assets sync (pw_page_assets, MediaHub-aware compare/sync)#2

Merged
cursor[bot] merged 7 commits into
mainfrom
cursor/page-assets-sync-2d5b
Apr 30, 2026
Merged

v1.10.0 — page assets sync (pw_page_assets, MediaHub-aware compare/sync)#2
cursor[bot] merged 7 commits into
mainfrom
cursor/page-assets-sync-2d5b

Conversation

@PeterKnightDigital

@PeterKnightDigital PeterKnightDigital commented Apr 30, 2026

Copy link
Copy Markdown
Owner

Summary

Closes the gap where PromptWire's site sync shipped page content but not the on-disk files attached to those pages.

The standard FieldtypeFile / FieldtypeImage widgets store uploads in site/assets/files/{pageId}/, and pw_file_sync already covered those when iterated through a page's fieldgroup. Two cases were missing:

  1. Module-managed files (notably MediaHub) also store files in site/assets/files/{pageId}/ keyed by page id, but expose them through their own admin UI rather than as a regular Pagefiles value on the page's fieldgroup. The field-aware pw_file_sync walked $page->template->fieldgroup, never saw those files, and silently shipped pages to production with broken media references.
  2. pw_site_sync's page-files step required a prior pw_page_pull — it only synced files when a local sync directory under site/assets/pw-mcp/ already existed. Pushing page content for a page the operator had never pulled left the assets behind on local. The remote→local direction was an explicit no-op ("Remote-to-local file sync is not yet implemented. Use SFTP to pull files.").

v1.10.0 fixes both by treating the page-asset directory as a first-class sync surface, walked directly from disk rather than via PW field iteration.

What's in this PR

New PHP commands (CommandRouter + remote API)

  • page-assets:inventory <pageId|/path/> — lists site/assets/files/{pageId}/ recursively as [{relativePath, size, md5, modified}]. Some modules nest one level deep (MediaHub uses variants/), so the relative path is the stable cross-environment identity used for diffing.
  • page-assets:inventory --all-pages [--exclude-templates=user,role] — site-wide variant, keyed by canonical PW path. One round-trip feeds the full site comparison.
  • page-assets:download <page> --filename=NAME — returns one asset as base64. realpath() sandbox prevents directory escape.
  • page-assets:upload (API endpoint) — writes a base64 payload directly into site/assets/files/{pageId}/, no field requirement. Same sandbox.
  • page-assets:delete (API endpoint) — removes the file plus any PW image variations sharing its basename, so a deleted JPEG doesn't leave its 480x270 cache behind.

PW image variations (name.WIDTHxHEIGHT[-suffix].ext) are filtered by default for inventory/sync because they're regenerated on demand and would otherwise produce noisy diffs purely from cache state drift between environments. --include-variations opts back in.

New MCP tool

  • pw_page_assets { action, pageRef, site?, dryRun?, deleteOrphans?, includeVariations? } — actions: inventory, compare (per-page or site-wide if pageRef omitted), push (local→remote), pull (remote→local). Dry-run by default for push/pull; produces a per-file plan before the operator confirms.

Changes to existing tools

  • pw_site_compare gains a pageAssets section alongside pages/schema/files. Per-page summary of changed / localOnly / remoteOnly. The compare attempts the assets diff in parallel with the existing inventory calls so latency doesn't stack. When the remote site is on an older PromptWire that doesn't ship page-assets:inventory, the section carries a warning field instead of failing the whole compare — backward-compatible with mixed-version deployments. Opt out via includePageAssets:false.
  • pw_site_sync swaps its page-files step for page-assets. Source of truth for "which pages have asset drift" is the comparison's pageAssets.diffs instead of "does the page have a local sync directory under site/assets/pw-mcp/". Now picks up MediaHub-style files, acts on every page with on-disk drift (not just pulled ones), and works in both directions. Orphan deletion is intentionally OFF for the orchestrated sync — accidental orphan deletion in a bulk operation is hard to undo, so use pw_page_assets directly with deleteOrphans:true for that.

page.meta.json format addition: pageAssets snapshot

Both pull paths now embed a baseline of the source side's view of site/assets/files/{pageId}/ directly in page.meta.json:

{
  "pageId": 1234,
  "canonicalPath": "/about/",
  "pageAssets": {
    "pageId": 1234,
    "capturedAt": "2026-04-30T15:00:00+00:00",
    "directoryExists": true,
    "assetCount": 7,
    "totalBytes": 4827341,
    "directoryHash": "md5:...",
    "assets": [
      { "relativePath": "hero.jpg", "size": 412330, "md5": "..." },
      { "relativePath": "variants/hero-mobile.jpg", "size": 87214, "md5": "..." }
    ]
  }
}

Why store this in the meta:

  • Answers "what has drifted on this page's media since I pulled?" with a local md5 walk against the snapshot — no remote round-trip required.
  • Meta files exchanged between environments (page:export-yaml writes one into the local sync tree on a remote pull) carry the source side's view of media, including module-managed files like MediaHub that the page's fieldgroup never exposes. The receiver knows about those assets without a separate page-assets:inventory call.
  • pw_page_assets push/pull results (dry-run and live) gain a driftSinceLastPull block listing adds / removes / modifications since pull, plus a snapshotSide: "local" | "remote" | "unknown" field that tells you which environment the comparison is against (matched by the snapshot's own pageId).

Strictly additive: older meta files without the snapshot fall through to the existing local-vs-remote diff. No migration — re-pulling refreshes any meta with the new block.

page.yaml and page.json format addition: self-describing header

Pulled page.yaml now opens with a comment block giving the operator immediate context — page identity, this site's pageId, and a one-line asset summary built from the same pageAssets snapshot embedded in page.meta.json:

# ============================================================================
# Page:     About Us  ·  /about/
# Template: basic-page  ·  Status: published
# Pulled:   2026-04-30T15:00:00+00:00 via page:pull (pageId on this site: 1234)
#
# Page assets (snapshot at pull time):
#   3 files · 1.2 MB
#   On disk (this site): site/assets/files/1234/
#   Full inventory in page.meta.json under `pageAssets` (sync with pw_page_assets).
#
# This file is hand-editable. page.meta.json is auto-generated — do not edit it.
# ============================================================================

fields:
  ...

page.json gets the equivalent as a top-level _meta object (since JSON has no comment syntax):

{
  "_meta": {
    "page":       { "id": 1234, "path": "/about/", "title": "About Us", "template": "basic-page", "status": "published" },
    "pulledAt":   "2026-04-30T15:00:00+00:00",
    "source":     "page:pull",
    "pageAssets": { "count": 3, "totalBytes": 1248320, "directoryHash": "md5:...", "note": "Full inventory in page.meta.json under `pageAssets`." }
  },
  "fields": { ... }
}

Why both forms work:

  • YAML. Comments are the natural way to add context that parsers ignore. Verified end-to-end: Symfony YAML, the in-tree simple parser, AND js-yaml on the MCP side all strip # comments cleanly — including the unicode bullet · and embedded backticks. Tested with a fixture covering _pageRef blocks and nested arrays; every field value comes through untouched.
  • JSON. Has no comment syntax, so the identity block is a sibling _meta key. The existing pushPage path reads content['fields'] only and the strict-shape check is on that key alone, so the new _meta key is strictly additive — no push or read code needs to change to ignore it.

Single source of truth: the header summary, the _meta.pageAssets block, and meta.pageAssets are all built from the same buildPageAssetSnapshot() call. No risk of the three presentations disagreeing.

The push-back code path also refreshes BOTH the header AND meta.pageAssets after a successful save, so subsequent driftSinceLastPull checks compare against the post-push state rather than a stale pre-push baseline.

Page-id drift between environments — handled and surfaced

Two ProcessWire sites started from independent fresh installs (rather than from a database clone) legitimately have different auto-increment sequences. The same canonical PW path /about/ can resolve to local id 1234 and remote id 5678 — and that's normal, not a bug.

The path-based matching that pw_page_assets and the pageAssets section of pw_site_compare already do handles this correctly under the hood. But the operator had no way to see which physical disk directory got read or written on each side. That visibility gap matters because the same failure mode can also indicate something is genuinely wrong — e.g. local /about/ resolves to a basic-page but remote /about/ is a redirect template — and it's hard to diagnose without the per-side ids in front of you.

Three places now surface this explicitly:

  • syncPageAssets (per-page push/pull) result payload includes localPageId, remotePageId, and an idDrift boolean.
  • pw_page_assets action="compare" (per-page diff) includes the same trio.
  • compareSiteAssets (site-wide, feeds pw_site_compare's pageAssets section) includes localPageId / remotePageId / idDrift on every per-page entry, plus a top-level pagesWithIdDrift counter — an early warning that the two sites are diverging in ways that affect more than just assets (cross-site Page references, hardcoded page ids in template code).

Numeric pageRef arguments are local-first by design: passing 1234 to pw_page_assets resolves the page on local and translates to the remote via canonical path. A numeric id never accidentally addresses the wrong remote page when local/remote auto-increment sequences have diverged. If the operator only knows a remote id, the error message tells them to look up its path with pw_get_page first.

Follow-up planned for v1.10.1 (NOT in this PR)

page.meta.json still has a single top-level pageId field — whatever side you last pulled from, that side's id wins. The follow-up adds an ids: { local?, remote? } block so each pull populates its own slot without overwriting the other, and refactors pushPage to resolve by canonical path with the per-environment id used purely as a sanity check. Documented in ROADMAP.md under the v1.10.0 section. Deferred from this PR because it touches pushPage (the most-used write tool in the suite) and deserves its own focused PR.

Safety

  • Path sandboxing. Every page-assets:* write/read resolves the target via realpath() and refuses anything that escapes the page directory.
  • Cross-environment matching. Pages are matched by canonical PW path; only the pageId on the side being read/written is used to resolve the on-disk directory, so local and remote IDs may differ freely. ID drift is now reported in every result.
  • Serial transfers. Sites with heavy media (a single page can hold dozens of MB of PDFs and images) would multiply memory by the worker count for no real benefit if parallelised.
  • Per-file failure reporting. Transfer failures are reported per file, not as a single binary success/failure, so operators can retry on the smaller set.

Files changed

  • src/Cli/CommandRouter.php — new commands page-assets:inventory, page-assets:download, plus version bump and help registration. resolvePageRef docblock spells out that it only ever runs against the site it's on.
  • src/Sync/SyncManager.php — new buildPageAssetSnapshot and buildYamlHeaderComment helpers; pullPage, exportPageYaml, and the push-back rewrite path all embed the snapshot under meta.pageAssets, render the YAML header / JSON _meta block from the same snapshot, and refresh both after a successful push.
  • api/promptwire-api.php — new endpoints page-assets:upload, page-assets:delete. Inline notes that production callers always pass canonical PW paths (numeric-id branch is for ad-hoc debugging only).
  • mcp-server/src/sync/page-assets.ts — new TS module: local inventory walker, remote inventory client, diffAssets, syncPageAssets, compareSiteAssets, snapshot loader, diffAgainstSnapshot. Surfaces localPageId / remotePageId / idDrift and driftSinceLastPull on every result.
  • mcp-server/src/sync/site-compare.tspageAssets section integration, including pagesWithIdDrift counter.
  • mcp-server/src/sync/site-sync.tspage-files step replaced with page-assets; remote→local now supported.
  • mcp-server/src/index.ts — new pw_page_assets tool definition + handler (compare action surfaces idDrift); pw_site_compare gains includePageAssets arg; tool descriptions updated to call out the cross-environment id-drift behaviour.
  • mcp-server/src/remote/client.ts — 120s timeout for page-assets:* (matching file:*).
  • PromptWire.module.php, mcp-server/package.json — version bump 1.9.3 → 1.10.0.
  • README.md, CHANGELOG.md, ROADMAP.md — docs updates and release entry (incl. v1.10.1 follow-up).

Migration impact

  • Zero for callers. pw_page_assets is new; pw_site_compare gains an additive pageAssets field; pw_site_sync swaps its page-files step for page-assets but keeps the same overall step ordering and the same "non-fatal: page content already pushed" failure semantics. The drift-related fields (localPageId, remotePageId, idDrift, pagesWithIdDrift, driftSinceLastPull), the new pageAssets snapshot in page.meta.json, the YAML header comment, and the JSON _meta block are all additive — nothing pre-existing changes shape.
  • Mixed-version deployments work. The remote API needs the v1.10.0 endpoint deployed for the new commands; older endpoints surface as a pageAssets.warning rather than breaking the rest of the compare/sync.
  • Older page.meta.json files without the snapshot keep working — the driftSinceLastPull block is only emitted when a snapshot is present. Re-pulling refreshes any meta with the new block.
  • Older page.yaml files without the header keep working — the header is added on the next pull or push-back rewrite. Push-side YAML readers (Symfony YAML, the in-tree simple parser, js-yaml on the MCP side) all strip comments, so a YAML-with-header pushed against an older PromptWire endpoint still parses cleanly.

Validation

  • npm run build passes (TypeScript strict).
  • php -l clean on every modified PHP file.
  • YAML header parses cleanly through both PHP parsers (Symfony + in-tree simple parser) and js-yaml on the MCP side. Tested with a fixture that includes _pageRef blocks, nested arrays, unicode (·) and backticks — all field values come through unchanged.
  • Cross-environment behaviour was verified by reading the existing file-sync.ts / siteInventory paths to confirm the new commands follow the same conventions (response shape, parseFlags flag naming, JSON output format, sandboxing pattern).
Open in Web Open in Cursor 

cursoragent and others added 7 commits April 30, 2026 15:22
Adds the four PHP primitives for syncing on-disk page assets directly
from site/assets/files/{pageId}/, distinct from the existing field-aware
file:* commands which iterate $page->template->fieldgroup.

Why a directory walker instead of field iteration:

- Standard FieldtypeFile / FieldtypeImage uploads still land in
  site/assets/files/{pageId}/ — already covered today, but the existing
  pw_file_sync only sees them via the page's fieldgroup.
- Module-managed files (notably MediaHub) store files in the same
  per-page directory but never expose them via fieldgroup iteration, so
  field-aware sync silently misses them. Walking the directory catches
  both.

PW image variations (name.WIDTHxHEIGHT[-suffix].ext) are filtered by
default because they are regenerated on demand and would otherwise
produce noisy diffs purely from cache state drift between environments.
--include-variations opts back in.

Site-wide --all-pages mode on inventory feeds pw_site_compare with the
full per-page diff in one round-trip, instead of one HTTP call per page.

upload/delete are in the API endpoint (not CommandRouter) because they
take base64 payloads handled the same way as the existing file:upload /
file:delete handlers. Both sandbox the target via realpath() so a
malformed filename cannot escape the page directory; delete also removes
PW image variations sharing the original's basename so a deleted JPEG
does not leave its 480x270 cache behind.

Co-authored-by: PKDigital <PeterKnightDigital@users.noreply.github.com>
New tool: pw_page_assets { action, pageRef, site?, dryRun?,
deleteOrphans?, includeVariations? }. Actions:

- inventory: list assets on the chosen side
- compare:   diff both sides (per-page, or site-wide if no pageRef)
- push:      sync local -> remote
- pull:      sync remote -> local

Closes the remote-to-local gap that the v1.7 pages/file-sync explicitly
left open with 'use SFTP'. Pull works by calling page-assets:download
on the remote (returns base64), writing to the local page-asset dir,
and preserving mtime via utimes() so subsequent compares stay clean.

Pages are matched by canonical PW path so local and remote pageIds may
differ — only the pageId on the side being read or written is used to
resolve the on-disk directory. Same cross-environment safety the rest
of PromptWire already gives page content.

Transfers are serial. Sites with heavy media (MediaHub directories
with high-res originals) can hold dozens of MB per page; parallelising
would multiply memory by the worker count for no real benefit. Failures
are reported per file rather than as a single binary success/failure
so the operator can retry on the smaller set.

Remote client timeout extended to 120s for page-assets:* commands
(matching the existing file:* handling), since bulk uploads can run
past the default 60s.

Co-authored-by: PKDigital <PeterKnightDigital@users.noreply.github.com>
pw_site_compare: gains a pageAssets section alongside pages/schema/
files. Per-page summary of changed / localOnly / remoteOnly files.
The compare attempts the assets diff in parallel with the existing
inventory calls so latency does not stack. When the remote site is on
an older PromptWire that does not ship page-assets:inventory, the
section carries a 'warning' field instead of failing the whole compare
— backward-compatible with mixed-version deployments. Opt out via
includePageAssets:false.

pw_site_sync: replaces the old page-files step with a directory-walking
variant driven by the comparison's pageAssets.diffs. Three behavioural
changes:

  1. Picks up MediaHub-style files and any other module-managed assets
     that the previous fieldgroup-iteration step silently missed.
  2. Acts on every page that has on-disk asset drift, not just pages
     that happen to have a local sync directory under
     site/assets/pw-mcp/. Pushing page CONTENT for a page that was
     never pulled used to leave its assets behind on local.
  3. Works in both directions. The remote-to-local 'use SFTP' message
     is gone — that direction now pulls each missing/changed file via
     page-assets:download.

Orphan deletion is intentionally OFF for the orchestrated sync. Use
pw_page_assets directly with deleteOrphans:true for that — accidental
orphan deletion in a bulk operation is hard to undo.

Co-authored-by: PKDigital <PeterKnightDigital@users.noreply.github.com>
- PromptWire.module.php and mcp-server/package.json bumped from
  1.9.3 to 1.10.0 so the version Cursor reports matches the new
  page-assets feature set.
- README tools table gains pw_page_assets row and total tool count
  goes from 40 to 41.
- CHANGELOG entry summarising the new MCP tool, new PHP commands,
  and the additive changes to pw_site_compare / pw_site_sync.
- ROADMAP entry documenting the gap that v1.10.0 closes (MediaHub
  files in site/assets/files/{pageId}/ being silently missed by the
  field-aware sync, plus the remote->local 'use SFTP' no-op that
  the previous file-sync explicitly refused to handle).
- package-lock.json had a stale 2.0.0 version — npm install
  corrected it to track package.json's 1.10.0.

Co-authored-by: PKDigital <PeterKnightDigital@users.noreply.github.com>
Two sites started from independent fresh installs (rather than a DB
clone) legitimately have different auto-increment sequences. The same
canonical PW path (/about/) can resolve to local id 1234 and remote id
5678 — and that's normal, not a bug.

The path-based matching that pw_page_assets and the page-assets section
of pw_site_compare were already doing handled this correctly under the
hood, but the operator had no way to SEE which physical disk directory
got read or written on each side. That visibility gap matters because
the same failure mode can also indicate something is genuinely wrong
(e.g. local /about/ resolves to a basic-page but remote /about/ is a
redirect template — different pages with the same path), and it's hard
to diagnose without the per-side ids in front of you.

Three places now surface this:

  - syncPageAssets (per-page push/pull) result payload includes
    localPageId, remotePageId, and an idDrift boolean.
  - pw_page_assets action='compare' (per-page diff handler in index.ts)
    includes the same trio.
  - compareSiteAssets (site-wide compare, fed into pw_site_compare's
    pageAssets section) includes localPageId / remotePageId / idDrift
    on every per-page entry, plus a top-level pagesWithIdDrift counter.

Numeric pageRef arguments are LOCAL-first by design — passing 1234 to
pw_page_assets resolves on local and translates to the remote via path,
so a numeric id never accidentally addresses the wrong remote page.
Tool description and error message updated to make this explicit.

PHP-side resolvePageRef gets a beefier docblock spelling out that it
only ever runs against the site it's on; the cross-environment
translation is the MCP server's responsibility.

API endpoint upload/delete handlers get inline notes that production
callers always pass canonical PW paths (so the remote resolves on its
own auto-increment sequence), and the numeric-id branch is for ad-hoc
debugging only.

Migration impact: zero. The new fields are additive on every result
shape; behaviour is unchanged.

Co-authored-by: PKDigital <PeterKnightDigital@users.noreply.github.com>
Adds a pageAssets block to page.meta.json that captures the source
side's view of site/assets/files/{pageId}/ at the moment of pull.
Shape:

  pageAssets: {
    pageId, capturedAt, directoryExists,
    assetCount, totalBytes,
    directoryHash: 'md5:...',
    assets: [{relativePath, size, md5}, ...]
  }

Why store this in the meta:

  - Answers 'what has drifted on this page's media since I pulled?'
    with a local md5 walk against the snapshot, no remote round-trip.
  - Meta files exchanged between environments (page:export-yaml writes
    one into the local sync tree on a remote pull) carry the source
    side's view of media — including module-managed files like
    MediaHub that the page's fieldgroup never exposes. The receiver
    knows about those assets without a separate page-assets:inventory
    call.
  - Snapshot uses the same directory-walking technique introduced for
    page-assets:inventory, so it picks up MediaHub and any other
    module-managed assets the same way the rest of v1.10.0 does. PW
    image variations are filtered (regenerated on demand, would only
    add noise).

Wired into both pull paths:
  - SyncManager::pullPage (local pull → page.meta.json on disk).
  - SyncManager::exportPageYaml (remote pull → inline meta payload).

Wired into syncPageAssets on the MCP side:
  - loadPulledSnapshot reads meta.pageAssets from
    site/assets/pw-mcp/<canonicalPath>/page.meta.json.
  - diffAgainstSnapshot compares the snapshot to the live inventory
    on whichever side it represents (matched by snapshot.pageId).
  - Result payload (dry-run AND live) gains a driftSinceLastPull block
    with snapshotSide:'local'|'remote'|'unknown' so the operator
    knows which env the comparison is against.

Strictly additive — older meta files without the snapshot fall through
to the existing local-vs-remote diff. No migration; re-pulling refreshes
any meta with the new block. ROADMAP also documents the planned v1.10.1
follow-up: per-environment ids in page.meta.json so meta.pageId stops
silently switching between local/remote on re-pulls. That's a real
refactor of pushPage's critical write path and gets its own PR.

Co-authored-by: PKDigital <PeterKnightDigital@users.noreply.github.com>
Page sync files now carry a human-readable identity block at the top:

page.yaml gets a comment header ABOVE the existing 'fields:' block:

  # ============================================================================
  # Page:     About Us  ·  /about/
  # Template: basic-page  ·  Status: published
  # Pulled:   2026-04-30T15:00:00+00:00 via page:pull (pageId on this site: 1234)
  #
  # Page assets (snapshot at pull time):
  #   3 files · 1.2 MB
  #   On disk (this site): site/assets/files/1234/
  #   Full inventory in page.meta.json under `pageAssets` (sync with pw_page_assets).
  #
  # This file is hand-editable. page.meta.json is auto-generated — do not edit it.
  # ============================================================================

  fields:
    ...

page.json gets the equivalent as a top-level _meta object:

  {
    "_meta": {
      "page":       { id, path, title, template, status },
      "pulledAt":   "...",
      "source":     "page:pull",
      "pageAssets": { count, totalBytes, directoryHash, note }
    },
    "fields": { ... }
  }

Why both forms:

  - YAML comments are the natural way to add context that a parser
    will ignore (verified: Symfony YAML, the in-tree simple parser,
    and js-yaml on the MCP side all strip # comments cleanly,
    including the unicode bullet · and embedded backticks). Tested
    end-to-end with a fixture that includes _pageRef blocks and
    nested arrays — all field values come through untouched.
  - JSON has no comment syntax, so the equivalent identity block
    is a top-level _meta object. The existing pushPage path reads
    content['fields'] only and passed a strict-shape check on that
    key alone, so a sibling _meta key is strictly additive — no
    push or read code needs to change to ignore it.

Single source of truth: the asset summary at the top of page.yaml
and inside _meta is built from the SAME pageAssets snapshot that
gets embedded in page.meta.json. No risk of the two presentations
disagreeing.

Wired into all three write sites:
  - pullPage (local pull → on-disk page.yaml/page.json + meta)
  - exportPageYaml (remote pull payload → identical structure)
  - pushPage rewrite-after-save (so the file the operator next
    opens after a successful push reflects the current page state,
    not the pre-push baseline)

The push-back path also refreshes meta.pageAssets so
driftSinceLastPull on the next sync compares against the new
post-push state rather than the stale pre-push snapshot.

Co-authored-by: PKDigital <PeterKnightDigital@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants