v0.6.0

Hardening from the second live smoke test: a heavy-blob heuristic that now sees stray directories anywhere in the install, a default-empty privacy gate for CRM/mass-mailer subscriber tables, and a deterministic wp-config.php marked-block writer — plus a batch of contract and documentation fixes across the transfer engine.

Added

• A new deterministic helper, scripts/wpconfig_block.py, now owns the wp-config.php marked-block write that clone/pull §9.4 previously prescribed as hand surgery. It takes the current config text, the resolved portable defines, the table prefix, and the cron decision, and returns the new full text with the skills' marked block written (delimited by // BEGIN kntnt-wp-skills / // END kntnt-wp-skills, replaced in place when present or inserted above the /* That's all, stop editing! line when absent) and every scaffold collision it supersedes removed. The collision set is computed — the portable defines plus DISABLE_WP_CRON intersected with whatever the scaffold actually shipped — never a hard-coded name list: the second live smoke test's scaffold carried five colliding defines (DISABLE_WP_CRON, EMPTY_TRASH_DAYS, WP_DEBUG, WP_DEBUG_LOG, WP_DEBUG_DISPLAY) where the SKILL prose named two, and a repeated define() on the same constant fatals. Both clone/pull §9.4 now invoke the helper and lint the result with ddev exec php -l wp-config.php instead of describing the edit by hand. The helper fails loud at the boundary on the malformed-input classes that would emit a duplicate define() on one constant — a runtime fatal php -l cannot catch: a name repeated within the input defines, DISABLE_WP_CRON smuggled into the defines (its write is solely the cron field's job), a define name outside the PHP constant-identifier shape (closing a PHP-injection path through a quote-bearing name from production's config), and a non-finite float value (NaN/Infinity, which json.loads accepts and would render as a bare undefined-constant fetch). (#42).
• A recognised CRM/mass-mailer's subscriber tables now get their own carry/empty privacy gate, default empty — a sibling to the user_submissions gate and a sixth classification family. scripts/classify.py tags FluentCRM, MailPoet, The Newsletter Plugin, and Mailster subscriber/address stores (fc_subscribers and its family, fc_campaign_emails, fc_campaign_url_metrics, fc_funnel_subscribers, fc_funnel_metrics, mailpoet_subscriber*, mailster_subscriber*, mailster_queue, The Newsletter Plugin's exact-matched newsletter table plus newsletter_sent*/newsletter_stats*/newsletter_user*) as crm_subscribers, while the engine's definitions (campaigns, funnels, lists, tags, terms, meta, url stores, and Newsletter's newsletter_emails campaign bodies) carry in full as site config. Every engine in bootstrap_parse.py's MAILER_ENGINES recognition registry (FluentCRM / MailPoet / The Newsletter Plugin) now has a gated subscriber store — pinned by a drift test — so a recognised mailer's addresses can never carry in full; Mailster is gated in addition, making the gated set a superset of the recognised engines. scripts/resolve_plan.py adds the crm_subscribers decision and folds each privacy gate's resolved carry independently into db_table_content, so one gate's carry never disturbs the other's tables. Closes the gap the second live smoke test exposed, where 584 real FluentCRM subscribers transferred in full and standing funnel automations could mail them from a dev copy under the mail=live + cron-runs defaults — a risk the mass-send valve does not catch (ADR-0019, #39).

Changed

• The Application Password's home is now a documented convention instead of living only in the operator's head: macOS Keychain, service kntnt-extractor-app-password, account <wp-user>@<host>, retrieved with security find-generic-password -s ... -a ... -w inside a subshell, with the KNTNT_EXTRACTOR_APP_PASSWORD environment variable as the portable non-macOS fallback. The health check now resolves and verifies the credential itself (§1) and fails with the exact remediation ("create a Keychain item: service ..., account ...") when it is absent, instead of presuming "the configured" password exists. The three subagent task envelopes (discovery-classify, extract-transfer, manifest-baseline-diff) now take a credential reference — a Keychain service+account or an env-var name — resolved by the calling code inside each authenticated call's own subshell, so the secret's value never transits the orchestrating agent's own context (#44).

Fixed

• The heavy-blob heuristic is no longer uploads-scoped: a heavy stray directory anywhere else — the 7.6 GB install-root 2026/ directory the second live smoke test found, which the heavy_blobs=exclude default could not reach and would transfer silently — is now flagged too. scripts/discovery.py adds a root and a content breakdown beside the existing uploads one, each summing its top-level subdirectories (loose files are not directories and do not count), and scripts/classify.py runs a deliberately simpler floor-only rule over them: any non-standard directory at or above the 1 GiB floor is flagged, no median-outlier test, since at these levels the peer population is small and dominated by standard directories. The standard sets are derived from the document's own content and uploads locations, so a non-default content dir or uploads dir is honoured and its payload never flagged; when the content path is absent the default wp-content segment is assumed rather than exposing the payload tree. All flags merge into the single blobs.flagged list, so the existing heavy_blobs gate and scripts/build_exclusions.py keep the newly flagged directory out of the transfer by default and give the operator a lever over it, with no gate or exclusion-assembler change. A pre-existing document without the new sections yields exactly today's uploads flags (#38).

• scripts/unseal.py's seal mode (the development/test round-trip aid) still reached into each segment with bare segment["name"] / segment["data"] indexing, so a malformed segment raised an uncaught KeyError traceback instead of the clean unseal.py: missing required config key: 'name' diagnostic the rest of the helper emits — a residue of #47, which deliberately narrowed main()'s except to stop masking stray KeyErrors but left these two accesses unguarded. Both per-segment reads now go through the same _required accessor, so a missing key fails loud and uniform like every other required-key check. seal is off the clone/pull hot path — the plugin seals in production — so the impact was confined to the test aid (#47).

• Clone §3 accepted the plan at the end of its gate walk but wrote it out immediately, before §4 scaffolds the site directory <directory_name>/ that plan belongs in — so a literal top-to-bottom implementation wrote the bare .kntnt-wp-skills.json one level up, in the operator's invocation cwd. That mismatch guaranteed two false smoke FAILs (check_saved_plan_present, check_baseline_present, both anchored at the site directory) and left a future pull unable to find its plan or baseline. §3 now defers the write; §4's final bullet, after the restart onto the corrected engine, persists the accepted plan into <directory_name>/.kntnt-wp-skills.json — pre-creating the directory in §3 was rejected, since mkwp (≥ 1.8.1) refuses to scaffold into a path that already exists. "The local project root" — the site directory <directory_name>/ both persistence artifacts live in — is now defined once, in CONTEXT.md, and referenced consistently from docs/spec.md and both skills (#40).

• The poll discipline no longer treats a single spurious 404 on GET /extractions/{id} as terminal. The second live smoke test of the Extractor cutover proved the bare 404-is-terminal rule wrong in the field: the status endpoint returned 404 twice mid-job while the job was alive and progressing, a server-side non-atomic job.json rewrite race (Kntnt/kntnt-extractor#20). A 404 is now terminal only when confirmed vanished: the first 404 is treated as a transport-class fault, retried under the existing 30 s / 60 s backoff, and cross-checked against GET /extractions; the job counts as vanished only when the re-poll also 404s and the id is absent from the listing — otherwise the blip is logged and retried, and polling continues within budget. ADR-0018, both clone/pull SKILLs, agents/extract-transfer.md, and agents/discovery-classify.md's compact reference now state the rule identically (#41, refines ADR-0018 / #34).

• Every documented keygen invocation was the bare uv run scripts/unseal.py keygen, but the helper always parses a JSON envelope from stdin and keygen requires a caller-chosen private_key_path (the private key must land outside the docroot), so following the docs verbatim crashed at the health-check preflight with a generic JSON-parse error. All eight call sites — three each in the clone and pull SKILLs, plus agents/discovery-classify.md and agents/extract-transfer.md — now pipe the real envelope echo '{"private_key_path": "..."}' | uv run scripts/unseal.py keygen, and keygen fed empty or malformed stdin now exits non-zero with a self-documenting unseal.py: keygen requires JSON on stdin: {"private_key_path": "..."} instead of a bare parse error. The stdin/stdout contract is now pinned and guarded by tests/test_unseal_contract_consistency.py. Found on the second live smoke test of the Extractor cutover (#43).

• Four documentation drifts surfaced by the second live smoke test, none behavioural but each misleading a literal reader: docs/implementation-notes.md's endpoint example used the non-existent /wp-json/kntnt-extractor/v2 (the 2 is the response-body api_version, not the REST namespace, which is /v1); its "saved plan — illustrative shape" section still showed the superseded nested camelCase structure instead of the flat snake_case shape scripts/resolve_plan.py actually emits; the clone/pull SKILLs described their companion docs as living "alongside this file" when they sit at the repo root; and agents/discovery-classify.md named the POST /extractions tables in prose but never gave the request-body shape. All four are corrected, and a new guard test binds the saved-plan example to resolve_plan.py so it cannot silently drift again (#45).

• scripts/discovery.py reported WordPress's empty-directory guard file wp-content/themes/index.php as a theme (the live smoke returned themes: ["index.php", "ollie"]), because _relative_children took the first path segment under a prefix unconditionally and never distinguished a file from a directory. The manifest holds files only, so directory-ness is now inferred — a segment counts as a child directory only when at least one manifest path continues past it with a further / — and both derive_themes and derive_uploads_subdirectories drop bare files at the prefix level, closing the same latent bug in the uploads derivation before the planned root-scope blob heuristic could inherit it (#46).

• scripts/unseal.py's unseal mode printed a raw quoted KeyError (e.g. unseal.py: 'sql_path') and exited 0 when a required config key was missing — a silent failure in the very step that produces the dump everything downstream imports, which an exit-code-checking orchestrator would read as success even though nothing was unsealed. A missing required key now raises a clear missing required config key: 'sql_path' diagnostic and exits non-zero, writing no .sql file; main()'s exception handling was narrowed so a stray KeyError is no longer masked as a user error. Found on the second live smoke test of the Extractor cutover (#47).

• scripts/build_selection.py required its files input as a flat list of path strings, but clone §5 feeds it scripts/filter_manifest.py's output — a {entries: [{path, size, mtime}], scope: {...}} object — so a literal implementation of the chained steps failed with files must be a list, got dict and forced an undocumented [.entries[].path] transform between them. build_selection.py now also accepts the {entries: [...]} form directly, extracting the paths itself, so the helper chain composes with no prose glue; the flat-list form still works (#48).

• scripts/wpconfig_block.py required its defines input as [{name, value}] objects, but the §9.4 pipeline feeds it the resolved wp_config_defines decision — which scripts/resolve_plan.py deliberately emits as names only (their values are re-fetched from live state every run), so a literal implementation piped a name list into the writer and aborted every clone/pull that ported a define with defines[0] must be an object, got str. The only join the SKILL prose offered would have had an agent hand-improvise the name→value merge on a fatal-sensitive file — and piping classifications.defines.portable wholesale to sidestep it would silently port the very defines the operator deselected at the wp_config_defines gate, defeating the gate. The writer now owns the join deterministically: defines takes the classifier's portable [{name, value}] records as the value source, an optional select takes the resolver's kept-name list as the gate selection, and the helper writes only the selected names each valued from its record — so a deselected define is never ported, a selected name absent from the records fails loud as a corrupt join, and an absent select writes every record (unchanged for a caller that already filtered). Both clone §9.4 and pull §9.4 now pass the two artifacts as they stand and forbid the hand-join. No test spanned the resolve_plan→wpconfig_block seam before; tests/test_wpconfig_block_resolve_seam.py now drives the real helpers end to end (#42).

• The discovery-classify phase left its unsealed bootstrap dump — 14 MB of cleartext user and subscriber rows — in the scratchpad after scripts/bootstrap_parse.py had consumed it, alongside the sealed container; harmless under an ephemeral /private/tmp scratchpad but unmanaged PII at rest once the operator points the scratchpad at a persistent directory. bootstrap_parse.py now deletes the unsealed dump, the sealed container, and the run's private key immediately after parsing — the local analogue of the Extractor's own POST /consume — with a best-effort sweep so one failed unlink never blocks the rest, and the subagent contract records the discipline. The main extraction's dump keeps its longer, import-scoped lifetime. Found on the second live smoke test of the Extractor cutover (#49).

Full changelog: https://github.com/Kntnt/kntnt-wp-skills/blob/v0.6.0/CHANGELOG.md