Skip to content

v0.6.0

Choose a tag to compare

@TBarregren TBarregren released this 16 Aug 20:03
· 90 commits to main since this release

The largest release so far, and a coordinated one: api_version moves from 6 to 7, so this build must not be pointed at by a kntnt-wp-skills older than 0.10.0. Install the plugin first, then the client, and never while an extraction is in flight — GET /extractions must list nothing before you upgrade.

Four things dominate it. A large extraction spent 96 % of its per-chunk time rewriting its own job record and got slower the further it went; the record is now split on what is unbounded, and per-chunk cost no longer varies with how much was selected. A stall no longer fails a job while any bound it spends can still shrink — it halves, persists, and continues. GET /environment stopped returning every non-core wp-config define in plaintext, which is how a third-party API key left a client's server; values now disclose from an allow-list, with a per-record disclosure discriminator so a caller can never confuse a withheld value with a genuinely null one. And a failed or abandoned run now cleans up after itself instead of accumulating one working directory per failure until uninstall.

Two smaller additions are worth knowing about because they are how you discover the rest: an authenticated GET /status now carries honours, naming the caller-visible behaviours this build implements, and GET /extractions/{id} carries attempts, the last eight packaging chunks a job has begun — which is what answers "what was attempted" on a long or stuck run.

Known limits, stated rather than glossed: the orphan-artifact walk is bounded per cycle and restarts from the beginning, so on a large downloads directory an entry beyond the bound is not guaranteed to be reached in any one cycle. The sealed container's framing still carries no MAC of its own — each segment is individually authenticated, and docs/container-format.md §7 states what a reader must therefore do. Over half of the production per-chunk cost this release did not explain remains unattributed and needs a real host to settle.

Fixed

  • POST /extractions/{id}/consume and DELETE /extractions/{id} (cancel) deleted a job's artifact and working directory without taking the per-job tick lock every other purging actor (the internal tick driver, the TTL sweep) already took, even though Job_Store::purge()'s own docblock named consume, cancel, and the sweep as its three callers. A cancel landing mid-tick could delete the directory a live build was still writing into, and if the driver's artifact-publish rename won that race, a sealed artifact landed in the served downloads directory for a job whose record had just been deleted — reachable by no id and invisible to the sweep, which walks job records and had none left for it (ADR-0019). Both routes now take the same lock, with the same try/finally discipline the sweep already uses; a lock a live tick is holding answers 409 kntnt_extractor_locked rather than a silent skip or a blocking wait, and the caller simply retries. api_version stays 7.
  • Nothing ever reclaimed a sealed artifact left behind with no job record at all — the shape the lock fix above closes going forward, but not what an unfixed build, or a crash between an artifact's publish and its own record settling, may already have left on disk. Job_Store::orphaned_artifacts() enumerates the served downloads directory for exactly that residue, pinned the same way every other destructive path in the store is pinned (a null-byte guard, the artifact's own filename shape, resolved strictly inside the downloads directory, never a followed symlink), and bounded per call so a directory with many files cannot make one sweep cycle's disk work unbounded. Sweeper::sweep() now reclaims what it finds, once an artifact has aged past its own grace period — the resolved TTL, reused rather than an invented constant, because it is already the window a never-consumed but still-recorded artifact is judged by. An artifact inside that window is left untouched, which is the assertion that matters most: it is what stops this fix from eating a live job's output. api_version stays 7.
  • GET /environment disclosed a wp-config define's value from a deny-list of six shapes (four exact core-secret names, any *_SALT suffix, any NONCE_* prefix), so every third-party credential a site happened to define — an SMTP password, a signing secret, an API key, a licence key, a secondary database credential — came back in plaintext to any caller who cleared the endpoint's capability gate. This is not theoretical: a real extraction run against a client site carried a 40-character third-party API key (KNTNT_PAPAPI_KEY) off the server this way, matched by none of the six deny-list shapes, and it sat in a local scratchpad for three days (ADR-0018). A define's value now discloses only from a curated allow-list of layout and behaviour facts (ABSPATH, WP_CONTENT_DIR, DB_NAME, WP_DEBUG, and similar) plus a heuristic backstop that withholds anything shaped like a credential (a name containing KEY, SECRET, TOKEN, PASS, SALT, NONCE, AUTH, CREDENTIAL, PRIVATE, LICEN, or API), applied after the allow-list and never overridden by it. Define names are unchanged and still always reported; only values are gated. Each defines record now also carries disclosure (included, secret, or not_allow_listed), present on every record including disclosed ones, so a caller can never confuse a value withheld by policy with a value that is genuinely null — see docs/define-disclosure.md. A site operator can opt a specific unlisted define in explicitly, per site, through the new KNTNT_EXTRACTOR_DISCLOSABLE_DEFINES constant or kntnt_extractor_config_disclosable_defines filter.
  • with_resume() forwarded $budget_keys_present from the stranded record, so a pre-adaptation resume saved without the schema-8 budget keys. The next tick reloaded the Config defaults and is_pre_adaptation_stall() still matched — an infinite resume, and the sweep kept sparing it. Resume (and with_budgets()) now stamp the keys present, which is the moment adapted sizes are written onto a record that never had them. No REST change.
  • A stall reason written by a later tick reported that tick's live memory_limit and max_execution_time rather than the pair in force when the chunk died (ADR-0015). Dispatcher::$host_limits held the pre-raise reading for the current PHP process only, so a run that adapted over hours described the wrong kill. The first tick now persists the pre-raise and post-raise pair on the job, and stall_reason() reads it from the record. A schema-8 record written before those fields still parses. No REST change.
  • is_pre_adaptation_stall() treated any diagnosed failure whose budgets were still zero as the 0.5.1 resume. This release always writes the schema-8 budget keys (as 0 until a stall shrinks them), so a structure-only or index stall — failed, diagnosed, keys present at 0 — was classified as that resume and the TTL sweep spared it forever. The predicate now means what ADR-0015 already said: failed, diagnosed, and the budget keys absent from the record. A this-release write that has the keys at 0 is ordinary residue. No REST change.
  • A failed job left its part-built container on disk, invisible to the stranded-job sweep because GET /extractions lists only non-terminal jobs. A job this release fails is never resumable, so the container and its index sidecar are now deleted at the moment of failure; the record stays so a poll still reports failed with its reason. A stall recorded by 0.5.1 or earlier still keeps its staging — that is the resume. No REST change.
  • Nothing ever reclaimed a failed job's record, so a site accumulated one working directory per failed run until the plugin was uninstalled — and after the record split each of those carries the whole selection in job.json, which is megabytes on a large one. failed was skipped by the TTL sweep along with every other terminal state, but it is the only one that leaves anything behind: consume, cancel, and the sweep itself all purge the directory at the transition. The sweep now applies its usual two windows — the heartbeat TTL and the absolute lifetime ceiling — to a failed record too, so a failure is residue for one TTL rather than forever, and a later poll of a reclaimed job is the same 404 any other purged job answers with. The stranded pre-adaptation stall is spared however old, because that record is what the resume path re-drives; Extraction_Job::is_pre_adaptation_stall() is now the one definition of that shape, read by both the resume and the sweep, so the two can never disagree about which container is still needed. No REST change.
  • A failed extraction could not be resumed, so a run an earlier release stranded cost the whole thing (ADR-0015). The state needed to continue already existed — Build_Progress, the append-only container, Sealed_Writer's truncate-on-resume — and what was missing was permission. A job stalled and failed by 0.5.1 or earlier, which had no way to adapt and so died at the first wall, is now re-driven from its persisted progress by the next tick or the watchdog: the container is truncated to the last acknowledged offset so an uncommitted tail cannot be duplicated, the spent attempt counter is reset, and the job re-enters running with smaller budgets. Upgrading over a dead run recovers it instead of restarting it. Two failures are deliberately never re-driven. An opaque failure (an unexpected throw, which records no reason) would retry a permanent error forever. And a stall this release recorded is by construction one it already shrank its way to the floor over, so re-driving it would re-run a search whose every remaining step has been tried — which is also why the resume condition is the absence of the schema-8 budget keys rather than the presence of a diagnosed stall: the latter reads more generous and would in fact be unsatisfiable. Re-entering running honours the same max_active_jobs ceiling the create path does; because the tick lock is per job and cannot serialise against a create for another one, the resume takes the slot, re-checks, and restores the failure verbatim if it lost the race, so an explicit create always wins. No REST API change — there is no new verb; the existing tick path and the watchdog are the trigger, and a status poll still cannot revive a failed job.
  • The chunk bounds were unmeasured constants, and the file-part one is what killed a 74-minute production run (ADR-0015). DEFAULT_CHUNK_SIZE is 8 MiB; of 28,021 files packaged successfully the largest was 3.87 MB, so no part anywhere near 8 MiB had ever actually been produced on that host. The job then died three times at byte 0 of the first file large enough to need a full-size part, and failed under the stall rule. Picking a smaller constant would repeat the mistake. A stall now halves the bounds the chunk that died actually spends, persists them on the job, and continues, leaving the job running. A file part spends one bound. A table slice spends two, and both give: table_chunk_bytes caps only what is rendered and sealed, while the rows are materialised in memory by $wpdb->get_results() under table_chunk_rows alone — the half a memory kill is likeliest to have happened in — so halving the bytes and leaving the fetch as large as the one that just died would be an adaptation in name only, burning another full attempt window at the same real size. Only a chunk whose bounds are all at their floor still fails the job. Halving is a slow search by design, and what bounds it is not the floor but the sweep's absolute lifetime ceiling, which is measured from the last real progress.
  • A large extraction spent almost all of its time writing bookkeeping rather than packaging anything, and got slower the further it went (ADR-0014). Dispatcher::advance_one_chunk() saves the job record twice per chunk — once before the work so a tick killed outside PHP still leaves its attempt counted, once after to persist the progress — and each of those saves re-encoded and rewrote the whole record: the entire file selection, which cannot change, plus a segment_names list that grew by one entry per sealed chunk. On the production run that forced this, the record started at 4.3 MB and grew from there, across 49,116 chunks: on the order of half a terabyte of JSON written to move 2.23 GB of files, at 3.9–4.8 files per second and falling. Measured on the real classes over a production-sized selection, save() was 96 % of the per-chunk time and packaging the file itself was 4 %; the same work with a small selection ran ten times faster, which is the whole finding, since the packaging was identical in both. The record is now split on what is unbounded rather than on what is immutable: job.json holds the selection and is written once, state.json holds everything a tick changes and is the only file a save rewrites, and it measures 598 bytes on a 49,116-path selection. Re-measured on the same fixture, per-chunk cost falls from 6.72 ms to 0.49 ms, a factor of 13.8 — and, more to the point, the same run with a 1,200-path selection now costs 0.54 ms per chunk against the big selection's 0.49 ms, where before it was 0.66 ms against 6.72 ms. Per-chunk cost no longer varies with how much was selected. That factor will not carry over to a slow host, and you should not expect it to: it is the factor where save() was 96 % of the work, and on the production run's own decay slope save() was 45–56 % of a chunk, so expect roughly a factor of two there — about 3.2 hours of file phase down to about 1.5. Over half of that host's per-chunk time is still unexplained by anything measured here; ADR-0014 says what is worth measuring next and why it needs a real host to settle.
  • The saves no longer pretty-print. The job files are only ever machine-read, and the flag added about a seventh to the bytes of every write for no reader's benefit. It was never the expensive part — encoding tens of thousands of strings is — so this is a byte saving, not the fix.
  • A table page that failed to read partway through a dump — a deadlock, a lost connection, a killed query, max_allowed_packet — came back from $wpdb->get_results() indistinguishable from a page that simply ran out of rows. Table_Dumper::fetch_rows() read that failure as the end of the table, dump_chunk() marked the table complete, the build moved on, and a ready job published an artifact whose dump for that table stopped at an arbitrary row and reloaded into MySQL without a single error — a silently truncated table, the one outcome Table_Dumper's own class docblock says must never happen. fetch_rows() now checks $wpdb->last_error immediately after the read, which wpdb clears at the start of every query and so describes this read alone, and throws when it is non-empty. The throw is an opaque failure (ADR-0015): it fails the job loudly and is deliberately never resumed, because a database fault is not the kind of stall a retry can adapt around. No REST change.
  • A table slice's row budget did not bind below 100 rows: dump_chunk() rounded it up to a whole INSERT batch before use, so every value from 1 to 100 fetched exactly 100 rows regardless. That budget is the half of the stall adaptation (ADR-0015) most likely to be the one a memory kill happened in, and it kept halving well past the point where the fetch stopped changing — roughly seven further halvings, each costing a full attempt window on a host that was already failing, before the job failed at the floor anyway. The same rounding also let the budget overshoot: 250 rows fetched 300, the wrong direction for a bound the adaptation is searching downward. The rounding now goes down to a whole batch while the budget still has room for one, and is taken literally below that, so it can shrink all the way to a single row. insert_statements()'s batching, the alignment that keeps several slices byte-identical to one, is unchanged; the byte-identical property now holds for any row budget at or above one batch and is deliberately given up below it, where the alignment was unreachable regardless. No REST change.
  • Sealed_Writer::finalize() deleted the index sidecar as its last act, right before Artifact_Builder renamed the finished container into the served downloads directory — the one window in the whole chunked build where a killed worker leaves a complete artifact with no way back to it. The next tick's resume() fails closed on the missing sidecar (correctly: a missing sidecar could otherwise mean segments already sealed would be silently dropped from the published index), the opaque throw failed the job with no reason, and a failure this release records deletes the in-progress container as staging residue — destroying an extraction that had, in fact, already finished. The sidecar is now pure working state until the container is actually published: finalize() leaves it on disk, and Artifact_Builder discards it, via the now-public Sealed_Writer::discard_index(), only after the rename succeeds. A crash in that window is no longer special — the next tick truncates the container and the sidecar back to the last committed offset through the same resume() path every other chunk boundary already uses, re-packages the last segment, and finishes normally. No REST change.
  • Artifact_Builder::read_part() cast a failed or short fread() below a file's end to an empty string via (string) fread(...) and sealed it as a legitimate empty part, rather than checking the result. The offset never advanced, so the file never finished — but the chunk was still recorded as progress, and progress resets all three safeguards that exist to stop a wedged build: the stall counter (cleared every tick), the heartbeat (refreshed every tick), and the sweep's absolute lifetime ceiling (measured from the same refreshed timestamp). A transient I/O fault on a networked or overlay-mounted uploads directory could therefore wedge a job forever, holding the single global concurrency slot with nothing left to stop it. A bounded read below the end that returns false or '' now fails the job outright (an opaque failure, per ADR-0015, never re-driven) instead of sealing an empty segment. A zero-byte file still completes as exactly one empty segment — that path is untouched. No REST change.
  • Sealed_Writer::add_segment() wrapped its caller's plaintext string in a php://temp stream and immediately read the whole stream back into another string — a round trip through the disk to arrive where it started. php://temp keeps only 2 MB in memory before spilling to a real file in sys_get_temp_dir(), and both of this plugin's default chunk budgets (8 MiB for a file part, 4 MiB for a table slice) sit above that threshold, so on a default installation essentially every chunk wrote production plaintext to an unencrypted file and relied on PHP to unlink it — the one thing this build cannot assume, since a host killing the worker mid-tick is the very case the resumable loop exists for. add_segment() now takes the plaintext directly as a string; the caller already held the whole segment as one before this call, so nothing is lost by skipping the stream. The container's wire format, framing, and api_version are unchanged — every byte add_segment() writes is identical to before. No REST change.

Added

  • docs/release-procedure.md documents the release procedure end to end — pre-tag checks, the version decision, the changelog step, building and publishing the archive, and the coordinated case with kntnt-wp-skills — including two guards that previously lived only in the maintainer's head: composer test:integration:mysql must pass before a tag is cut, and Status_Controller::HONOURED_BEHAVIOURS must name every caller-visible behaviour a build honours (ADR-0017). The three steps the document could not establish from the repository — who publishes the GitHub release and by what command, where the "Cross-repo release order" reference resolves, and how the production site actually receives the update — are now answered in §7, §8, and by tracking plans/, with the questions kept beside their answers so a reader can tell a decided step from a forgotten one. No REST change; no code change.
  • GET /extractions accepts an optional state=all query parameter (ADR-0019). By default the listing is unchanged in every byte — the caller's own non-terminal jobs only, proven by re-running the pre-existing listing assertions unmodified against the same fixtures. state=all additionally admits the caller's own terminal jobs (consumed, cancelled, failed, expired), each carrying only id, state, created_at, and updated_at — never progress, never download_url — so a caller can answer "is there sealed data of mine still on this site" without a second, administrative surface. The owner scope is unconditional and unaffected: another user's job, terminal or not, is never listed. A malformed state value is 400 rest_invalid_param, decided by the route's own validate_callback before the callback runs. state is added to Status_Controller::HONOURED_BEHAVIOURS, sorted, rather than bumping api_version: the parameter is additive and absent-by-default, and changes no artifact shape (ADR-0017). api_version stays 7.
  • docs/container-format.md specifies the sealed container's byte format normatively: the header and per-segment framing, the trailer's sealed index, the reading algorithm, the reassembly rule (same-name segments concatenate in index order), what the framing does and does not reveal, and how FORMAT_VERSION relates to api_version. It replaces a class docblock as the single place both this repository and the only production reader, kntnt-wp-skills, can point at. classes/Crypto/Sealed_Writer.php, docs/adr/0009-per-segment-encrypted-artifact-sealed-to-caller-key.md, AGENTS.md, and README.md now cross-reference it. No REST change; no wire-format change.
  • An authenticated GET /status now carries honours, a sorted list of the caller-visible behaviour names this build implements — attempts, chunks_done, disclosure, skipped_files, state, strict, and unknown_resource_names (ADR-0017). The last of those was found missing by the release procedure's own pre-tag review (docs/release-procedure.md §3 step 3) rather than by any automated check, which is what that step exists for: POST /extractions has named every offender in data.tables and data.files on a 404 kntnt_extractor_unknown_resource since earlier in this same unreleased batch, and a caller had no way to discover that except by inferring it from api_version — the inference this list exists to remove. api_version answers a different question, "may I proceed with this artifact contract at all", and stays exactly where it was: unauthenticated, unchanged, still exactly { api_version } for a caller with no credentials. honours answers "what does this build actually do", a question api_version cannot answer — strict shipped without moving it, so no version number ever will distinguish a build that honours it from one that does not. This does not move api_version. The existing capabilities member — the caller's own WordPress capabilities — is unchanged and keeps its meaning; honours is a distinct member so the two are never confused.
  • GET /extractions/{id} now carries an optional attempts member — the last eight packaging chunks the job has begun (at, kind, name, offset) — so a long or stuck run can answer "what was attempted" without widening the audit log (ADR-0016). A queued job omits the member; a failed job keeps it. GET /audit-log is unchanged and still records only the kntnt_extractor_job_ready transition. The persisted ring is four scalars per entry and holds no selected path. Schema 8 (unreleased) holds the new field; the REST API version stays 6 because the member is additive — an old client that ignores unknown keys still works.
  • POST /extractions now names every missing table and every missing file on a 404 kntnt_extractor_unknown_resource, in data.tables and data.files. The message is unchanged; what was missing is no longer a guess. An optional strict member, defaulting to true, keeps today's hard fail. strict: false skips vanished files — gone between the manifest walk and the POST — records them on the job as skipped_files, and returns that list on the 201 and on every later poll. A missing table still 404s, because silence there is data loss; a traversal or a null-byte path is never a skip either. A selection that is empty after the skip is the same 404, with every vanished name still in data. Schema 8 (unreleased) holds the new field; the REST API version stays 6 because the new members are additive and the default is current behaviour — an old client that omits strict and ignores unknown data keys still works. Clients that must understand the new members to be correct would have required a bump; these do not.
  • Every tick now asks the host for execution time and memory before it packages anything, through the new KNTNT_EXTRACTOR_MAX_EXECUTION_TIME (default 900 seconds) and KNTNT_EXTRACTOR_MEMORY_LIMIT (default 1 GiB) knobs (ADR-0015). This is the cheap half of the stall remedy and runs first: shrinking the chunk searches downward at two attempts per step, while this asks upward once, and a granted ask removes the search entirely where a host limit rather than the work itself was the killer. Both are requests. A managed host may lock either directive or disable set_time_limit(), a container cap overrules memory_limit whatever PHP reports, and PHP's timer does not count time spent in system calls on Unix — so a grant is neither certain nor sufficient. The raise never lowers a limit the host already grants, and either knob set to 0 asks for nothing. The first tick persists the pre-raise and post-raise pair on the job, and a stall reason reads those two pairs from the record: what the host is configured with, and what the run is actually under. Equal pairs mean the host refused, so only its configuration can help; differing pairs mean the raise was granted and the chunk still died, so the kill came from the web server or the container rather than from PHP — a diagnosis nothing in the plugin could previously make.
  • A GitHub Actions workflow runs composer gate — phpcs, PHPStan, the WordPress Playground integration suite and the release-archive build test — on every push to main and every pull request. The gate was already comprehensive but ran only when someone remembered to run it, while the release archive is built straight from committed content with no verification step in between. No REST change.
  • tests/Integration/DDEV/table-dumping-test.php drives Table_Dumper directly against a real MySQL/InnoDB database across a single-column key, a composite key, no key at all, and a table of few fat rows: every row is carried exactly once and in the key's own order, a composite key is paged on all its columns rather than only the first, a row-bounded slice's concatenation is byte-identical to a single-slice dump, a byte-bounded slice never comes back empty and a page it cut short is never mistaken for the end of the table, the keyless fixture completes via the LIMIT/OFFSET fallback, and — the assertion only a real engine can make — the dumped SQL reloads into a table row-for-row identical to the original, including NULL, an empty string, a quote, a backslash, a newline, and a multi-byte value. The fast Playground suite runs on SQLite, whose translation of Table_Dumper's SHOW KEYS, keyset predicate, LIMIT/OFFSET fallback, and SHOW CREATE TABLE DDL had never been checked against the engine the plugin actually runs on in production. No REST change.

Changed

  • ADR-0015 now names the single condition that retires the pre-schema-8 back-compatibility branches together — the sweep's pre-adaptation exemption, is_pre_adaptation_stall(), the resume path, Sealed_Writer::seed_index(), and Build_Progress::$legacy_names — and records that the condition is not checkable in general. Documentation only; no code changed.
  • api_version moves from 6 to 7 (ADR-0018), for the define-disclosure allow-list above. Read against 0017, which bounds api_version to the artifact's shape and routes every other caller-visible change through honours instead, this bump is a deliberate exception rather than a shape claim: kntnt-wp-skills currently classifies a null-valued define by name alone and ports an unrecognised one into a local wp-config.php as define('X', null), which then reports defined('X') === true and silently defeats that plugin's own fallback default. Against the wider allow-list, defines that were never null before now are, and the client's own exclusion list does not yet know to treat that as withheld rather than legitimate — an already-shipped client's existing, unmodified behaviour becomes unsafe, not merely incomplete, which honours' "an old client is free to ignore an unknown name" guarantee cannot cover. kntnt-wp-skills must be updated and released in step, and both must be installed, before any extraction run against this release — it pins a verified ceiling on api_version and refuses to run against anything higher until that ceiling is raised in a coordinated release (plans/README.md → "Cross-repo release order"). disclosure is also added to Status_Controller::HONOURED_BEHAVIOURS, so a caller can check for the new member by name like any other honoured behaviour.
  • Sealed_Writer::resume() truncated both the container and its index sidecar unconditionally on every resume, even though the anchor check just above it had already proved the common case — a tick killed cleanly between chunks — leaves the file exactly at the committed offset, so the truncation discarded nothing while still spending a synchronous metadata round trip per chunk, on a container that can grow past a gigabyte; both truncations are now guarded on the file being strictly longer than its anchor, and a file already at that length skips the call outright. resume() also stat'd each path with is_file()/filesize() before opening it, a check-then-act pair with a gap between what was checked and what the following fopen() actually reopens; it now opens the container and the sidecar first and validates their length with fstat() on the already-open handle, closing that gap and dropping the now-redundant stat call. This is a static count of filesystem round trips removed per chunk, not a measurement on a real host, and no throughput claim is made. A container or sidecar shorter than its committed offset, or one that cannot be reopened, still fails closed, and the two faults now throw distinguishably worded exceptions on both the container and its sidecar. No REST change.
  • max_stall_attempts now defaults to 2 rather than 3, because what exceeding it does has changed (ADR-0015). It used to fail the job, so a false positive cost the whole run and caution was worth several attempts; it now halves the chunk and carries on, so a false positive costs throughput and nothing else, while each attempt costs a full execution-time limit — paid 23 times over on the way from 8 MiB to the floor, and only ever on a host that is already failing. Not 1: an FPM reload, the kernel's OOM killer picking this worker because of a neighbour, and a momentary I/O stall all look identical to the counter, and at one attempt each of them would permanently halve a budget that never recovers within a run.
  • The ordered segment names left the job record. They accumulate in an append-only index sidecar beside the in-progress container, owned by Sealed_Writer — the only thing that ever read them, and only at finalize — and the record keeps their count and the sidecar's committed byte length instead. The sidecar is anchored exactly as the container is: both committed lengths are persisted together and both files are truncated back to them on resume, so a tick killed between appending a segment and recording it can never leave the index naming a segment the container no longer holds. A missing sidecar fails the build closed rather than publishing an index that omits everything sealed before it.
  • The persisted record's schema version is now 8. Schema 7 split the record and moved segment names to the sidecar; schema 8 adds the per-job chunk_size, table_chunk_bytes and table_chunk_rows a stall persists when it halves them, and — still unreleased — the first-tick host_memory_limit, host_max_execution_time, raised_memory_limit and raised_max_execution_time a stall reason reads, the skipped_files a strict: false create records, and the bounded attempt_log a tick records so a poll can name what was begun. A record written by an earlier release still parses: an absent budget reads as zero (the Config default), an absent limit pair reads as not-yet-measured, an absent skip list reads as nothing skipped, an absent attempt log reads as nothing begun, and a schema-6 segment_names list is still rebuilt into the sidecar on the first resume. Installing this release over a running extraction still is not advised — the plugin allows one active job at a time — but it no longer discards one.
  • The concurrency ceiling moved to Job_Store::has_free_slot(), beside the count it bounds. Two callers now ask whether another job may occupy the slot — the create endpoint and the resume path — and the knob, its default and its clamping were about to exist once per caller. Extractions_Controller no longer reads the Config seam at all as a result.
  • No REST API change. The API version stays 6. progress.chunks_done means what it has always meant and still moves once per sealed chunk; it is read from a counter now rather than from the length of a list, which no client can observe. The sealed container's wire format is untouched, so an artifact from this release is byte-compatible with one from 0.5.1 and kntnt-wp-skills needs no change.