feat: complete the beta admin and runtime config stack - #2
Merged
Conversation
Replace the constant-HMAC session cookie with a signed payload carrying subject, label, kind and expiry, verified server-side on every request. This is the foundation OIDC plugs into. Close three login weaknesses: - ADMIN_TOKEN no longer falls back to FILEGATE_TOKEN, so brute forcing the admin login can no longer yield the Filegate master token. Startup fails when the two are equal. - Rate limit POST /login to 10 attempts per 5 minutes per client via @valentinkolb/sync. In-memory by default, Redis-backed when REDIS_URL is set. Note that Bun resolves REDIS_URL at process start. - Drive the Secure cookie flag from ADMIN_COOKIE_SECURE instead of the request URL protocol, which is http behind a TLS-terminating ingress. X-Forwarded-For is only trusted with ADMIN_TRUST_PROXY, otherwise a client could pick a fresh rate-limit bucket per request. Also upgrades @valentinkolb/ssr to 0.11.2 and stdlib to 0.16.0, adds DOM.Iterable to tsconfig so typecheck passes, and sets up bun test with an SSR plugin preload. BREAKING CHANGE: ADMIN_TOKEN is now required and must differ from FILEGATE_TOKEN. Existing session cookies are invalidated.
Authorization code flow with PKCE, enabled by setting OIDC_ISSUER, OIDC_CLIENT_ID, OIDC_CLIENT_SECRET and OIDC_REDIRECT_URL together. Endpoints come from discovery, and the ID token is verified against the provider JWKS with issuer, audience and nonce checked. OIDC_ALLOWED_GROUPS is optional: providers like Authentik restrict access on the client itself, so requiring a second allowlist here would be duplicate bookkeeping. When unset, startup warns that access control is delegated to the provider. ADMIN_TOKEN becomes optional once OIDC is configured and stays available as a break-glass login. Dropping it hides the token form entirely. The flow state cookie is SameSite=Lax because the callback arrives as a cross-site navigation; Strict would drop it and fail every sign-in. Kept deliberately small: no refresh tokens, no userinfo call, no session store, no provider-side logout. Sessions last 12 hours, then re-login. Extracts the signing helpers into lib/signed.ts so the session and the OIDC flow state share one implementation.
These subsystems measured useful state and kept it to themselves, so an operator had no way to see queue saturation, cache effectiveness, or a detector falling behind. - jobs: Stats reports workers, queue depth against capacity, in-flight count, and cumulative ErrQueueFull rejections and job panics. Queue pressure previously surfaced only as a 503 to the caller that hit it. - cache: Stats reports entries, capacity and cumulative hits/misses. Hit ratio was tracked nowhere, for either the path or thumbnail cache. - detect: Stats joins the Runner interface, reporting backend, interval, cycles, last scan time and duration, scan errors and outbound channel depth. The btrfs backend also reports the last generation per base path, which makes detector lag computable for the first time. Also counts scan errors that were previously silently swallowed: poll lstat failures other than "gone", and every logged btrfs failure.
Exposes state the server already tracked but kept in logs, Prometheus, or memory. All additive; no existing route changes behavior. - GET /v1/system/info: build version and commit, uptime, detector backend, effective versioning mode, per-mount health including writable and xattr support, and a curated set of limits. Mount health was computed once at startup and thrown away. Config exposure is an explicit allowlist, not a dump, so no secret can leak by default. - GET /v1/system/runtime: detector cycles, staleness and per-path btrfs generations, worker pool saturation, path and thumbnail cache hit ratios, and upload sessions by phase with write-slot usage. Every field is an in-memory counter, so this one is safe to poll. - GET /v1/health: real dependency checks (index, detector, mounts), returning 503 on failure. The bare GET /health stays byte-identical for existing probes. - GET /v1/uploads/sessions: lists sessions, optionally by phase. An interrupted upload previously left a session nothing could find, since abort needs an id that was no longer known anywhere. Health deliberately probes only mount existence; the write probe stays on system/info so a polled endpoint never writes to every mount. Adds Service.PingIndex as a cheap liveness probe, since Stats walks every entity and is unfit for a health check.
- Prometheus gains filegate_detector_stale_seconds, _cycles_total and
_errors_total plus filegate_path_cache_lookups_total{result}. Detector
staleness is the signal that detection died and the index is drifting.
- Authentication failures now reach the activity log as auth.denied.
The activity middleware only records requests with a known actor, so
401s previously left no trace at all.
- Go and TS SDKs cover the new system, health and upload-session routes.
- docs/http-routes.md and docs/metrics.md document both.
Worker-pool gauges are intentionally absent from Prometheus: the
scheduler lives inside the router, out of the metrics provider's reach.
They are on GET /v1/system/runtime, and metrics.md says so.
Filegate attributes every audit entry to the bearer token, so with one shared token several admins were indistinguishable in the log. It already accepted X-Filegate-Actor and recorded it as delegatedActor; nothing sent it. The admin now sends the session label on every upstream request. The actor lives in async-local storage rather than a parameter because client() is called from route handlers and from load helpers several frames deep, and threading an argument through all of them would touch every call site to move one string. The System page needed no change: actorName already prefers delegatedActor over the credential id.
Six defects where the interface reported something untrue. - Directory listings silently truncated at 100 entries and the item counter reported the truncated number as the total. The server has cursor pagination; the UI took page one and rendered it as the whole folder. Now follows the cursor, with a 5000-entry cap that is labelled when it bites rather than hidden. - The topbar health dot was hardcoded green and stayed green through a total outage. It now reflects GET /v1/health: Healthy, Degraded, Unreachable, or Unknown. - /search and /system had no error handling at all, so a Filegate outage produced a raw 500 instead of the error banner the other pages show. - Failed delete, rename, metadata and transfer threw the user back to the mount roots, losing their place. They now return to the folder they were working in. - Index rescan was fire-and-forget with console.error, so "rescan started" appeared even when the call failed. It now awaits and reports the actual outcome. - Search declared an error prop that was never passed, so glob failures were invisible. Verified against the running stack: a folder of 250 files renders all 250 with a correct counter, and with Filegate stopped all four pages return 200 with an error banner and the dot reads Unreachable.
Completes the remaining correctness gaps in the files view.
- Upload conflict mode was pinned to skip-existing even though the
broker supports skip-identical, rename, overwrite and error. It is now
a control next to the upload buttons, still defaulting to skip-existing
since that is the only mode that cannot destroy data when someone
re-drops a folder.
- Uploads had no cancel; the panel could only be closed, leaving the
transfer running and the sessions orphaned. A Cancel button now aborts
the client via AbortSignal and aborts every session created so far, so
staged bytes are released instead of waiting for the cleanup loop.
- The transfer dialog let the target folder be empty, which reached
resolveDirectory("") and failed with a bare "folder required". The
field is now required and prefilled with the current folder, and the
errors name the actual problem.
client().versions was used nowhere, so eight REST endpoints and the whole versioning feature were unreachable from the UI despite being a headline capability of Filegate. The detail pane now carries a Versions panel for files: history newest first with timestamp, size, label and pinned state, plus snapshot with an optional label, pin and unpin, restore in place or as a new file, delete a single version, and download the bytes of any version. Version content has no signed direct-URL endpoint, so unlike normal file downloads this one streams through the admin server. Mounts without versioning answer 404 on every versions endpoint, which the SDK documents as the capability check; the panel reports that as an explanation rather than as an error. The restore dialog states that the pre-restore snapshot is subject to the versioning cooldown. That is not a detail: with a recent capture the current bytes are replaced without a new version, which was confirmed against a running server rather than assumed from the docs.
nodes.thumbnailRaw was unused, so the server's thumbnail endpoint with its LRU cache and worker pool was never exercised by the UI. The files list gains a List/Grid toggle carried in the URL, so it survives navigation without any client state. Grid tiles show image previews for jpeg, png, gif and webp. Thumbnails proxy through the admin server because the browser has no Filegate token. ETag, Cache-Control and Last-Modified pass through in both directions, so a repeat view costs a 304 instead of a re-encode; this needed ifNoneMatch on the SDK's thumbnailRaw, which had no way to send a conditional request. The preview stacks on top of the file icon rather than replacing it and carries no background, so a thumbnail the server refuses -- unsupported format, oversized source, or a full job queue answering 503 -- leaves the icon visible with no error-handling script. Verified against the running server: previews render, a conditional request returns 304, and an unknown id answers 404 so the tile falls back to its icon.
Upload is one button again. It opens a modal holding everything the action needs: a dropzone that takes files and whole folders, the conflict-mode select, Upload folder bottom-left, and a primary Upload bottom-right that stays disabled until something is staged. Clicking the dropzone opens the native picker. Staged entries can be removed individually before starting. Dropped folders keep their structure. The drag-and-drop entry API yields File objects without webkitRelativePath, so directory entries are wrapped in a small object carrying the relative path; the SDK's upload contract is structural, so name, size and slice are enough. Directory reads loop until an empty batch because readEntries returns at most 100 entries per call and a large folder would otherwise lose its tail. Uses stdlib's showFileDialog and showFolderDialog instead of hand-rolled hidden inputs, and stdlib's writeCookie for the layout preference. The List/Grid choice now persists in a cookie, so it survives navigating away. The cookie is written client-side: the SSR handler builds its own Response and drops cookies set on the Hono context, which is the same reason the theme cookie works that way. Grid tiles lost their per-item borders; the thumbnail is boundary enough.
Adds Scope, Secret and Reason to the existing flag spec table rather than a second registry that could drift from it. 10 of 55 keys stay static: the three listener binds, write_timeout (an http.Server field), the two storage paths, detection.backend, the two metrics keys that decide route mounting, and auth.bearer_token, which stays static on purpose so a damaged runtime store cannot lock an operator out of the API used to repair it. Each carries its reason. The other 45 become runtime-adjustable. Three guard tests: every key is classified and every static one explains itself; secret-looking keys are on the deny list, since the config endpoints are meant to be complete and a new credential would otherwise leak by default; and the runtime majority is asserted, so a careless classification pass shows up as a failure.
A durable home for config overrides and runtime resources such as S3 access keys, in its own Pebble instance under a path separate from the index. That separation is the point, not a preference. `fg index rescan --new` removes the index directory outright, and with --skip-backup it does so without a copy, so credentials sharing that store would be destroyed by a routine rebuild. The index is derived and must stay disposable; runtime config is authoritative and reconstructible from nothing. A test simulates the rebuild and asserts the store survives it. Only overrides are stored, never the resolved configuration, so a value nobody touched keeps following its built-in default when that default changes in a later release. A bootstrapped_at marker makes environment and CLI seeding one-time rather than reconciling. Deletion therefore survives a restart even with the seed variable still set, which is what keeps a revoked credential from reappearing at the next deploy. An unparseable marker still counts as bootstrapped, since treating corruption as "never seeded" would resurrect exactly what the marker exists to prevent. Close is idempotent: the shutdown path has several error branches that can each reach it, and closing Pebble twice panics.
Introduces the snapshot the runtime half of the configuration will be read from, plus the resolver that produces it. Precedence is runtime override, then environment, then config file, then built-in default. Runtime wins because it is the most deliberate: someone changed it through an API while the service was running. domain.ConfigHolder publishes the resolved config behind an atomic pointer, so reads are lock-free and a reload swaps everything at once rather than letting a request observe half an update. ConfigManager applies changes: validate, persist to the runtime store, republish. A static key is accepted and stored but comes back in a restart-required list naming the running and desired values, instead of being silently ignored -- the failure a generic config API falls into most easily. Validate runs the same resolution without applying, so a UI can check input as it is typed. Provenance is tracked per key, because an operator needs to tell "I chose 3s" from "3s is the default". Viper cannot answer this (IsSet is true for defaults too), so the file layer is probed with a second viper that has no defaults registered. Reading a value by dotted path walks the existing mapstructure tags rather than a second 55-case switch that could drift. Writing keeps its explicit switch, since it must reject unknown keys loudly. Seven tests cover a runtime change taking effect without a restart, a static change reporting instead of lying, invalid input leaving the previous snapshot intact, overrides outranking the file and surviving a reload, clearing an override falling back to the file rather than the default, provenance across all four sources, and secrets rendering as a presence flag whose value tracks reality.
Wires the pieces together: serve opens the runtime config store, re-resolves the configuration with any overrides a previous run stored, and hands the resulting snapshot to the router. Adds storage.runtime_config_path, defaulting to /var/lib/filegate/config next to the index rather than inside it. Validation rejects a path that nests under storage.index_path, because rebuilding the index removes that directory and would take every stored credential with it. That is the one invariant this design rests on, so it is enforced rather than documented, with a test covering the nested, sibling and empty cases. RouterOptions.Config carries the snapshot. It is optional: a nil holder falls back to the values captured in the options struct, so existing callers and every current test keep working while consumers are converted one at a time. Verified against the docker stack: the service starts with the new store (startup fails if it cannot be opened), health and system/info answer, and the admin app passes its live checks unchanged.
Five routes over the config manager:
GET /v1/config/schema every key with type, scope, default, usage
GET /v1/config effective values with provenance
PATCH /v1/config apply a batch
POST /v1/config/validate resolve and validate without applying
POST /v1/config/reload re-read the sources
Schema exists so clients stop hardcoding the key list: an admin UI picks
its input control from the declared type, and a key added later appears
without a client change.
Changes arrive as a batch because some settings are only valid together
-- cors.allow_credentials cannot combine with a wildcard origin, so
one-at-a-time edits would have to pass through an invalid state.
A patch has three outcomes rather than two: applied and live, applied
but restart-required with the running and desired values named, or
rejected. The middle case is what a config API most easily gets wrong.
Secrets report {"configured": bool} and never their value, decided by
the spec deny list rather than by whichever fields someone remembered.
The service is an interface on RouterOptions because the specs, defaults
and validation live in the CLI package, which already imports the HTTP
adapter; passing the implementation in keeps the dependency pointing one
way. A nil service leaves the routes unmounted, so existing callers and
tests are unaffected.
Verified against the running stack: 56 keys reported as 11 static, 45
runtime, 5 secret; the bearer token renders as configured-only; a
runtime change took effect with no restart and its source flipped to
runtime; a static change came back as restart-required and did not take
effect; invalid values and unknown keys are refused with 400; validate
left the value untouched; and a runtime override survived a container
restart while a cleared one fell back to its environment value.
Makes the configuration API honest. Until now it reported keys as runtime-adjustable and answered success, while handlers kept using the values they captured when the router was built. liveConfig reads each runtime-scoped setting from the published snapshot per request, falling back to the boot-time options when no holder is supplied, so existing callers and tests are unaffected. CORS, real-IP and access-log middleware now evaluate their settings per request instead of being decided once when the chain is assembled. Upload body caps and the capabilities endpoint read live values too. Also fixes a pre-existing bug this made easy to observe: a body that exceeded upload.max_upload_bytes answered 500. Only the direct-upload handler translated http.MaxBytesError; path and node writes fell through to the internal-error branch, telling the caller the server had broken when the correct answer is 413 and "send less". Handled centrally in statusFromErr so every write path benefits, with a test. Verified end to end against the running stack: a 300 KB upload succeeds, the cap is lowered to 1000 bytes over the API, the same upload is refused with 413 and /v1/capabilities reports the new limit -- all without a restart -- and clearing the override restores the original behaviour.
Access keys move out of the config file and into the runtime store with real CRUD: create, rotate, update grants and limits, disable, delete. An access key with a secret, bucket grants and a rate limit is a principal, not a setting, and every change to one belongs in an audit trail. The adapter reads its key set through an atomic pointer, so a key created or revoked a moment ago takes effect on the next request rather than at the next restart. Validation runs before the swap, so a rejected update leaves the running set untouched. Secrets are returned exactly once, at creation or rotation, and never by the list endpoint. Seeding from the static configuration happens once, into a store that has never held keys, and is per resource kind so a deployment that enables S3 only later still gets its configured keys imported. Later starts log what they ignored and why. Two bugs found while wiring this up: - Attaching the adapter published before seeding ran, leaving it with an empty key set on first boot. Seeding now precedes attachment. - buildKeyStore rejects an empty set, which is right at startup and wrong at runtime: deleting the last key must mean nothing authenticates, not that the previous set stays live. SetKeys now installs an explicitly empty store. Verified against the running stack with the S3 listener enabled: the configured key is seeded and listed without its secret; create returns a 40-character secret once; rotate changes it; disable and delete work and a second delete answers 404. The property that matters most also holds: deleting the seeded key and restarting with FILEGATE_S3_ACCESS_KEY still set does not bring it back, and the log says the configured key was ignored because the store owns it now.
Removes the last two settings a fresh install had to provide. storage.base_paths defaults to /var/lib/filegate/data. Only that exact default is created on startup: a configured path is left alone so a typo fails the mount health check loudly instead of quietly serving a freshly made empty directory, which looks indistinguishable from data loss. auth.bearer_token is no longer required. When unset, a token is generated on first boot, stored in the runtime store, and printed once in a framed block that says it will not be shown again. Later starts reuse it silently. The REST auth middleware still fails closed until a token exists, so dropping the requirement cannot open the API. The image now ships its runtime directories. Distroless has no shell, so they are staged in the build image and copied with the nonroot uid; without them a container that configures nothing could not create its own mount and refused to start. Verified: docker run with no environment and no config file starts, prints a usable token, answers 200 for that token and 401 without one, and a restart neither reprints nor regenerates it.
The previous test required auth.bearer_token unless s3.enabled was set. Serve now generates and stores one on first boot, so rejecting an empty value at load time would make a zero-configuration start impossible. Loading is not the same as opening the API: the auth middleware still fails closed while no token exists. Adds coverage for the generated token being created once and reused, and for a configured token always winning over the stored one. A credential that changes on every restart is not a credential. This should have been part of the previous commit; it was committed with this test failing.
Adds a Settings page rendering every configuration key grouped by section, with its value, provenance (default, file, env or runtime) and scope. Runtime keys that are neither secret nor structured get inline editing; a runtime override can be reset so the key falls back to the file or its default. Controls are chosen from the schema endpoint rather than hardcoded, so a key added in a later release shows up without touching the UI. Static keys whose stored value differs from the running one appear in a restart-required banner naming both, instead of looking applied. S3 access keys get full management: create, rotate, enable, disable and delete. Secrets appear once in the confirmation notice, because they cannot be retrieved afterwards, and never in the table. TS SDK gains fg.config and fg.s3Keys with types. Verified against the running stack: all sections render, the bearer token shows as configured without its value, an int and a duration edit both apply, an invalid value is refused with the server's message, reset restores the previous source, and an S3 key can be created with a one-time secret, found in the table and deleted.
Completes the files view. Every action was strictly one file at a time, which hurts as soon as a folder holds more than a handful. Sorting by name, size or modified time, carried in the URL so it survives navigation and is shareable. Clicking the active column flips direction; a different column starts ascending. Headers carry aria-sort, which the plain th elements never did. Directories stay ahead of files regardless of the field, so sorting by size does not scatter folders through the listing. A filter box narrows the folder by name, and the item counter says "2 of 3 items" so a filtered view cannot be mistaken for a small folder. Bulk selection adds per-row checkboxes with a select-all that reflects the indeterminate state, plus bulk delete and bulk move. Each item is applied on its own and results are reported per item: "Deleted 2, 1 failed" with the failing ids, because one aggregate outcome for twenty files tells an operator nothing about what needs attention. Items run sequentially so a bulk delete cannot saturate the server. Selection deliberately lives only in the DOM and is not carried across navigation: acting on files you can no longer see is how accidental mass deletions happen. Sorting and filtering run in the admin server rather than as new query parameters upstream, because the whole directory is already loaded for the listing and a round trip per sort click would buy nothing. Verified against the running stack: default and both size directions order correctly, aria-sort is set, the filter narrows and the counter says 2 of 3, a bulk delete of two of three reports the count and leaves the third, an empty move target is refused with a usable message, and a bulk move succeeds.
Two layout defects on the settings page. Row borders ended at different x positions depending on whether the row had action buttons. The cause was display:flex on a <td>, which takes the cell out of the table box model so it no longer participates in column sizing. The cell is a table cell again and an inner span carries the flex layout. The stacked section panels had no vertical gap. Every other page nests its panels in .stack or .grid, which supply it; these sat directly in .main. Added a page-stack container rather than per-panel margins, so spacing stays a property of the layout instead of the component.
Three fixes to the settings page. versioning.retention_buckets rendered as "[object Object], ..." because the generic array path stringified its objects. Retention rules are the answer to "what happens to my versions?", so they now read as sentences: "all within 1h · max 24 within 1d · max 30 within 30d". Values are formatted through stdlib rather than printed raw: byte counts as 64 KiB, durations without Go's trailing 0s, counts with thousand separators. The exact value stays one click away in the edit dialog, which is where precision actually matters. The schema gained a unit field for this. The type alone cannot carry it: a byte limit and a max-count are both ints, so 65536 and 100 are indistinguishable without knowing what the number means. Scrollbars use scrollbar-gutter: stable, which is what removes the layout shift when one appears between pages of different height, plus a thin thumb over a transparent track. Deliberately not a JS-driven overlay scrollbar: forcing overlay behaviour everywhere takes the element out of the platform's keyboard handling, assistive tech and the OS "always show scrollbars" setting, which is a bad trade for a few pixels.
…editable The config endpoint emitted Go field names and nanosecond durations for versioning.retention_buckets, because RetentionBucketConfig carries only mapstructure tags and nothing told encoding/json otherwise. The UI could not read it, so every rule rendered as "all within ?". Adds apiv1.RetentionBucket with keepFor and maxCount, converted at the boundary. The domain struct stays as it is; the API shape is now a decision rather than a side effect of Go's defaults. Retention is also editable now. It was excluded as "structured", but it is the policy an operator most wants to change: it decides what happens to their versions. Editing uses the syntax the CLI flag already documents, keep_for=1h,max_count=-1, one rule per line, instead of inventing a second format for the UI. Note the asymmetry, which is commented at the parser: reading returns camelCase while writing must send the mapstructure names, so echoing the response shape back would silently produce empty buckets. s3Keys stays non-editable here on purpose. Access keys are managed as resources above, with rotation and an audit trail, not as a config value. Verified against the running stack: rules read as "keep all within 1h · keep 24 within 1d · ...", a new policy applies and renders, a rule missing keep_for is refused by name, and reset restores the default.
The raw text field was the wrong shape for this value, and it was also broken: rules were joined with newlines, which a single-line input collapses, so the dialog showed them run together as one string. Retention is now edited as what it is -- a list of (window, count) rules. Each row has a window with presets for the common spans, a count, and a "keep all" checkbox instead of asking anyone to type -1. A live summary reads back the policy as sentences while it is being edited. Raw mode stays one click away, because copying a policy between installs wants text and a structured form should never be the only way to express something; switching between the two carries the current rules across. Saving refuses an incomplete policy rather than accepting it. That is not cosmetic: tidyDuration turned an empty window into "0s", which is a valid zero-length window, so an unfilled row would have silently started pruning recent versions. Empty now stays empty and the editor says which rows need a window. Both modes serialize to the syntax the CLI flag documents, so the server sees one format from one parser. 10 tests on the pure helpers, since the text round trip is exactly where a bug produces empty buckets without failing. 53 admin tests total.
Fixes a leak I introduced with the settings page: a newly created or rotated S3 secret was carried in the redirect as ?notice=..., which puts it in the browser history and in every access log between the browser and the server. Creation and rotation now go over JSON and the secret is shown in a dialog that says it cannot be recovered, with a copy button using stdlib's clipboard helper rather than asking anyone to select 40 characters by hand. Byte counts and durations get a number-plus-unit editor instead of a raw field. Typing 1073741824 by hand is an error waiting to happen, and the plain field gave no hint which unit a number was even in. The two kinds cover roughly a third of the configuration. Values are decomposed into the largest unit that divides them exactly, so 1073741824 opens as 1 GiB and goes back as an integer. The key dialog now offers the mounts that actually exist instead of free text: a grant naming a bucket the server does not have is rejected on save, which is a slow way to learn it. Key generation stays on the server. Doing it in the browser with stdlib's crypto would mean the secret exists in the page before the server ever sees it, and would depend on browser entropy for a long-lived credential. quantity.ts imports prompts lazily, because prompts registers document listeners at module load and that made the file unimportable without a DOM, which in turn made its pure helpers untestable.
Icon plus label on every action, with the label collapsing on narrow screens so buttons become icon-only there. The label stays in the DOM rather than being dropped, so the accessible name never changes with the viewport. The font is self-hosted. An admin panel for a self-hosted file gateway must not depend on a CDN at runtime; it may well run somewhere with no route to the internet. The shipped @font-face is rewritten at build time to reference only the woff2 we serve, and to use font-display: block so a slow load leaves the space blank instead of flashing tofu boxes. Every icon element is aria-hidden: an icon font renders private-use codepoints that a screen reader would otherwise read as garbage. Meaning comes from the adjacent label, or from aria-label where the button is icon-only. The sort indicator replaces the ▲▼ characters, which some screen readers announce as "black up-pointing triangle". Both copy buttons in the credential dialog are plain now. Marking the secret copy as primary put two competing primaries in one dialog when only the flow-completing action should be one. Copy feedback swaps the icon to a check, or to a warning when the clipboard is denied. Found while wiring the font routes: c.header on a handler that returns a fresh Response is discarded, so Cache-Control never reached the client. The existing static routes have the same mistake, masked because Bun.file infers Content-Type from the extension. The new routes set their headers on the Response. Cost worth stating: 460 KB woff2 plus 205 KB CSS covers all ~5900 Tabler icons for the 31 this app uses. Both are immutable-cached, so it is a one-time transfer, but a generated subset would be roughly 10 KB if that ever matters.
Completes the icon pass. Several buttons were missed by the first sweep; an audit over every element styled as a button now reports none without one. Dialog buttons get theirs from the single prompts factory rather than at each call site, so confirm, cancel and danger are consistent everywhere by construction. Every numeric config key now declares what it counts. This started from cache.path_cache_size reading as a bare 100,000: the obvious fix looked like formatting it as bytes, but it is an entry count, and rendering it as 97 KiB would have claimed a cache holding 100,000 paths occupies 97 KiB when it is closer to tens of megabytes. Three keys are named "size" while holding a count, which is what makes the guess tempting. They now read "100,000 entries", "1,024 entries" and "500 events", and their usage text no longer says "size" where it means a maximum number of items. Byte keys are unchanged and still render through stdlib as 1 GiB or 64 KiB. Two tests pin this: every numeric key must declare a unit, and the four count-holding keys must not be marked as bytes.
Six panels on the System page over the endpoints from phase 1: service health with its individual checks, change detection with a stalled state, worker and write-slot saturation, cache hit ratios, version retention, and an upload-session inspector that can abort an orphan. Also closes the gap phase 1 left open. The pruner computed six numbers, fed three to Prometheus and dropped the rest along with any record of when it last ran, so "is retention working?" had nothing to answer it. A lifecycle register keeps the last run: all six PruneStats fields, duration, next scheduled run, and the error when a run fails, which previously left only a log line. Values refresh by polling rather than SSE. The payload is one small JSON document, the server keeps no per-client state and a dropped request needs no recovery path; an SSE stream would hold a connection per open tab for values that move on a seconds timescale. One request covers runtime, health and sessions so a tick is not three round trips. The first frame is server-rendered, so the page is complete without JavaScript and the poll only keeps it current. Each part degrades on its own: a stalled detector must not blank the panels that would explain it. Only changed nodes are written, so a value being read is not reflowed under the cursor every five seconds, and after three consecutive failures the grid says updates are interrupted rather than showing numbers that quietly stopped moving. Verified against the running stack: panels render with real values, detector cycles advance between polls, and with Filegate stopped the endpoint answers 502 while the page still renders with an error banner and per-panel fallbacks.
Three items left over from the retention analysis. POST /v1/versions/prune runs a retention round on demand. The background loop runs on an interval an operator cannot see the effect of, so after tightening a policy the obvious question -- did that do anything -- had no way to be answered. Both the ticker and the manual trigger go through one guard: the per-file locks already keep concurrent rounds from corrupting anything, but two scans would duplicate the work and report halves of the same result separately, so a second request answers 409. It deletes data, so it is a POST, it is confirmed in the UI, and it appears in the activity log as versions.prune. Without versioning the route answers 501 rather than pretending a round ran and found nothing. List-valued settings get a chips editor. A comma-separated field made it easy to lose an entry to a stray comma and gave no feedback until after saving. Each entry is now discrete and removable on its own, and pasting a comma-separated value splits it rather than creating one long entry. Covers the four CORS lists and trusted proxies. The editor validates against the server before it closes. Cross-field rules live there -- allow_credentials cannot combine with a wildcard origin -- and learning that from a redirect means retyping the list. Found while running the Linux suite in a container: a linux-only test still called runVersioningPruner with the old signature. It does not compile on macOS, so a local go build never saw it. Verified against the running stack: the prune button reports all six PruneStats fields and lands in the audit log; an invalid backend and the credentials-plus-wildcard clash are both refused by the validate endpoint before anything is stored; a two-entry origin list applies and reads back.
SystemClient.UploadSessions folded "?phase=..." into the endpoint string, but newRequest assigns the endpoint to URL.Path, which escapes the "?" to %3F. The request went to /v1/uploads/sessions%3Fphase=in_progress, matched no route and 404'd, so the filter never worked at all.
The session listing and abort endpoints already satisfy the design ticket's cleanup and security criteria, but three properties were untested: a session token only works against its own session, abort refuses tokens minted without the abort scope, and the expiry sweep removes aborted rows. Aborting also has to leave the row behind so a late segment PUT cannot resurrect the upload.
The TypeScript SDK has had folder orchestration since the batch-upload work; the Go SDK only had the session primitives, so every caller wanting to upload a directory had to rebuild hashing, scheduling and progress themselves. uploadtree mirrors the browser pipeline: bounded hashing, batched session creation, one global segment-concurrency limit rather than a per-file one, and a single progress view. It differs in one place on purpose — a Go caller holds the bearer token, so it creates sessions itself instead of going through the browser's "allow" callback. Beyond parity it adds retry with jittered backoff (transport errors and retryable statuses only), resume by adopting in-progress sessions whose checksum still matches, and a direct threshold that routes small files to a one-shot PUT. That last one matters: sessions cost 2.5-3x a plain PUT for small files (see bench/results/2026-07-26-many-small-files.md).
The smart-batched-uploads design ticket made measurement an acceptance criterion and it was never done. The existing load generator answers "how many ops/s at steady state", which cannot show per-file overhead or session scheduling because it never runs a corpus to completion, so --mode tree uploads a fixed corpus and reports wall time plus a hash/create/PUT/commit split. Shapes and levers come from the ticket: photo folder, tiny logs and a node_modules-like tree, against keep-alive, HTTP/2, parallelism, batched session creation and segment sizing. HTTP/2 needs a TLS edge because the Filegate listener serves cleartext HTTP/1.1 only, so the bench stack includes one and the tool records the protocol it actually negotiated. Findings in bench/results/2026-07-26-many-small-files.md: per-file server work dominates, concurrency saturates near 32-64 in flight, HTTP/2 and batched session creation change nothing measurable, and commit is 2.4-10x the segment upload time because it copies every byte a second time. Packed segments are rejected on that basis. The runs also surfaced a 404 under load on PUT /v1/paths, documented in the same file.
Indexing a path took the parent's xattr as proof that the parent was in the index. Claiming an ID on the file and writing the entity are two separate steps, so a sibling request landing between them read the ID, skipped indexing the parent, and anchored its child to an entity that did not exist yet. Walking that parent chain answered not-found, surfacing as a 404 from PUT /v1/paths for a directory the caller was creating itself. parentIDForSync now requires an index row, not just an attribute, and ensureIndexed applies the same check before handing an ID back. First-time ID assignment also goes through claimID, which uses XATTR_CREATE so concurrent claimants adopt the winner's value instead of each keeping its own.
The config rewrite left the docs describing a service that no longer exists: a required bearer token, S3 keys owned by the config file, and no mention of the static/runtime split or the settings API. `fg config schema` prints the key catalog as a table, JSON, or the published reference page. `make docs-config` writes the page and a test fails when the committed copy drifts, so 55 keys across two scopes cannot quietly fall out of sync with the spec table the server validates against. Prose updates: the static/runtime split, starting with no configuration at all, runtime store persistence and backup, seed-once resources, the config and operations routes, and runtime S3 key management. The beta scope the epic asked for is now stated where a reader meets it -- single tenant, one all-or-nothing bearer token, trusted network, admin access equals config authority -- on the overview and in full on the security page.
Assembling a one-segment upload read the staged segment back and wrote the whole file out again, then read it a third time to hash. Renaming produces the same file for one directory update, and the per-segment checksum the copy verified is covered by the whole-file size and SHA-256 check the commit already performs. Halves assembly for single-segment commits, 2.29 ms to 0.91 ms median on 16 KiB files, and cuts photos-corpus commit time from 34-41 s to 30 s where the copied bytes actually mattered. The move consumes the segment, so commit now reuses an already-assembled file instead of rebuilding it. That is sound because a recorded segment is immutable -- re-uploading different bytes is refused with a conflict -- so the assembled file can only hold what the session declared, and the checksum is verified on every attempt regardless. Without it a commit that failed after assembly, on a destination conflict say, could not be retried.
Both SDKs had a direct-upload threshold and left it off, so the slow path was the default for the file shape that dominates real trees. A session costs three round trips and several fsyncs per file, and for a file that fits in one segment its resumability amounts to retrying that one segment. The threshold now defaults to the segment size: a file that would have been a single segment takes one PUT instead. Measured at 2.4x the throughput of sessions on 5000 files averaging 16 KiB. Go uses a negative value as the opt-out because Options' zero value has to keep meaning "use the defaults"; the browser helper uses 0. Four existing tests now opt out explicitly -- they exercise the session machinery and only used small files as fixtures.
The ticket blamed commit's cost on assembly writing the file a second time. That holds for large files and not at all for small ones: assembly is 8-20% of a 16 KiB commit, while publishing plus index write is 40% and the three bookkeeping phases are another 47%, all fsync bound. Removing the copy leaves the logs and node-modules totals inside the noise. Adds a commit preset to the harness -- the narrow set of session runs plus a large-file regression check, small enough to run once per build in one sitting, which this comparison needs because host drift here reaches 2.4x.
Committing an upload into a tree that does not exist yet cost one synced index write per path level, because syncSingle indexes one path per call and reaches ancestors by recursing. Writing a file at depth 8 took 10 durable writes against 4 at depth 2; it now takes 3 either way. Three changes, each measured by counting index batches rather than by wall time -- the host was saturated while this landed and wall-clock arms taken then disagreed by 3x in both directions. indexNewDirChain writes a freshly created chain in one batch. The levels were created together and are only reachable through each other, so one atomic write is also a stronger guarantee than a chain a reader can observe half-indexed. A crash before it leaves directories on disk and out of the index, which is the state the index is built to recover from. ensureSessionParent resolves the parent before trying to create it, which is what every commit after the first file in a directory does. The per-level loop took a path lock and walked its own prefix for each level, once per file. removeSessionArtifacts derives segment paths instead of globbing the stage directory, which every session on the mount shares -- a 5000-file upload was doing 5000 scans of a 5000-entry directory. Its two directory fsyncs are gone: they only made the deletion of staging garbage durable, and a resurrected staging file is swept by the cleanup loop. The committing marker and the commit record keep their fsyncs. The marker is what makes a crash between publishing bytes and recording the commit recoverable instead of a permanent conflict.
The listener spoke cleartext HTTP/1.1 only, so an operator whose proxy or service mesh is configured to speak h2 to its backends had no way to connect, and the absence was undocumented. server.http2_cleartext accepts unencrypted HTTP/2 alongside HTTP/1.1 on the same port. Go 1.24's Protocols.SetUnencryptedHTTP2 covers this in the standard library, so no new dependency: the server switches only for a connection that opens with the HTTP/2 preface, leaving HTTP/1.1 clients untouched. The listener's timeouts keep their meaning, because Go arms ReadTimeout and WriteTimeout per stream for HTTP/2 and IdleTimeout is set explicitly rather than falling back to ReadTimeout. Off by default. Accepting a second protocol on a listener should be an operator's decision, not something that arrives with an upgrade. Static, since the protocol set is fixed once the listener accepts connections. REST only -- S3 clients sign and stream over HTTP/1.1 in practice. Not a performance setting: across four repeats per configuration the two protocols differ by 1.5% to 6.7% at the median, inside a spread that reaches 2.6x within a single arm. The first run of the session pair looked like a clean 20% regression and the repeat reversed it; both are recorded in bench/results/2026-07-27-h2c.md. The bench client learned h2c too, so protocols can be compared on one port without a TLS edge in between.
Add authenticated plan/apply workflows and read-only configuration views. Move S3 resources into a dedicated operator page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification
No release is part of this PR.