v1.11.0

seekfs v1.11.0

This release combines faster resident searches, deferred result materialization,
and the minimal prompt-style desktop UI.

Search

• Distinct filenames are scored once per query with character masks and cached
  term bits. Ancestor bits serve supported path queries without reconstructing
  every path. Heap-backed compact indexes load into deduplicated packed storage.
• Broad component queries keep record IDs through bounded local rank heaps,
  then materialize at most limit × volumes base entries for global sorting.
  Unsupported queries retain the existing verifier, and overlays are merged
  with hidden base records excluded.
• Corrected volume/root prefix handling, filename-only prefix matches,
  directory aggregate-size filters, memo identity lifetime, and cancellation
  during name scoring, directory folding and global candidate verification.
• Added memory reporting for the identity and memo arrays. Low-memory mode
  leaves this lane disabled; SEEKFS_NAME_MEMO=0 also disables it.

Desktop UI

• Minimal dark prompt, compact logo header, footer counts and index health,
  quieter result rows, and context-menu shortcut hints.
• Ctrl+K or / focuses search; Escape clears a nonempty prompt. The clear
  button resets the query and sort and returns focus to the prompt.
• Reuse the existing multi-resolution icon for the favicon, excluding a 2 MB
  raster-wrapper SVG from the desktop executable.

Performance evidence

On a synthetic packed 50,000-file fixture with a warmed broad path query and
limit 20, the new memo/deferred path measured 4.85 ms versus 11.96 ms for the
existing verifier. Allocated bytes fell from 41.1 MB to 0.82 MB per query.
These are sample microbenchmark results, not a verified 10x end-to-end gain
or a measured real-volume resident-memory reduction. See
the review and reproduction command.

Included since the last published release

The unreleased v1.10.3 changes are included: the
direct command alias, simplified service environment names, watch command
paths with spaces, refactoring, dependency updates and expanded CI gates.
Remove the deprecated -skip-startup-sync flag from old service launch scripts.

Upgrade

No index rebuild is required. Replace both executables and restart the service
to load the new resident engine. Existing query syntax and index formats remain
compatible. The Windows binaries are unsigned. The release includes the CLI,
desktop UI, documentation, and a SHA-256 checksum for the zip archive.

seekfs v1.10.2

seekfs v1.10.2

A torn WAL tail no longer puts the service into a back-to-back fold loop, and
its memory with it.

Fixed

• A corrupt WAL tail blocked the post-fold WAL rewrite. After a fold, the
  service rewrites the WAL to keep only the frames that are not yet in the
  persisted index. If the WAL ended in a partly-written frame -- which happens
  when a persist is killed mid-append -- the read failed and the rewrite was
  skipped entirely, so the WAL kept every frame, including the ones just folded
  in, and stayed above its 64 MB size trigger. The next fold was then due
  immediately.

  Observed on a live F: index:

  10:46:58 background persist wal rewrite skipped volume=F: err=wal frame crc mismatch
  10:46:58 background persist complete volume=F: duration=3m15.538s
  10:46:59 background persist start volume=F: wal_bytes=93659099
  

  Every 3m15s fold writes ~4 GB and rebuilds the name trigram, which is what
  held the service at ~10 GB RSS. The rewrite now keeps the frames that read
  cleanly and drops the unreadable remainder, so the WAL returns under its
  trigger. A dropped tail is unrecoverable from the WAL either way, and the
  journal still covers that USN range on the next startup catch-up.

Validation

• go test ./..., go vet ./..., and the CI -race subset are green.
• New tests cover the corrupt-tail read (the good prefix is returned with the
  error) and a rewrite of that prefix (the WAL reads cleanly afterwards), plus
  rewriting with no frames resetting the WAL.

Upgrade

Drop-in for v1.10.1. No index rebuild or configuration change is required. A
service already stuck in the loop stops folding back to back as soon as the
first fold completes on the new build.

seekfs v1.10.1

seekfs v1.10.1

The service no longer indexes its own files, so it can no longer drive its own
persist loop, and the persist trigger scales with index size instead of firing on
a fixed change count.

Fixed

• Self-write churn fed the persist loop. Every USN change seekfs made to its
  own staged index, WAL, log, and builder spill files was replayed back into its
  own overlay. Measured on a live index: 91.7% of the 274,558 changes pending in
  the WAL came from seekfs's own files (251,870 of them), dominated by the .wal
  write events themselves. The service now resolves the directories that hold its
  own artifacts to NTFS file references and drops changes whose parent directory
  is one of them, before the WAL append and before the overlay apply, while still
  advancing the checkpoint past them. After the fix the same WALs carried 0
  self-write changes.

• The persist watermark was a fixed change count. It had to reach 64K pending
  overlay entries, which on a large index is a small fraction of the records, so a
  single busy folder (a build tree, node_modules, a spool) could make a fold due
  the moment the previous one finished. The watermark now scales with the record
  count (about 1/32 of the records, never below 64K), keeping folds proportionate;
  the WAL size, tombstone ratio, and dirty-age triggers still bound how long
  changes stay unfolded.

• A full disk became a permanent persist-failure loop. An interrupted persist
  leaves a multi-GB staged temp file next to the index, and the sweep that reaps
  them only ran at service startup. A service that stayed up for days accumulated
  them until every later persist failed for lack of space (4.6 GB observed on a
  drive with 2.6 GB free). Running services now sweep abandoned temps at most once
  an hour, and only files older than an hour, so a live persist is never touched.

Notes

• The name-gram spill directory now defaults to %ProgramData%\seekfs\gram-spool
  (inside the never-indexed seekfs dir) instead of the system temp directory, so
  the external builder's spill files fall under the same self-write exclusion.
  SEEKFS_NAME_GRAM_SPOOL_DIR still overrides it; see docs/CONFIG.md.

• Volumes that hold no seekfs files keep the previous behavior and pay only a map
  lookup per change.

Validation

• go test ./..., go vet ./..., and the CI -race subset are green.

• Verified against the real C: (10.0M) and F: (17.2M) indexes: 0 self-write
  changes in both WALs (was 91.7% of C:'s), no fold in over 20 minutes where the
  service previously folded every ~2.5 minutes, and persist_failures back to 0
  from 4.

• The NTFS file-reference form was checked against a real index (a directory
  resolves to the FRN the index stores, with its children linked by ParentFRN),
  which caught a bug where the 16-bit MFT sequence number was not stripped and the
  filter would have matched nothing.

Upgrade

Drop-in for v1.10.0. No index rebuild or configuration change is required, and
existing indexes load and search unchanged. Leftover <index>.gsi.<n>.tmp files
from an interrupted persist are reaped automatically on the next service start or
within an hour of running.

seekfs v1.10.0

seekfs v1.10.0

Full-volume index rebuilds are several times faster and use far less memory,
and regex: queries can now pre-filter candidates by the literal text the
pattern must contain instead of scanning every record.

Highlights

• Much faster, lower-memory builds. A full rebuild of a real
  9.9M/17.1M-record volume dropped from 7m19s/13m06s to about 1m08s/1m54s, and
  peak memory from ~3.9/8.5 GiB to ~2.3/4.4 GiB. The rank builders
  (name/size/modified/extension/type/path) now compute their sort keys once
  instead of re-parsing records and re-lowercasing whole paths inside every
  comparison; the path rank uses an allocation-free memo. The name-trigram
  builder merges a whole gram at a time and groups a spill buffer in an
  open-addressed table, and the record table deduplicates names by their source
  token id instead of hashing every name string.

• Bounded-memory builds by default. Real volumes build their name trigrams
  with the external merge builder, whose peak memory is flat instead of growing
  with the index. Set SEEKFS_NAME_GRAM_EXTERNAL=0 to force the in-memory
  builder.

• Bounded regex pre-filter. regex: queries are pre-filtered by the
  literal runs that every match must contain, expanded as a disjunction of
  alternatives, so a pattern such as ^.*\.(go|rs|py|js)$ bounds-scans a
  candidate subset instead of every record. The rewrite only widens the match
  language, so it can never drop a real result.

Notes

• Non-ASCII names are case-folded with the same Unicode lowercasing the query
  side uses, so searches for names with accented or other non-ASCII characters
  match. The previous selective builder folded only ASCII and could miss those.

• The external builder spills to the system temp directory by default; set
  SEEKFS_NAME_GRAM_SPOOL_DIR to place spill files elsewhere. See
  docs/CONFIG.md for the build-tuning variables.

• Regex pre-filter extraction declines patterns it cannot prove sound
  (non-ASCII patterns, backreferences, \K, conditionals), which simply falls
  back to the existing scan.

Validation

• go test ./... and go vet ./... are green.

• Build time and peak measured on the real C: (9.9M records) and F: (17.1M
  records) indexes.

• Name-gram output verified byte-identical between the external and in-memory
  builders, including forced multi-level spills and repeated-gram names; the
  path rank verified equal to the original comparator on real data.

Upgrade

Drop-in for v1.9.0. No index rebuild or configuration change is required.
Existing indexes load and search unchanged. Rebuild (seekfs index-volumes or
index-usn) to pick up the faster builder, the lower peak memory, and the
Unicode case-fold fix.

seekfs v1.9.0

seekfs v1.9.0

File sizes and dates stay current, folder sizes update live, and size sorting
ranks directories by their recursive content — the way Everything maintains
them.

Highlights

• Live file metadata. The USN change journal reports that a file changed
  but carries no size or timestamp, so a created, modified, or renamed file
  used to report size: 0 (and a zero date) until a full index rebuild — and
  the periodic persist folded that zero into the base index. Now the file is
  re-read from disk when the journal reports a change, at most once per file
  per batch, and the overlay carries its real size and modification time. This
  applies to live replay, startup catch-up, and WAL replay, matching how
  Everything re-reads file metadata on a journal event.

• Live folder sizes. When a file is created, modified, moved, or deleted,
  the size delta propagates to its ancestor directories. The deltas are
  published as an immutable snapshot so lock-free queries see a stable view,
  are applied to directory results and size: filters, and are cleared when
  the volume is persisted (the new base already includes them).

• Directory-aware size ranking. sort:size and the scalar size: ranges
  now rank a directory by its recursive subtree total instead of its stored
  0. The offline builder computes directory totals in a read-only pre-pass
  and feeds them to the size rank; the persist path builds the child graph and
  directory totals before the size rank and re-emits SUBS; and the query-side
  rank compares by effective size.

Notes

• A directory created since the last index build reports 0 until it is folded
  into the base (its own size is not persisted yet).
• Because the journal has no size, a file's size updates when its change event
  is applied; a file being written can lag until it is closed, as with
  Everything.

Validation

• Live-validated against the real C: volume: a created 12,345-byte file
  reported 12,345, a rename preserved it, a rewrite to 25,000 reported 25,000,
  and a folder holding a 25,000-byte file plus a 10,000-byte subdirectory
  reported 35,000, with sort:size ordering directories by that total.
• go test ./... and go vet ./... are green.

Upgrade

Drop-in for v1.8.0. Rebuild the indexes (seekfs index-volumes or
index-usn) to populate SUBS and the directory-aware size rank immediately;
an existing v9 index without SUBS loads with directory sizes of 0 and ranks
directories by their stored size until the next persist.

seekfs v1.8.0

seekfs v1.8.0

Directories now report the recursive size of their contents, matching
Everything's folder-size behavior, and size: filters see that aggregate.

Highlights

• Recursive directory sizes. Folder results no longer report 0. The v9
  index gains a SUBS section: one uint64 per record holding the summed size
  of every file in that record's subtree (0 for plain files, which report their
  own size). The offline builder computes it in a single post-order pass over
  the children graph it already walks for SUBT, and the persist path
  recomputes it on every rewrite, so a folder's size refreshes when the index
  is folded.

• size: filters apply to directories. Because directory entries now carry
  their aggregate size, size:>1gb (and the other operators) match a folder by
  its recursive content size instead of always excluding it.

• Backward compatible. SUBS is an additive v9 section; readers that do not
  know it skip it and load the index normally, and an index written before this
  release decodes with directory sizes of 0. No format version bump.

Notes

• The USN change journal carries no file size, so a folder's recursive size
  tracks live, size-bearing changes only at the next persist/rebuild, like file
  sizes already do. Deletes and new zero-byte files between persists can make a
  displayed folder size lag.
• The SUBS section is 8 bytes per record (about 78 MB for 9.8M C: records and
  137 MB for 17M F: records). In low-memory mode it is file-backed through the
  mmap; in resident mode it is loaded into the heap.

Testing

• New TestDirectV9DirectorySubtreeBytes builds a known tree and checks the
  persisted aggregate, the reported directory entry size, and size: filtering.
• go test ./... and go vet ./... are green.

Upgrade

Drop-in for v1.7.2. Existing v9 indexes load with directory sizes of 0 until the
next rebuild or persist; run index-volumes (or delete and rebuild the index)
to populate SUBS immediately.

seekfs v1.7.2

seekfs v1.7.2

The USN replay loop can now be stopped promptly, and a retired replay loop can
no longer overwrite the state of the loop that replaced it.

Highlights

• Cancelable change-journal reads. The replay loop read the USN journal
  through a synchronous FSCTL_READ_USN_JOURNAL that stayed outstanding until
  a filesystem change arrived, so stopping the service or retiring a stalled
  loop on a quiet volume could block for seconds (or until the next change).
  The read now runs on a volume handle opened with FILE_FLAG_OVERLAPPED and
  is canceled with CancelIoEx when the service is stopping or a watchdog
  restart bumps the replay generation.

• Retired replay loops no longer clobber volume state. A replay loop that
  was replaced while its read was in flight could still mark the volume stale
  with its own error, overwriting the reason the stall watchdog recorded. The
  watchdog now retires the loop's generation under the same lock that records
  the rebuild reason, and a retired loop's error path leaves the volume state
  alone. This also removes a race that made the replay-stall tests flaky under
  load.

• Faster shutdown. With a canceled wait the loop returns immediately
  instead of sleeping its idle delay before re-checking the stop signal.

Testing

• New opt-in live test TestUSNOverlappedReaderLive
  (SEEKFS_LIVE_USN_TEST=1, elevated) verifies cancellation returns promptly
  and that a created file's USN record surfaces through the overlapped read.
  On the development machine a created file was observed in ~3.9 ms.
• go test ./... and go vet ./... are green.

Upgrade

Drop-in for v1.7.1. No index rebuild or configuration change is required.

seekfs v1.7.1

seekfs v1.7.1

Freshness: a file change is now picked up and applied as soon as the USN
journal reports it, instead of after a fixed delay.

Fixes

• No more 500 ms replay throttle. The background USN replay loop slept
  500 ms after every batch, including successful ones, so a change arriving
  right after a batch waited up to 500 ms to become visible and a large
  backlog drained at one batch per half-second. The loop now sleeps only when
  a read reports no progress; a productive read re-arms the blocking journal
  wait immediately, so visibility latency is bounded by apply time.

• Bounded in-call drain. One replay pass now holds the volume handle open
  and applies up to eight batches (resuming from the applied checkpoint,
  including the truncated remainder of an oversized batch) before yielding.
  This avoids a CreateFile + journal query per tiny batch on a busy volume
  while still letting the stall watchdog retire the loop between passes.

The change reuses the existing blocking wait read (BytesToWaitFor=1); it does
not alter the persisted index, WAL format, or wire protocol.

Upgrade

Drop-in for v1.7.0. No index rebuild or configuration change is required.

seekfs v1.7.0

seekfs v1.7.0

Client/server groundwork lands and the service stops wedging: a principal-aware
pipe, a read-only remote transport, and a persist that no longer freezes the
whole index. Broad queries get materially faster on the grammar and regex
lanes.

Highlights

• Principal-aware service pipe. The named pipe is now deny-by-default:
  read-only commands (search, info, status, watch-delta) are available
  to any caller, while the privileged index-usn mutation requires an elevated
  caller. The service derives the caller's principal by impersonating the pipe
  client (ImpersonateNamedPipeClient), inspects the token while impersonating,
  and always reverts before doing engine work. Mutation is granted only to
  LocalSystem or a UAC-elevated member of Builtin Administrators; every
  token/group-query failure fails closed to read-only. The pipe is created with
  PIPE_REJECT_REMOTE_CLIENTS, so SMB clients cannot reach the local-only pipe.

• Mode L remote transport (opt-in, disabled by default). A loopback TCP
  listener carries a versioned length-prefixed frame envelope
  ({type,id,v,payload}: hello/request/response/cancel/error) and dispatches
  through the same command handler with a read-only remote principal. Non-loopback
  binds are refused; frames are size-bounded; unknown versions and malformed
  frames close the connection. Every remote reply passes through one
  sanitizeRemoteResponse projection that strips process identity, pipe name,
  memory, DB paths, journal ids, checkpoints, and planner internals while
  preserving results, counts, health, and coarse planner source. Remote
  requests carry connection-scoped cancellation and a server-owned deadline
  clamp, and cannot cancel another client's or the local GUI's query. Enable
  with -remote-addr / the remote_addr config key; empty (default) is off.

• Persist no longer freezes search. The background persist used to hold the
  global index lock for the entire multi-GB fold+stage, blocking every
  info/search for the persist window (measured 8m14s of staleness on
  production volumes). Persist now snapshots the overlay in milliseconds,
  releases the lock so USN replay keeps applying changes, carries the
  post-snapshot overlay delta into the replacement base, and swaps under a brief
  lock. The WAL is rewritten down to the post-snapshot frames so nothing folded
  is kept twice and nothing applied is lost on crash.

• No more duplicate service. A single 500 ms info attempt treated any
  hiccup (such as a persist swap holding a lock) as a dead service and spawned a
  second service over the same pipe and index files; the two then fought over
  the .gsi files (commit failures, lock errors, interleaved WAL frames, CRC
  failures, multi-GB duplicate memory). pollServiceInfoReady now retries slow
  replies and only falls through to spawning after repeated fast connect
  failures.

Search

• Global bounded-scan fallbacks are pre-filtered by ext:/glob-ext: postings
  and a bounded type:dir subtree union, so scans skip non-matching records
  while preserving exact scan order and top-N semantics.
• The global components lane streams candidates and keeps only the top-N in a
  bounded heap (one verification pass, O(limit) memory): a broad regex path
  went from 28s to tens of ms.
• Pure-substring single-literal regexes (regex:.*test.*) on multi-volume
  services are declined from the components lane and served by the bounded-scan
  literal pre-filter. regex:README\.(md|txt)$ dropped from a 2m13s full-volume
  scan to ~0.5s and regex:.*test.* from 28s to ~1.6s, both order-correct.
• Filename globs with a literal run now drive the gram lane instead of always
  declining to a bounded scan (glob:*report*: 21.9s → 88ms; a mid-word
  extension glob: 1.9s → 165ms). Globs with character classes or single-char
  wildcards still decline.
• Short (1–2 rune) companion terms ride the gram-intersection lane as
  verify-only companions, with the fold run before the maxIDs gate so a broad
  driver cannot decline first. -fuzzy short terms get insertion-variant
  rewrite trials, ranked ahead of longer terms' deletion variants.
• Bare path-separator tokens (/, \) are dropped at parse time so they stop
  gating the fast lanes. seekfs agent gained dogfood-driven guidance and
  seekfs --help now points agents at it.

Fixes

• index-usn persists the applied WAL checkpoint on truncated replay batches and
  logs persist windows, so a truncated batch no longer forces a re-replay of
  frames already folded.
• Stale-recovery no longer re-enters its rebuild loop on a locked index; rebuilt
  index memory is released on failure.
• The replay loop backs off instead of re-running a full locked-index rebuild on
  every tick.
• The UI binary embeds a requireAdministrator manifest so it self-elevates on
  launch, and the window icon is generated into the same .rsrc section at the
  resource id Wails loads (fixes the missing title-bar icon).

Notes

• Mode L is a development/proof-of-transport loopback only. Production LAN
  exposure (broker + auth) is future work; the default configuration does not
  open any socket.
• The release artifact remains unsigned. Windows will warn on first run.

Upgrade

Extract over the previous install. The service uses the v9 columnar .gsi
format (GOSRCH09); an index written by an older format is rejected and
rebuilt on first start. No configuration change is required and Mode L stays
off unless -remote-addr / remote_addr is set.

seekfs v1.6.0

seekfs v1.6.0

The service now polices itself: a silently stalled USN replay loop is
detected and restarted in place instead of quietly missing every change on
the volume.

Highlights

• Replay-stall watchdog. The C: volume failure mode was silent: state
  ready, no errors logged, checkpoint frozen while the OS journal reported
  data past it — every file change after the freeze was missed. A watchdog
  now checks each volume every 15s: if the checkpoint is frozen across two
  consecutive observations while the journal's next USN is ahead, the stall
  is acted on.

• Graduated recovery, no reflex rebuilds. Strikes 1-2 restart only the
  replay loop: a generation bump retires the wedged goroutine and a fresh
  loop resumes from the persisted checkpoint — the index is untouched, so
  recovery costs seconds, not a multi-minute rescan. Only after 3 strikes
  does the watchdog escalate to a full rebuild via the stale-recovery path.

• Correctness hardening around recovery. A restarted or rebuilt loop
  retires quietly instead of double-applying (generation guards in both the
  replay and persist loops), an in-flight USN batch read by the retired loop
  is dropped, recovery sets a recovering flag so the watchdog cannot
  interfere with itself, and healthy applies reset strike bookkeeping.

• Replay health surfaced. info JSON now reports per-volume
  last_replay_at, last_replay_error, and last_replay_next; doctor
  and status print per-volume state with the stall/replay-error reason, so
  a stalled volume is visible instead of invisible.

  seekfs doctor
    volume C:: state=ready entries=10060292
    volume F:: state=ready entries=16895463
  

Fixes

• Watch telemetry from v1.5.0 stays honest on C:: the frozen-checkpoint
  stall that kept C: watch events from advancing is now detected within
  minutes and recovered without operator intervention.

• Duplicate persist loops after a watchdog-triggered rebuild are prevented
  (persist generation retirement), matching the replay loop guard.