Skip to content

Releases: ozgurulukir/seek

v0.6.4

Choose a tag to compare

@github-actions github-actions released this 21 Oct 13:12

What's Changed

  • 🧹 extract parser row format string constant by @ozgurulukir in #123
  • 🧪 unit tests for xberg client constructor and health check by @ozgurulukir in #124
  • 🧪 test(builtin): add unit tests for New and newBuiltin constructors by @ozgurulukir in #125
  • 🧹 remove deprecated RRFk constant by @ozgurulukir in #126
  • ⚡ optimize AnalyzeToken slice pre-allocation and ASCII fast-path by @ozgurulukir in #122
  • 🧪 [add unit tests for ConversationToText and ClaudeConversationToText] by @ozgurulukir in #128
  • 🧪 test ParseCodexFileWithImagesContext in internal/source by @ozgurulukir in #129
  • 🧪 add unit tests for NewWithDependencies and New constructors in indexer by @ozgurulukir in #130
  • ⚡ batch core schema creation in Store initCoreSchema by @ozgurulukir in #131
  • ⚡ batch missing chunk embedding queries in incremental vector index sync by @ozgurulukir in #132

Full Changelog: v0.6.3...v0.6.4

v0.6.3

Choose a tag to compare

@github-actions github-actions released this 18 Oct 08:59

What's Changed

  • ⚡ Quicksilver: pre-trim lines in chunk AssignLineNumbers by @ozgurulukir in #118
  • ⚡ optimize query tokenization and analyzer slice allocations by @ozgurulukir in #120

Full Changelog: v0.6.2...v0.6.3

v0.6.2

Choose a tag to compare

@github-actions github-actions released this 15 Oct 07:04

What's Changed

  • ⚡ [perf] optimize code chunker line scanning to reduce allocations by @ozgurulukir in #99
  • ⚡ [performance] eliminate allocations in agent hook command matching by @ozgurulukir in #97
  • ⚡ optimize line number assignment allocations in chunking by @ozgurulukir in #100
  • 🧪 test: add unit tests for internal/indexer/semantic_tags.go by @ozgurulukir in #101
  • ⚡ pre-allocate slice capacities in doctor fixPermissions by @ozgurulukir in #102
  • 🧪 add unit tests for internal/config/redact.go by @ozgurulukir in #104
  • 🧪 add unit tests for internal/agenthooks/target.go by @ozgurulukir in #105
  • ⚡ Pre-allocate roots slice capacity in layering test by @ozgurulukir in #106
  • 🧪 add unit tests for internal/jsonl scanner by @ozgurulukir in #107
  • 🧪 Add unit tests for internal/indexer/handlers.go by @ozgurulukir in #111
  • 🧪 add unit test suite for internal/search/content.go by @ozgurulukir in #112
  • 🧪 add unit tests for internal/indexer IndexWriter interface by @ozgurulukir in #113
  • 🧹 reduce deep nesting in collection reindex error handling by @ozgurulukir in #108
  • 🧪 add test file for internal/app/path.go by @ozgurulukir in #109
  • ⚡ Batch chunk embedding lookups in UpdateChunkEmbeddingsBatchContext by @ozgurulukir in #110
  • ⚡ replace custom string concatenation loop with strings.Join by @ozgurulukir in #103
  • ⚡ optimize splitMembershipTokens to avoid strings.Split allocations by @ozgurulukir in #115
  • ⚡ optimize code chunking allocations and speed in internal/chunk by @ozgurulukir in #114
  • ⚡ [perf] Eliminate strings.Split allocations in AssignLineNumbers by @ozgurulukir in #117

Full Changelog: v0.6.1...v0.6.2

seek v0.6.1

Choose a tag to compare

@ozgurulukir ozgurulukir released this 09 Oct 08:47

seek v0.6.1

Highlights

  • Keyword search no longer silently drops common English words. In the default (parsed) query mode, body, city, study, and their inflections returned nothing because the query analyzer turned their Porter stem (bodi, citi, studi) into a stem* FTS5 prefix. The index is unstemmed, so a stem that is not a prefix of the surface word can never match — such terms are now left as their surface form. The failure was visible in keyword-first (--lex, no embedding provider) mode; hybrid search masked it behind the vector leg. (issue #98)
  • seek sync --json and seek_capabilities now agree with reality. A run that embedded (or deliberately skipped) chunks is no longer reported as embeddings: "not_run", and a provider the offline/privacy policy refuses to reach is reported as unavailable.
  • The local semantic service starts dramatically faster. Importing sentence_transformers was JIT-compiling numba dispatchers for 10+ minutes; disabling numba's JIT (which the tagger never uses) drops startup to ~38 seconds. (issue #95)

Fixed

  • Non-prefix Porter stems (issue #98). AnalyzeForQuery now only emits a stem* expansion when the stem is a genuine prefix of the surface token (or shorter, which preserves Turkish root reconstruction such as kitabı→kitap); otherwise it falls back to the surface token. Covered by analyzer and ToFTS5 unit tests plus an end-to-end parsed-mode regression test that indexes "body of the study and the city" and searches body/study/city.
  • seek sync embedding outcome. The embedding stage is classified before the per-file-failure check, so the embedding stage still runs when the indexer recorded individual file failures and seek sync --json no longer reports embeddings: "not_run" for a run that embedded or deliberately skipped chunks.
  • seek_capabilities. A configured client whose endpoint the privacy.offline_only policy refuses to reach is treated as unavailable, matching the sync report.
  • seek sync <collection> --path. Restored the %w error chain, stopped printing the "validated inside collection" summary when the path guard rejected the run, and kept the empty-index hint off filtered invocations.
  • MCP seek_search. Tolerates empty-string list arguments (collections, fields, …) emitted by some clients instead of failing the call, and honors sort_by/sort_order — _score maps to relevance order and a non-descending sort_order with _score is rejected.
  • Semantic service startup (issue #95). NUMBA_DISABLE_JIT=1 is exported (via setdefault, so a user-set value wins) before importing the service, avoiding the JIT warm-up stall. The tagger uses PCA rather than UMAP at runtime, so no hot path regresses; tools/ and the bundled skill mirror are updated in sync.

Performance

  • Agent hook target matching (internal/agenthooks) memoizes the compiled target-hook regexes instead of rebuilding them on every invocation, with a benchmark guarding the cache.

Validation

  • Full test suite, go vet, and gofmt verified with CGO and FTS5 tags (-tags "fts5 sqlite_fts5"); the release gate re-runs them in CI before publishing.
  • New coverage includes MCP argument-parsing tests, sync-outcome tests, a schema-driven parser availability test, and the issue #98 analyzer/end-to-end regression tests.

Upgrade notes

  • No migration or configuration change is required.
  • If NUMBA_DISABLE_JIT is already set in your environment it is respected; otherwise the semantic service now exports it for a much faster startup.

What's Changed

  • fix: disable numba JIT to avoid 10+min semantic service import by @ozgurulukir in #95
  • ⚡ memoize agent hook target regex compilation by @ozgurulukir in #96

Full Changelog: v0.6.0...v0.6.1

seek v0.6.0

Choose a tag to compare

@ozgurulukir ozgurulukir released this 05 Oct 20:20

seek v0.6.0

Highlights

  • Documented config keys now do what they say. A wiring audit found five config keys that were parsed from config.yaml but never consumed — search.default_limit, filters.enabled, filters.default_collection, aggregations.enabled, and rerank.top_n — and wires them into the shared search planner (so CLI and MCP behave identically) and the rerank client.
  • Scanned PDFs no longer fail silently. Per-page OCR failures, embedded-text extraction failures, and textless pages without OCR enabled now surface as counted WARN lines in the sync summary, instead of indexing image-only pages with zero text and no signal.
  • The built-in xberg endpoint default pointed at the wrong service. DefaultXbergBaseURL targeted port 8000 — that's flashrank_server's port; the xberg server binds 8001. With extractor.backend: xberg and default config, extract requests went to the reranker.

Added

  • search.default_limit resolves the result limit when neither seek search -l nor the MCP seek_search limit argument specifies one (default 10, as before).
  • filters.enabled + filters.default_collection apply a default collection filter when a request names no collection/repo. An unknown collection name errors with seek collection list guidance instead of silently blinding every search to zero results.
  • aggregations.enabled: false now gates seek search --aggs and the MCP aggs argument with an actionable error (default true — no change unless opted out).
  • rerank.top_n caps how many candidate hits reach the cross-encoder endpoint. Truncation is a prefix cut, so reranked result indices stay aligned with the caller's candidate slice (pinned by test).
  • Extraction warnings (extractor.Result.Warnings) are logged per document by both indexer extraction call sites and count toward the sync report's warning total.

Changed

  • vector_index.hnsw.dimension removed. The HNSW index dimension always followed embedding.dimensions; the key was a silent no-op that implied an override existed. It (and its default constant) is gone. A leftover key in an existing config is simply ignored.
  • rerank.top_n is uncapped when unset. The config loader no longer forces a default of 10 — 0 means rerank the full candidate pool. Users who set top_n while it was dormant will now actually get the cap; seek auth status prints top_n: auto when uncapped.

Fixed

  • xberg default endpoint port: http://127.0.0.1:8000 → http://127.0.0.1:8001 to match tools/xberg_server/server.py.

Documentation

  • Fact-check pass across README, AGENTS.md, docs/, and the bundled skill: Go version (1.25) and LOC counts, the background service description (it schedules a single seek sync; embedding runs in-process), seek status/seek rm described as standalone commands rather than aliases, the missing hermes parser schema row, codex's archived_sessions path, lz4 in the compression algorithm list, and a stale -l reference in the analyze-lang section of docs/local-setup.md. plugin.json now tracks the release version.

Tests

  • New planner tests pin the limit resolution chain, filters.default_collection (applied / overridden / disabled), and the aggregations gate against a real temp SQLite store.
  • TestRerankTopNCapsCandidatePool drives the provider factory through an httptest endpoint and asserts the candidate cap plus index alignment.
  • Golden help snapshots regenerated for the reworked -l flag help.

Validation

  • Full test suite, go vet, and gofmt verified with CGO and FTS5 tags (-tags "fts5 sqlite_fts5"); the release gate re-runs them in CI before publishing.
  • The wiring change passed a two-agent review (behavior review plus a silent-failure audit); all findings from that review are included in this release.

Upgrade notes

  • No migration is required. Defaults are unchanged: default limit stays 10, filters and aggregations are enabled, top_n unset behaves as before.
  • If you previously set rerank.top_n or filters.default_collection while they were inert, they are active now — double-check the values against what you intend.
  • Remove vector_index.hnsw.dimension from config.yaml at your leisure; it is ignored.

Full Changelog: v0.5.12...v0.6.0

seek v0.5.12

Choose a tag to compare

@ozgurulukir ozgurulukir released this 30 Sep 17:02

seek v0.5.12

Highlights

  • Safer macOS service install: seek service start verifies the launchd plist was written and flushed successfully before registering it with launchctl, so a disk-level failure can no longer silently install a broken service definition.
  • Architecture is now enforced, not just documented: a package layering guard runs on every go test and fails on import cycles, upward imports, or third_party reaching back into the main module.
  • Skill service scripts are a generated artifact: the skills/seek/scripts/services/ mirror of the optional Python services is guarded by an SSOT test — drift, hand-edits, and unlisted files fail go test with a remediation hint instead of requiring every fix to be applied twice.

Fixed

  • launchd plist close (cmd/service.go): startDarwinService now checks the error from closing the plist file before launchctl bootstrap; previously a failed flush truncated the plist silently. Also drops an unused launchctl print output variable in the status command.
  • Line endings for mirrored service scripts: tools/{semantic,xberg_server,flashrank_server,embed_server} and skills/seek/scripts/services/ are pinned eol=lf in .gitattributes (the same convention as the golden help files). On core.autocrlf=true Windows checkouts this prevents git from rewriting setup.sh to CRLF — a latent breakage — and keeps the byte-based bundle comparison stable.

Refactoring

  • syncConversation (internal/indexer/indexer.go): the ~200-line shared Claude/Codex conversation sync path is decomposed into conversationTitle, buildConversationChunks, pruneEmptyConversationBatch, conversationBaseSeq, and writeConversationDocument. Behavior is preserved exactly: same WARN messages, same failure counters, same error returns, same sequence numbering and line-span offsets. The append base-seq error path and the empty-batch pruning contract are pinned by new unit tests against a real temp SQLite store.

Tests

  • layering_test.go: import-graph guard via go list -json — cycles, layer directions (root → cmd → internal), renameio-leaf rule; includes a non-vacuous guard against go list returning zero packages.
  • TestSkillServiceBundleIsSynchronized: parses the pair list from scripts/sync-skill-services.py (no duplicated list), byte-compares all pairs, walks both trees for orphans and undeclared directories; verified to catch stale copies, mirror hand-edits, and unlisted canonical files.
  • conversation_sync_test.go: title precedence (parser title / base name / session lookup), chunk seq continuation and fromLine line-span offsets, images-after-text sequencing, empty-batch prune (delete vs mtime-only vs no-op), append base-seq against a seeded store.

Validation

  • Full test suite, go vet, and gofmt verified with CGO and FTS5 tags (-tags "fts5 sqlite_fts5"); the release gate re-runs them in CI before publishing.
  • All changes passed a two-agent review (behavior-parity review plus a silent-failure audit) before landing.

Upgrade notes

  • Patch release: drop-in binary replacement. No config, index schema, or vector-space changes; no reindex required.
  • After pulling this release on Windows, run git status once — the new eol=lf rules may mark the mirrored service scripts as modified until renormalized (git add --renormalize tools skills); content is unchanged.

Full Changelog: v0.5.11...v0.5.12

v0.5.11

Choose a tag to compare

@ozgurulukir ozgurulukir released this 27 Sep 17:16

seek v0.5.11

Highlights

  • Faster reindex orphan cleanup: stale-document deletion no longer scales as four round trips per document. Deletes are grouped into 500-id IN clauses, and the full-collection purge path (reindex) uses set-based subquery deletes.
  • Faster embedding runs: each embedding API batch is persisted to SQLite in one transaction instead of one transaction per chunk, with the live HNSW/linear vector index updated incrementally for new chunks and rebuilt atomically when embeddings are replaced.

Performance

  • Orphan cleanup (internal/store.DeleteOrphansContext): the per-document loop issued DELETE statements against fast_fields, documents_fts, chunks, and documents one id at a time. Deletes now batch 500 ids per statement; a nil livePaths map (full purge) skips the id scan entirely and deletes by collection_id via subqueries. Partial purges keep live documents and their projections untouched; a repeat purge is a no-op. Covered by TestDeleteOrphansContext and benchmarks.
  • Chunk embedding writes (internal/store, internal/pipeline): text, VL image, batch, and realtime embed paths now collect their batch into store.ChunkEmbeddingUpdate values and persist them with one prepared statement in one transaction. Vector-index handling mirrors the per-chunk contract: new IDs take the cheap incremental Add path; replacing an existing ID triggers an atomic off-to-the-side rebuild of the graph, and if that rebuild fails, previous embeddings are restored to SQLite — with restore failures now reported rather than swallowed.

Refactoring

  • seek auth login: prompts extracted into promptProviderSelection, promptCustomProvider, promptAPIKey, and promptMultimodal; Run is now a readable sequence of prompt calls. Behavior is unchanged (custom providers default to 1024 dimensions either way).

Tests

  • TaskPrefixes: uppercase nomic/e5/bge model names, explicit document-prefix override, empty-model fallback.
  • quoteQualifiedIdentifier: embedded double quotes, multi-part names, empty identifier.
  • Config defaults: all typed default constants (timeouts, permissions, chunking, HNSW params, compression, rerank) sanity-pinned.
  • SchemaRegistry.DefaultSchema: zero-value registry, complete field definition set, call-to-call consistency.

Validation

  • Full test suite, go vet, and gofmt verified with CGO and FTS5 tags (-tags "fts5 sqlite_fts5"); the release gate re-runs them in CI before publishing.
  • Every PR merged in this release passed its own CI (Linux + Windows) plus local targeted tests; squashed with conventional-commit titles.

Upgrade notes

  • Patch release: drop-in binary replacement. No config, index schema, or vector-space changes; no reindex required.
  • Collections rebuilt with seek collection reindex after upgrading get the faster orphan-cleanup path; syncing/embedding gets the batched persistence automatically.

What's Changed

  • ⚡ optimize orphan document batch deletion during reindex by @ozgurulukir in #94
  • 🧹 extract interactive prompts from AuthLoginCmd.Run by @ozgurulukir in #86
  • 🧪 [add edge case tests for SchemaRegistry DefaultSchema] by @ozgurulukir in #84
  • 🧪 [test default configuration constants in internal/config] by @ozgurulukir in #83
  • 🧪 [add test for quoteQualifiedIdentifier] by @ozgurulukir in #82
  • 🧪 [test edge cases for EmbeddingConfig.TaskPrefixes] by @ozgurulukir in #81
  • ⚡ Batch text chunk embedding updates to eliminate N+1 queries by @ozgurulukir in #92

Full Changelog: v0.5.10...v0.5.11

v0.5.10

Choose a tag to compare

@ozgurulukir ozgurulukir released this 26 Sep 13:53

seek v0.5.10

Highlights

  • Second code-review verification pass: every finding from a fresh multi-agent review was re-verified against the current code; the nine that still held are fixed here, each with a regression test. (The rest were confirmed already fixed or intentional.)
  • Optional Python services ship with the skill: the semantic tagger, embedding server, and flashrank reranker are packaged inside the seek skill/plugin with setup scripts, so agents can stand up the optional NLP endpoints without cloning the repo.

Fixes

  • Agent hooks (crash): a hand-edited settings file containing the literal JSON null returned a nil map from ReadSettings, panicking hook install/repair on first assignment. It is now treated as an empty writable settings map (missing-file contract); invalid JSON still errors with the file name.
  • Windows service (quoting/injection): the Task Scheduler task was registered as cmd.exe /c ""<bin>" sync" with the binary path interpolated unescaped — %VAR% expands even inside cmd.exe double quotes and an embedded quote breaks the quoting. The task action is now a plainly quoted argv ("<bin>" sync) launched via CreateProcess.
  • Semantic enrichment (churn): a transient semantic-service /health failure used to degrade the enrichment fingerprint basis to an empty capability set. That basis mismatches every persisted fingerprint, re-selecting the whole collection for re-enrichment — twice (this pass and the next healthy one) with a half-up service, and the degraded basis was memoized for the process lifetime. A health failure is now an error: backfill warns and no-ops the pass, the sync path records nothing (documents keep their previous fingerprint) and warns once per process/provider.
  • Vector index (cancellation safety): cancelling an embedding-space reset (Ctrl-C) also cancelled the vector-index restore, leaving the live HNSW index empty for the rest of the process while SQLite still held the embeddings — an empty graph a normal Close would then persist. The restore now runs with a cancellation-proof context.
  • Indexer (race hardening): WithLogger is called per operation (sync, backfill) on the shared Runtime-owned Indexer; the logger swap is now guarded by the instance mutex like every other shared field.
  • xberg extractor (robustness): extraction responses were read with an unbounded io.ReadAll; a broken or hostile server could OOM the sync. Response bodies are now capped at 64 MiB with an explicit over-cap error.
  • MCP (version reporting): the MCP server hardcoded version "dev", so release builds misreported themselves. Version/commit now live in internal/buildinfo, injected via ldflags, and are reported by both seek --version and the MCP server identity.
  • Search (actionable errors): field-scoped query terms are validated against the FTS columns (title, content) at parse time. Unknown fields previously failed at query time as an opaque FTS5 no such column — or, in hybrid mode, silently killed the BM25 leg while vector results kept flowing. They now fail fast with unknown field "x" (supported: title, content).
  • Docs: corrected the stale "SyncVectorIndex clears the index" description in AGENTS.md (full sync builds off to the side and atomically publishes).

Validation

  • Full test suite, go vet, and gofmt verified with CGO and FTS5 tags (-tags "fts5 sqlite_fts5"); release gate re-runs them in CI before publishing.
  • Regression tests added: JSON-null settings, Windows task action rendering, semantic health-failure no-op contract, bm25 title-weight ranking, xberg response cap, unknown-field sentinel propagation.

Upgrade notes

  • Windows seek service users: re-run seek service stop && seek service start once to re-register the task with the safer action string. The previously registered cmd.exe-based task keeps working; it is just deprecated.
  • Search queries: query strings using unsupported field:value prefixes (e.g. tags:go) now fail with a parse error instead of a runtime FTS5 error. Use seek search --field tags:go for fast-field filtering — behaviour of title:/content: is unchanged.
  • Patch release: drop-in binary replacement. No index schema migrations or vector space changes required.

Full Changelog: v0.5.9...v0.5.10

v0.5.9

Choose a tag to compare

@github-actions github-actions released this 25 Sep 14:58

What's Changed

  • fix: high-impact findings from multi-agent code review by @ozgurulukir in #79

Full Changelog: v0.5.8...v0.5.9

v0.5.8

Choose a tag to compare

@github-actions github-actions released this 18 Sep 07:14

seek v0.5.8

Highlights

  • ZCode conversation support: a new embedded zcode parser schema indexes
    ZCode model-I/O rollout files (~/.zcode/cli/rollout/*.jsonl) — add it with
    seek add --parser zcode.
  • The JSONL parser driver gained a sliding-window mode (messages_path +
    offset_field): rollout files carry a cumulative conversation window per
    request line, and the driver dedups messages by global index so every turn
    is indexed exactly once. Each line's response.text becomes the assistant
    turn.
  • ZCode sessions can sync automatically via a manually configured Stop hook
    in ~/.zcode/cli/config.json (see docs/service.md); no registry change or
    seek hooks install involvement.
  • Documentation updated across the parser/service guides, README, and the
    bundled seek skill.

Fixes

  • CI: the Windows test job installs Zig from the renamed 0.16.0 archive
    (zig-x86_64-windows-*.zip); the previous URL 404'd and blocked the job.
  • Test: the relative sync-path test skips when the temp dir and CWD live on
    different Windows drives instead of failing on filepath.Rel.

Validation

  • Full FTS5-tagged Go test suite, go vet, and gofmt are green on Windows
    (Zig CGO) and in CI on both ubuntu-latest and windows-latest.
  • End-to-end: seek add --parser zcode detected 3 rollout files, embedded
    615 chunks through the local Ollama provider, and a hybrid search over the
    collection returned this session's own transcript with correct user and
    assistant turn attribution.
  • Independent code-reviewer and silent-failure-hunter passes ran on the
    parser change; findings were fixed or explicitly documented (a few turns
    around a partial history compaction may be emitted twice — negligible for
    search scoring).

Upgrade notes

Patch release; standard installer. To index ZCode history:

seek add --parser zcode

and optionally install the manual Stop hook from docs/service.md.
Full Changelog: v0.5.7...v0.5.8