Skip to content

Releases

Release 2.2.0

Choose a tag to compare

@github-actions github-actions released this 03 Nov 08:56
adef5da

Changed

  • The default .rogoignore that the first --index writes now also lists
    dependency directories: node_modules/, bower_components/,
    jspm_packages/, .venv/, venv/, site-packages/, __pycache__/,
    .tox/ and .nox/. An existing .rogoignore is never rewritten; add the
    lines to it by hand.
  • The chunker's token estimate counts digits, punctuation, whitespace runs
    and line breaks the way the tokenizer spends tokens on them, instead of a
    flat ~4.8 characters per token for Latin text. Tables, version lists,
    changelogs and data files tokenized at ~1-2 characters per token, so 11%
    of a notes vault's chunks and 3-6% of a node_modules tree and a
    virtualenv passed the model's 2,048-token context, and embed truncated
    them (the truncated N of M inputs warning). On the same corpora no chunk
    of the vault passes it now, and 5 of ~207,000 in the two dependency trees
    still do (tables of rare Unicode letters). Chunks get smaller in real
    tokens (the vault's average went from ~1,290 to ~860, about 1.5 times as
    many chunks for about the same tokens embedded).
  • CHUNKER_VERSION is 3, so the next --index of an existing corpus
    rebuilds it once.

Release 2.1.0

Choose a tag to compare

@github-actions github-actions released this 19 Oct 23:49
ad283da

ckq is now rogo (Latin "I ask"). Same engine, same index format, new
name everywhere: the package and binary, the on-disk names, the environment
variables, the URLs (bitboxx/rogo; the old ones redirect).

Distribution

  • On crates.io as rogo (with the helper crate rogo-gpu):
    cargo install --locked rogo.
  • A Homebrew tap over the release binaries: brew install bitboxx/tap/rogo.
  • Every release archive carries signed build provenance;
    gh attestation verify <archive> --repo bitboxx/rogo checks it.

Renamed

  • The package and binary ckq → rogo, the helper crate ckq-gpu → rogo-gpu.
  • The index directory .ckq → .rogo, the ignore file .ckqignore →
    .rogoignore (flag --no-rogoignore), the cache ~/.cache/ckq →
    ~/.cache/rogo (models and daemon sockets, rogo-<alias>-…), the model
    check marker <file>.ckq-verified → <file>.rogo-verified.
  • Every environment variable CKQ_* → ROGO_* (including the build-time
    ROGO_EMBED_GGUF, ROGO_EMBED_ALIAS).
  • Release archives are rogo-<version>-<target>.

Migration

  1. mv .ckq .rogo in every directory you indexed.
  2. mv .ckqignore .rogoignore wherever you have one.
  3. mv ~/.cache/ckq ~/.cache/rogo.
  4. Rename any CKQ_* variable you set to ROGO_* (ROGO_IN_PROCESS,
    ROGO_DAEMON_IDLE_SECS, ROGO_DAEMON_TIMEOUT_SECS,
    ROGO_QUERY_DEVICE, ROGO_GGUF_FILE, ROGO_TIMING).

An index moved this way needs no rebuild: the index signature does not
contain the name, and nothing else in the store ever did.

Legacy handling

A command that finds no .rogo index but a .ckq/index.db fails with one
message naming the mv above; it never moves or writes anything itself.
Until .ckqignore is renamed, rogo keeps reading it — so results do not
change silently — and warns once on stderr; --index writes a default
.rogoignore only when neither file exists. A leftover .ckq directory is
skipped by the walker and left alone by --clean, exactly like a legacy
.ck. A model download that starts while ~/.cache/ckq/models still
exists says once which mv would have skipped it.

Release 2.0.0

Choose a tag to compare

@github-actions github-actions released this 18 Oct 15:38
3c6d1f3

The rewrite, as one crate: a single ckq crate at the repository root replaces
the eight-crate ck workspace. Ten modules, one binary, the same CLI surface
(--sem, --lex, --hybrid, regex mode, --json/--jsonl).

Added

  • One store: .ckq/index.db. SQLite with WAL, a contentless FTS5 table
    for BM25 over file text and path, and f16 vectors in the same database. It
    replaces the old manifest plus per-file sidecars plus vectors.cache plus
    the tantivy index, one file to copy, back up or delete.
  • A query daemon. --sem and --hybrid embed their query through a
    resident daemon (unix) instead of loading the model in every search process.
    The first search starts it (the same executable, hidden --query-daemon
    flag), it binds its socket before it loads the model, so two clients
    starting at once load it once, and it exits after CKQ_DAEMON_IDLE_SECS
    without a request (default 900) removing its socket. The socket is
    ~/.cache/ckq/sockets/ckq-<alias>-<16 hex of blake3 over the binary's path, size, mtime>.sock, so a rebuilt binary never talks to a daemon running the
    code it replaced. The client sends the bare query text and the daemon calls
    embed_query, so the query prompt is still applied in exactly one place;
    the daemon's stderr goes to a log beside its socket. Indexing never uses
    the daemon, CKQ_IN_PROCESS=1 skips it, and an unreachable or failing
    daemon falls back to in-process embedding with one warning. A warm
    --hybrid search over a 1,200-file corpus takes about 55 ms, against about
    0.2 s before (loading the model costs about 0.4 s per process otherwise).
  • Hybrid search, fused per file by CombMAX over weighted reciprocal ranks.
    The lexical half is the same BM25 that --lex runs (the path scored at
    half weight), the two rankings are fused into one result per file, and a
    file keeps the better of its two weight / (3 + rank) scores while the
    other method adds 0.01 of itself to break near-ties. The constant is 3: at
    60 the top of each list was so flat that a file both methods ranked fifth
    outscores one a method ranks first. The lexical weight is 1.05 with
    --term and 0.15 without: a question is not the words its answer contains,
    and at full weight gemma-q4 dropped from 35 to 15 of 40 answers first on
    bench/multilingual. Summed RRF was measured against CombMAX on a
    48-question evaluation and lost at every k (3 to 60) and every lexical
    weight (0.4 to 2.0): 34 to 36 answers first against 37.
  • --term, repeatable, gives the lexical method of --lex and --hybrid
    its own keywords in place of the query, each matched as a phrase. In
    --hybrid the query still drives the semantic method, so a question and
    the words a matching file would contain can be given separately.
  • --json and --jsonl report where each method ranked a hybrid result,
    as signals.lex_rank and signals.vec_rank. A file both methods found is
    corroborated; when none is, the search may have missed.
  • Concurrent first --index runs succeed. The journal-mode change in the
    schema batch is retried (it ignores busy_timeout), writes use IMMEDIATE
    transactions, and a reader requires a populated meta before it trusts the
    schema. 15/15 concurrent pairs succeed.
  • The query device defaults to GPU by measurement: median of 10 cold
    --sem runs on bench/multilingual (Apple M1 Ultra), 0.41 s on GPU
    against 0.59 s on CPU. CKQ_QUERY_DEVICE overrides it.
  • Vulkan by default on Linux and Windows (macOS stays on Metal). The GPU
    backend is a cargo default away from macOS, forwarded to llama-cpp-2
    through the tiny gpu/ path crate because cargo cannot make a default
    feature platform-specific. The binary then needs a Vulkan loader at runtime
    (libvulkan1 on Linux, which a GPU driver brings; vulkan-1.dll on
    Windows) and falls back to the CPU when a loader exists but no device does.
    --no-default-features builds CPU-only,
    --no-default-features --features cuda builds CUDA alone.
  • A one-file build: embedded-model. The optional cargo feature builds
    one GGUF into the binary (build.rs reads CKQ_EMBED_GGUF and
    CKQ_EMBED_ALIAS, validates the alias against the registry and hashes the
    bytes; scripts/build-onefile.sh <alias> wraps it). The embedded alias
    becomes that build's default model, --status and --version name it, and
    the first use extracts it once to ~/.cache/ckq/models/embedded/, so the
    single binary works offline on a fresh machine.
  • --hidden includes dot-files and dot-directories in search and
    indexing; --no-ignore and --no-ckqignore turn each ignore file off.
  • CKQ_TIMING=1 prints one stderr line of where an index run spent its
    time: walking, reading and chunking, embedding.

Changed

  • A document is its root-relative path (RelPath), everywhere. Both
    ranked methods return it, fusion compares it, output prints it; the working
    directory never matters. The old engine's four disagreeing path forms are
    what broke hybrid fusion and cwd-independence; both hold by design now.
  • The on-disk names are ckq's own. The index directory is .ckq (was
    .ck), the ignore file .ckqignore (was .ckignore, flag
    --no-ckqignore), and the cache root ~/.cache/ckq for models and daemon
    sockets. No migration: the walker skips a legacy .ck so it is never
    indexed, a search that finds only a legacy .ck/index.db says the index
    moved to .ckq and names ckq --index, and --clean removes .ckq only,
    reporting a legacy .ck left in place.
  • Search never writes the index. Only --index, --add and --clean
    do. A ranked search warns when files changed since the last index; after
    --clean it refuses and names --index instead of silently rebuilding.
  • The default model is gemma-q4 (EmbeddingGemma 300M, QAT Q4_0),
    queried with the task prompt its model card specifies. Granite was tried
    as the default on the way and stays in the registry under granite-gguf.
  • --index with no --model uses the index's own model, not the
    registry default, so a plain re-index after editing a few files works. An
    explicit --model wins, and a different model than the index holds means a
    full rebuild, in the same run.
  • --index loads the model in the process that runs it (the daemon is
    for queries) and skips a file whose size and mtime match the index without
    reading or hashing it, so keeping a corpus current stays cheap.
  • The embedder owns the model on one dedicated thread
    (LlamaContext borrows the model and is !Send), applies the model's
    prompts in one place, and tears down in the order ggml requires (context
    before model, plus an atexit hook), without which ggml aborts at exit.

Removed

  • The eight crates (ck-core, ck-models, ck-embed, ck-chunk,
    ck-index, ck-engine, ck-tui, ck-cli) and everything only they
    needed: the old embedding daemon, the ONNX, fastembed and mixedbread
    backends (GGUF through llama.cpp only), the reranker (--rerank-model),
    tree-sitter chunker queries, the TUI, the MCP server, the VS Code
    extension, the docs site, npm packaging, PDF extraction, --reindex,
    --switch-model and CK_INDEX_DIR. Switching models is
    ckq --index --model ALIAS.
  • crates.io and npm publishing: Cargo.toml is publish = false and a
    release is a tag that builds GitHub release binaries.

Fixed

  • ckq --index /dir indexed the current directory. A command flag takes
    no query, so a lone positional argument is its path; clap filed it under
    pattern, which made the command index ..
  • Audit, scope and paths: a search scoped to the index root canonicalized
    every indexed path
    , twice, about 12 ms on a 1,100-file index; both passes
    now see that the root covers everything. Output paths print with forward
    slashes on every platform: on Windows text output said .\a.md and
    --jsonl carried backslash-escaped paths.
  • Audit, daemon timeout and weights checks. The client times out a
    request (120 s, CKQ_DAEMON_TIMEOUT_SECS) and falls back to in-process
    embedding on any failure, so a daemon that accepted and went silent never
    hangs a search. The socket key and the index signature both track
    CKQ_GGUF_FILE (path, size, mtime), so a daemon holding one set of
    weights cannot answer queries for a file that replaced them, and a search
    refuses instead of mixing two models' vector spaces.
  • Audit, model hash checks. Where the registry pins a blake3, the
    download is verified against it in one streaming pass, and a corrupted or
    rewritten file is an error naming the file. A passed pass leaves a marker
    beside the file holding size, mtime and hash, so hashing 278 MB (0.2 s) is
    not paid on every daemon start and index run; writing the file moves its
    mtime, so a rewrite is hashed again.
  • Audit, chunking of over-long lines. A line longer than the model's
    context used to become one chunk over the limit and lose everything past it
    at embed time (minified JavaScript, an unwrapped pasted paragraph). Such a
    line is now split into fitting pieces, estimated at no more optimistic than
    2.5 characters per token, the rate minified code actually hits; the token
    estimate no longer undercounts emoji or the Arabic, Hebrew and Indic
    scripts; a markdown fence closes only with its own marker and length; and a
    lone CR counts as a line ending, so chunk spans, printed line numbers and
    grep agree. CHUNKER_VERSION enters the index signature at 2.
  • A model-switch rebuild failed on Windows with a sharing violation: the
    read connection was still open when clean removed .ckq. The connection
    closes first; unix is unchanged.
  • An empty index no longer fails the second writer. open_read treats an
    index.db with no schema yet, what another process's first still-running
    --index looks like, as no index, instead of ...
Read more