Skip to content

v0.2.3

Choose a tag to compare

@github-actions github-actions released this 17 Sep 23:11
· 214 commits to main since this release
dd6c68b

⚠️ Upgrading

  • Every PDF re-extracts and re-chunks itself on the next ordinary refresh, because the
    extractor version is part of the content fingerprint. Nothing to run by hand.
  • /healthz gains a missingModels array. Additive, and a client that ignores it is
    unaffected; the UI reads it defensively, because an older container returns a response
    without the field.

Fixed

  • Files no source covered were invisible rather than reported. A file outside every
    source root is not skipped and not failed: it has no row in any count, and a search for
    it returns other documents, which is indistinguishable from a ranking result. A library
    of 138 files indexed 95 of them; one book sat directly in orly/ while every source was
    orly/<topic>/, and a keyword search on its exact title returned four other books.

    index_status now reports a directory when two or more of the corpus's sources share it
    as their parent, which is where the children were enumerated deliberately and a file left
    loose among them was passed over. One source under a directory says nothing about that
    directory and is not reported, so a corpus that indexes a single folder stays silent.
    Only files that would have been indexed are listed: the always-exclude list, a
    .gitignore in the directory, the size caps and binary sniffing all still apply.

  • PDFs were read in content-stream order, not in reading order. A producer writes the
    content stream in whatever order it likes, which on a two-column page is often left line,
    right line, left line. Read that way the columns interleave into text that is grammatical
    nonsense but looks like prose, so nothing downstream detects it: it embeds, it chunks and
    it comes back as a search hit. A table is worse, every cell being its own column.

    The quieter half affected every PDF. Content order ends a line where the text met the
    right margin, so a paragraph arrived as ten fragments. Over a 424-page book, against the
    EPUB of the same title: 14,954 lines averaging 43 characters became 5,559 averaging 118,
    where the EPUB gives 5,727 averaging 109. Character counts barely move, so this is not
    about gaining or losing text. It costs 1.2x the extraction time, 5.8 ms a page.

    Measured and not claimed: this does not improve chunk parity between a title's PDF
    and its EPUB. Parity was already close and is marginally worse at the smallest chunk
    size, because chunks are budgeted in characters and the character count is what did not
    change.

  • get_context returned the chunks holding the lines, not the lines. Asking for three
    lines of context returned forty: the stitcher emitted every chunk that overlapped the
    window rather than trimming to it, so a request was answered with whatever the chunk
    boundaries happened to be. (#12)

  • The audit trail could be forged by an unauthenticated caller. A request path is
    decoded before it is logged, so %0A arrived as a real newline, and the console output
    template rendered string values literally. A request to
    /x%0A[19:05:31Z INF] DELETE /api/corpora/books -> 200 (token bootstrap) wrote that
    second line into the log as an entry of its own. No valid token was needed: the
    rejection is logged before the 401 is returned. docs/07 names the container logs as
    the audit trail, so the trail itself was forgeable.

    Fixed in the output template rather than at the call sites, which cannot be forgotten
    by a later log statement: {Message:j} instead of {Message:lj}, dropping the
    "literal" flag so Serilog renders string values as JSON and escapes the newline. String
    values in the log are now quoted.

  • The secret scan covered one push, not the history it claimed. The job was named and
    commented for full history; the action scans the commits in the push it runs on, so
    fetch-depth: 0 fetched a history nothing then read. It also fired on
    SecretHygieneTests.cs, whose fabricated token is token-shaped on purpose: the test
    asserts a principal cache key never contains the credential it came from. Allowlisted as
    that exact literal rather than by path, because a path in the global allowlist is exempt
    from every rule, which would stop the file being scanned for AWS keys and private keys
    too. (#8)

  • A UI assertion passed only on an English runner. Documents.test.tsx asserted on the
    literal 2,940 chunks beside a .replace(',', ','): a comma replaced by a comma.
    The component renders the count with toLocaleString(), whose thousands separator is
    2.940 under de-DE and 2 940 under fr-FR, so the no-op left an assertion that
    fails anywhere but an English locale. It now formats the expected value the same way the
    component does. (#11)

  • Every button in the app drew the wrong cursor. Tailwind v3's preflight set
    cursor: pointer on buttons; v4 dropped the rule without replacing it, and the component
    library does not put one back. Three call sites had hand-added it, which is how a fix ends
    up covering only the controls someone happened to look at.

  • Every badge failed text contrast, in both themes. A token tuned for a 12% fill and a
    40% border is the wrong value for the words sitting on that fill: --warn as text
    measured 2.95:1 on the light surface against the 4.5:1 needed at 12px, --ok 3.31:1, and
    dark ran 3.63:1 to 4.24:1. The palette is now two values per meaning, one for fills and
    borders and one that can carry text. Fills and borders are byte-identical; only what is
    read moved.

  • color-scheme followed the operating system rather than the chosen theme, so an OS on
    light with the app on dark gave pale scrollbars down every code block. The tokens colour
    only what this stylesheet draws; scrollbars, the caret, the reveal button in the token
    field and autofill are the browser's, and it picks those from color-scheme.

  • prefers-reduced-motion was read nowhere while the app animated. It now collapses
    durations rather than setting animation: none, because a data-[state=closed] exit
    animation that never runs leaves the element on screen, and a dismissed dialog would
    simply stay.

  • The one line saying results could not be trusted read as decoration. "Corpus 'books'
    is still indexing; results are incomplete." was drawn in the accent blue, the same blue
    that badges the default chunk set and the corpus a hit came from. It is a warning, and it
    is now announced as one.

  • A book title pushed the page past the viewport. A result header's min-content is a
    whole title plus an unshrinkable section label, and a grid item will not let its track be
    narrower than its own min-content, so the citation truncated only after widening the track
    past the screen: a horizontal scrollbar under the whole page, with the search box and
    results count pushed off the right edge. At phone width the result actions sat 80px beyond
    it, and on Models the Delete button sat outside the card's overflow-hidden, where
    nothing could reach it.

  • The health panel's reachability badge carried the model name, so the panel never
    plainly answered the question it exists for, and colour-coded a model by whether the
    backend was up. It now reports reachability the way Qdrant does and names the provider
    instead of assuming Ollama.

  • The workspace picker named a folder that does not exist. It joined a parent path on
    '' rather than /, so the button above books/orly/Architecture was labelled
    booksorly. Navigation was correct; only the label was wrong.

  • A deduplicated upload reported itself to console.info. That is the case that looks
    most like nothing happened, and it was written where nobody is looking.

  • Progress bars announced nothing, being a pair of divs, so an index running was
    invisible to a screen reader.

  • The file list stopped at 300 rows in silence, so a corpus of 4,000 files looked like a
    corpus of 300.

Added

  • /healthz reports a model a chunk set can no longer reach. The failure is silent
    until someone searches: the set's vectors are still in Qdrant and its row still says
    ready, but the query cannot be embedded, so the set answers nothing and a re-index of
    it cannot start. Deleting a model is already refused while a set uses it, so what this
    catches is a model pulled out from under the catalogue, or a volume restored without it.
    Only providers whose models can be listed are checked: a hosted provider cannot be asked
    what it has, and a guess there would be a false alarm on a working deployment. Reported
    per corpus:set, because the fix is per set, and surfaced in the health panel.

  • Notice, one warning component. Nine of these were hand-written: text-[var(--warn)]
    on a paragraph in one place, a card with a warn border in another, a ⚠ typed into the
    sentence in four more. No two matched and one was the wrong colour outright. Tones are
    Badge's five, the icon follows from the tone rather than from a character in the string,
    and role="status" by default, so a message that appears in response to something the
    reader just did also reaches anyone driving by keyboard and screen reader. Polite rather
    than alert, which interrupts; ErrorBanner keeps alert, because a failed request
    should.

  • CodeQL for C# and TypeScript, on pull requests, on main and weekly. docs/10
    listed this among the supply-chain controls while no such workflow existed, and code
    scanning needs Advanced Security on a private repository, so the line was false for as
    long as it had been written. Advanced setup rather than GitHub's default: C# here needs
    a .NET 10 SDK the runner does not ship, which the default setup cannot install.

  • A tag now cuts a GitHub Release, with that version's CHANGELOG section as the body,
    plus the pull command and the digest actually published. The workflow previously pushed
    an image, an SBOM and an attestation and left the Releases page empty. The notes are read
    at the top of the job, so a tag with no CHANGELOG section stops the release before
    anything reaches the registry. Releases start at 0.2.3; the five earlier tags have
    none. Questions now route to Discussions rather than the issue tracker.

Changed

  • /healthz names the endpoint of the provider actually in use. It reported
    Ollama.Endpoint unconditionally, so a deployment defaulting to OpenAI showed
    http://dexicon-ollama:11434, an address it never calls, beside a reachability badge for
    a backend on the other side of the internet. A default naming a provider that is not
    configured now says so rather than having an address invented for it.

  • Deleting a corpus or a chunk set asks in the app's own dialog. Both used
    window.confirm, which the page has no say over: it ignores the theme, places the
    destructive action wherever the browser likes, and after the second one a browser offers
    to suppress further dialogs for the session, at which point deleting a chunk set stops
    asking at all.

  • A neutral professional register across the docs and the code comments, 102 files.
    The pass removes the authorial voice: em-dashes where a colon, comma, full stop or
    parentheses were what the sentence wanted, editorialising asides, sentences admiring the
    previous sentence, and headings that were phrases rather than labels. Quoted system
    messages, facts, figures and links are unchanged. The README drops a fifteen-row table of
    contents that restated the filenames beside it, 22 lines to 11.
    (#9)

  • docs/09 documents the release order, numbered, because each step exists to stop the
    next one failing: CHANGELOG, then the regenerated OpenAPI document, then the tag.


docker pull ghcr.io/merp4/dexicon:0.2.3

Digest sha256:0c4687fe5c9a8fed85042be957087d7cf283de63ae0cb069d75c971f7f3ff878. An SBOM and build provenance are attached to the
image in the registry.