Skip to content

v0.3.1

Choose a tag to compare

@github-actions github-actions released this 05 Oct 16:48
· 22 commits to main since this release
edd78bc

Mail you labelled after it arrived is no longer missed, PDFs full of charts read like the old engine again, and a real Ubuntu test found and fixed the problems of running on Linux: the nightly cron line, git without an identity, long and case-only titles, and a start with no home folder.

Changed

  • Supported platforms: macOS only, for now. Linux and Windows (WSL included) are not supported. Ubuntu 22.04 and 24.04 were tried in containers and the problems found there are fixed (below), but no support is promised: the package no longer lists Linux as a supported system, and the README, the landing page and the documentation say so, and sb doctor shows a platform warning off macOS.

Added

  • [run].max_batches_per_day (a whole number of at least 1, default 6): the hourly tick (sb run --if-due) runs no more than this many scheduled batches a day, counted from the nightly time and including the nightly run itself. Before, a large backlog was drained one batch per hour all day with no limit. Runs that stopped for an outage or an interruption, and manual runs, do not count.

Fixed

  • A file or disk problem is one line, not a traceback. A read-only vault, a full disk or a missing permission made sb add, run, lint, ask and init print a Python traceback. Now they print error: Permission denied: /path (or, for the queue and index databases, the vault's state database cannot be used (...). Is the vault folder read-only, full or in use by another program?) and exit 1; with --json the error object has the new code os_error (additive: contract 1 stays).
  • sb starts where there is no home folder (a uid with no passwd entry and no $HOME: a container, a service). import esbi_cli raised RuntimeError: Could not determine home directory, so even sb version crashed; now commands that need a home say to give explicit paths.
  • sb run | head no longer parks a source. The closed pipe was recorded as a failed attempt of the source being read (BrokenPipeError), and three of those parked it. Now the run stops quietly and the source goes back in the queue with no attempt counted.
  • sb schedule status on a system without launchd exits 0 and prints the cron line: there is nothing to query, which is not a failure (install and uninstall still exit 1, they cannot do what was asked).
  • sb update no longer says "Updated." when nothing changed. uv tool upgrade on an exact pin, or a release the package index does not list yet, exits 0 having done nothing. Now it compares the installed version before and after: Updated to X.Y.Z. or Nothing changed: X.Y.Z is still installed ....
  • A new vault's .gitignore was never committed (git status showed ?? for ever, and the rules that keep .esbi/ out did not travel with a clone). sb init now makes a first commit of the new vault, and .gitignore is one of the files the worker versions.
  • Linux and WSL get Linux advice. sb doctor told a Linux user to run brew services start ollama and said "launchd is not available here" with no way out; it now says ollama serve (or systemctl start ollama) and points to the cron line. sb today (and any command that opens an obsidian:// link) used open, which Linux does not have, so it only printed the link; it now uses xdg-open there (on WSL, install wslu). With no keyring on the system (a server, a container, WSL), sb email set-password and sb email fetch said "No recommended backend was available. Install ... keyrings.alt" (plaintext storage); they now say there is no keyring, where email capture works, and that cron cannot reach a desktop keyring. sb export --help showed "(default: /site)" because <vault> was read as a tag; it now says "the site folder in the vault".
  • On Linux, sources titled Foo, foo and FOO became three pages. macOS and Windows see those as one file name, so the second source got foo (2); Linux kept all three and every [[Foo]] link was ambiguous (and a vault shared with a Mac or Windows machine could not hold them). The same goes for a title that differs only in Unicode form (é as one character or as e plus an accent). A source page name, and a file dropped in inbox/ or saved from mail, now counts as taken when only case or Unicode form differs; a re-read of the same address still updates its page.
  • A source with a long Japanese, Chinese, Korean or emoji title could not be saved on Linux. File names are limited to 255 bytes there (ext4, also WSL), not characters, and a page title was cut at 100 characters: 90 Japanese characters or 64 emoji gave "File name too long" and the source was parked as failed. A title is now also cut to 240 bytes (never inside a character).
  • The cron line for systems without launchd (Linux, WSL) never worked. It started with a bare sb, but cron's PATH is /usr/bin:/bin, so an sb installed in ~/.local/bin was "not found" every hour and the nightly job silently never ran. The line printed by sb init --nightly and sb schedule now names sb by absolute path, quotes paths with spaces and escapes %.
  • On a machine where git has no name or email (a fresh Linux user, a container), the vault got no history. Every commit failed with git's "Author identity unknown" and a long warning after each source. Now a commit uses esbi-cli <esbi-cli@localhost> when git has no identity of its own; a name and email you configured are kept. Found by running 0.3.0 in a fresh Ubuntu 22.04 and 24.04.
  • Mail you labelled yesterday but that arrived long ago was never taken. The mailbox search counted the day a mail was received, so anything older than 14 days stayed invisible however recently you tagged it. Now every unread mail in the label is taken, however old, and the highest UID taken is remembered (per mailbox generation) so a mail added to the label later is taken even if you had already read it. A mail the server refuses to mark as read is no longer silent: sb email fetch says how many.
  • The notice [llm.x] timeout is now timeout_seconds is printed once a day per config file (the time is kept in notices.json in the cache folder, next to the update check's file; an unwritable cache only means more notices), not on every command. sb doctor still prints it every time.
  • A clip added from outside the inbox (sb add clip.md, sb ingest clip.md) now keeps its original: the file is copied unchanged into raw/inbox/ (never overwriting: a taken name gets (2)), because the raw/<date>-<title>.md snapshot holds the text after the page chrome was stripped. Clips dropped in inbox/ already ended up there.
  • sb doctor no longer tells an installed copy to run uv tool install --editable <site-packages folder> when sb is not on the PATH: the fix: line now matches how esbi-cli was installed (brew upgrade, uv tool upgrade, pipx upgrade, pip install -U), and the editable hint is only for a source checkout.
  • The messages no longer assume macOS: sb init says "this machine" (not "this Mac") and the keychain errors, sb email set-password and sb doctor say "system keyring" on Linux; on Linux sb doctor says the nightly job needs launchd, which only macOS has, and points at the cron line.
  • A config whose [paths].vault does not exist now fails with error: vault not found: ... (--json: {"error", "code": "vault_not_found"}, a new code, additive) instead of creating the folder and a .esbi/ inside it. Every command that opens the vault goes through the same check; sb doctor, sb init and the commands that only edit the config (sb ocr ..., sb email set-password) still work without it.
  • PDFs full of charts and diagrams read like the old engine again. A paper whose figures are drawn in layers (one test paper had its labels printed 80 times) came out with more than twice the text it has, and with no headings at all, because more than 200 big lines switched heading detection off. Now: a short line that a document repeats 8 times or more is a chart label and is dropped (unless it is a symbol inside a line of prose, such as the V of where V is); short lines packed into a small region of a page are dropped; and the number of headings is capped by the size of the document (8 per page, at least 40) by keeping the numbered and biggest ones instead of keeping none.
  • Figures with several panels are no longer cut, and a table beside a figure is no longer part of it. A drawn figure whose panels stand more than 30 points apart lost all but the one next to its caption (the left panel of the Transformer paper's attention figure, the first panel of a three-panel agent-loop diagram, the top row of a six-panel one). Now the panels under one caption, up to 120 points apart, are one figure, for drawings and for pictures alike (a panel narrower than a figure counts together with its sibling); a table of numbers is no panel; a figure keeps out of the column of another caption beside it (a drawn table next to a figure was taken into it, and the table's text was lost from the note); and another figure's caption, short or long, ends a figure's search upwards, so two stacked figures stay two.
  • Words cut at a line end are healed better. The test for a hyphen at the end of a line (a word broken in two, or a compound such as feed-forward) counted the second half of every cut word as a word of the paper, so multi-plicative, de-composed, ma-ture and lever-aged kept their hyphen. Now the halves of a cut word are not words of the vocabulary; a joined word is also accepted if it is a known word with an ending (leverage gives leveraged, omit gives omitted, similar gives similarity); and a first half that is a word of at least four letters, or of, in, non, ..., still makes a compound (sequence-aligned, mixture-of-experts). On 15 test PDFs (some papers twice) 42 of 1,367 line-end decisions changed, and by hand count about 38 of those were broken words made whole. Still wrong: a compound whose first half is short and whose second half is not a word of the paper (gpt-researcher), and a broken word with a long first half that is a word (multi-plicative).