Skip to content

v0.2.0

Choose a tag to compare

@nnemirovsky nnemirovsky released this 06 Sep 12:28
· 8 commits to main since this release
d972557

New Features

  • Photo OCR through macOS Vision — text in a photographed announcement, price list or screenshot is read locally and becomes searchable, in Russian, English and Turkish. Needs the media extra; Russian recognition needs macOS 15.
  • PDF and DOCX text extraction through the same pass.
  • grepogram extract — a bounded, resumable pass that reads media after a sync, never inside one, so a slow OCR or a large PDF cannot eat a sync's budget. Media this build cannot read is parked without a single network request.
  • Reactions as a ranking signal — a bounded bonus that reorders near-ties without letting a popular message outrank a relevant one. search.reaction_weight = 0 disables it.
  • grepogram import <tdesktop export dir> — index a Telegram Desktop export of a chat the account can no longer open.
  • sources prune — drop chats that have left an indexed folder.
  • grepogram prune-deleted — a resumable sweep that removes messages deleted in Telegram.
  • PyPI publishing, behind a repository variable and trusted publishing (no API token).

Improvements

  • units.window_max_chars is now a real ceiling. It used to be a floor — a window closed only after passing the cap — so 15.8% of windows overflowed the embedder's 512-token limit and their tails were never vectorised. Existing indexes are re-cut once, automatically.
  • Deletions are noticed during an ordinary sync, not only by the explicit sweep.
  • The one-time re-cut runs a bounded number of whole chats per sync, each in its own transaction and resumable through per-chat markers, so search keeps answering while it works.
  • thread and context show extracted media text, and a chat can be named by its import:<slug> id wherever a chat spec is accepted.

Bug Fixes

  • grepogram extract resolved no chat at all on a real Telegram account: the client's entity cache starts empty by design and the two new by-id passes never warmed it. Found only by running against a real account — 1,757 tests missed it because the test client resolved any id, which it no longer does.
  • A search hit for an OCR'd photo linked to the wrong message, because extracted text never reached the message-level index.
  • An imported chat's import: tag could be overwritten by four separate code paths, after which sources prune would offer its history for deletion.
  • A replaced attachment kept the previous file's extracted text forever, unreachable even to --retry-failed.
  • sources prune re-validates every candidate inside the deletion lock, so a chat imported between the scan and the confirmation is no longer deleted.

Upgrading from v0.1.x

The next grepogram sync re-cuts and re-embeds every unit once — about an hour on a 47k-unit index — a few chats per run, each atomically, with search working throughout. Run grepogram extract afterwards to read the media already in your index.