Skip to content

v0.6.0

Choose a tag to compare

@github-actions github-actions released this 28 Oct 23:01
· 26 commits to main since this release

Stop the daemon before you upgrade and start it again after: the first open
of a cache replaces its search trigger and drops an index nothing used
(#87). Then tg db vacuum gives that index's space back. tg messages search
now reads a query the way FTS5 does: hello world finds both words
anywhere, and --phrase gives the old exact-phrase search (#84).

Added

  • tg messages search filters by date and ranks by relevance. --since and
    --until take a date, a date and time, or a duration back from now (7d,
    24h); --sort relevance puts the best matches first; --count also
    returns how many messages match over all pages, as count. total is
    still the number of results on the page (#85).
  • tg messages search --snippet adds a short fragment of each message around
    the matched words, with the words in **, for Latin and Cyrillic text and
    prefixes like квартир*. For tg sql, the new view message_search_text
    returns full-text matches with the message text, chat and sender, so
    WHERE message_search_text MATCH 'deploy' ORDER BY rank needs no join. The
    view stores nothing, so the cache stays the same size (#81).

Changed

  • raw_json stores byte fields, such as a photo's fileReference and its
    stripped thumbnail, as base64 strings instead of objects with a key per
    byte. A made-up photo message's raw_json drops from 10,213 to 1,896
    bytes. Every cached message, chat and user is written this way; rows cached
    before keep the old form until they are fetched again.
    docs/database-schema.md shows how to read the fields with json_extract
    (#86).
  • A message fetched again with nothing changed is no longer written again,
    and its search index entry is left alone. Catch-up after every daemon
    restart or reconnect fetches again the messages that arrived live, so each
    of them used to cost a row rewrite and a re-index. A message is re-indexed
    only when its text, sender or media path changes; a change only in
    raw_json, such as a channel post's views, updates the row but not the
    index, and marking a message deleted no longer re-indexes it. A side effect
    of the old re-indexing goes away: a re-fetch no longer refreshes the sender
    and chat names stored in the index, so a renamed user or chat is found by
    the new name only in messages indexed after the rename. Results still show
    the current names (#93 tracks re-indexing after a rename). The first open of
    an existing cache replaces its search trigger (#87).
  • The cache no longer keeps an index on messages_cache.fetched_at: no query
    used it, and it took space for every message. The first open of an
    existing cache drops it; tg db vacuum gives the space back (#87).
  • tg messages search reads a query the way FTS5 does. hello world now
    finds messages with both words anywhere in them, not only the exact phrase;
    pass --phrase for the old behaviour. Operators, phrases, column filters and
    prefixes work as --help and docs/search.md describe them: error OR timeout, "exact phrase", sender:alice, chat:team, and квартир* for
    every form of a word. A query FTS5 can't parse returns INVALID_ARGS with a
    suggestion instead of TELEGRAM_ERROR (#84).

Fixed

  • tg sql can no longer change the cache with a pragma. It let
    PRAGMA user_version = 0 through, which made the next start run the
    one-time migrations again. Queries now run on a read-only connection and
    such a statement, in either form (PRAGMA writable_schema(1) too), returns
    SQL_WRITE_NOT_ALLOWED. A query with more than one statement returns
    SQL_OPERATION_BLOCKED instead of running only the first. Reading pragmas such as
    PRAGMA table_info(messages_cache) works, and no longer fails with a
    syntax error from the row limit (#88).