Skip to content

v0.8.0

Choose a tag to compare

@github-actions github-actions released this 29 Oct 01:48
· 21 commits to main since this release

After the upgrade the daemon rewrites what older versions cached, a little each
second, in the background (#89): raw_json gets smaller and older media messages
get their attachment rows. tg status shows the progress; without the daemon,
tg db migrate does it in the foreground. When it is done, tg db vacuum gives
the freed space back to the disk.

Added

  • Background migrations: a rewrite of every row of a big cache now runs in
    small batches, each a short write, and resumes where it stopped. The
    daemon runs them a little each second, after its sync work; live updates
    and sync jobs wait for one batch at most. tg db migrate [--account]
    finishes them in the foreground, with or without the daemon. tg status
    lists each unfinished one with its progress (migrations, and
    accounts.accounts[].migrations) and suggests tg db vacuum for the
    space they free (#89).
  • The first background migrations rewrite what older versions cached:
    raw_json of messages, users and chats into the current form, a
    message_attachments row for every older media message, and the
    message_type of older history messages (a video is video, no longer
    document). Reads give the same answers while they run (#89, #22).
  • bun run bench:storage: a storage benchmark that builds a made-up cache of any size through
    tg's own write paths and reports bytes per message by table, write rates, query times,
    tg db migrate on old-form rows, tg db vacuum and a search index rebuild. docs/storage.md
    gives the numbers for 100,000, 1M and 10M messages and the storage decisions they back (#80).

Changed

  • raw_json stores 64-bit values (id, accessHash, groupedId and
    others) as decimal strings, as it did bigints, instead of objects with
    low, high and unsigned: json_extract(raw_json, '$.accessHash') in
    tg sql gives the hash (#108).
  • raw_json no longer stores fileReference: Telegram's file references
    expire, and tg media download fetches the message again for a fresh
    one. A made-up photo message takes 1,776 bytes instead of 1,898 (#95).

Fixed

  • tg sql print-schema --output=sql and --output=text print the schema
    again: tg sql rejected every format but json and csv before
    print-schema ran. An unknown --output now returns INVALID_ARGS in the
    JSON error schema, and --output may also come before the subcommand
    (tg sql -o sql print-schema) (#109).
  • tg db migrate next to a running daemon no longer locks the daemon's
    writes out: it ran its batches back to back, and on a big cache a daemon
    write could give up with database is locked. With the daemon running it
    now pauses 150 ms after each batch of at most 50 ms, so a daemon write
    waits for one batch at most; the run takes four to five times as long as
    with the daemon stopped (#89).