Skip to content

v0.3.0

Latest

Choose a tag to compare

@z7vian z7vian released this 14 Nov 11:45
v0.3.0
b8ba274

vaegram 0.3.0 makes a bot's large uploads and downloads faster and harder to kill, fixes downloads from data centers that serve files on a media-only address, fixes the bugs an audit found in the session, transport, parser and dispatcher code, stops updates being lost silently, adds widgets and the rest of Bot API 10.4, adds what a bot built on vaegram was working around (rich ephemeral and guest messages, replacing the message a button is on, handler error context, run_until_signal, stream_rich_message), and adds PostgreSQL session storage.

Breaking

  • get_users returns one slot per requested id, in request order, with None for an id Telegram has no user for. It returned only the users found, and failed with PeerNotFound when one was missing.

    let users: Vec<User> = app.get_users([a, b]).await?;          // 0.2
    let users: Vec<Option<User>> = app.get_users([a, b]).await?;  // 0.3
  • Markdown parsing gives different text or spans for some input:

    • A [ inside link text starts a link of its own, and the outer [ stays text.
      [a [b](v)   0.2: link "a [b" to v      0.3: text "[a b", link on "b"
      
    • In a URL, \ escapes the next character, so \) no longer ends the URL.
      [a](x\)y)   0.2: link to "x\", then text "y)"      0.3: link to "x)y"
      
    • An escaped \], or a ] inside a code span, no longer closes link text.
      [a\](u) [b](v)   0.2: the "[" is lost and no later link is read      0.3: text "[a](u) b", link on "b"
      
    • No two spans cross. A span that crosses a quote is cut at the quote's edge and leaves out the line break beside it; a span that crosses a link's edge is cut there; a custom emoji or date that would be cut by a quote is dropped.
      **a\n> b** c   0.2: one bold over "a\nb", across the quote's start      0.3: bold "a" and bold "b"
      
    • A **> quote is expandable only when its last line ends in a || that closes no spoiler.
      **>||s||\nt   0.2: expandable quote      0.3: plain quote
      
    • Empty spans (****) are dropped. Custom emoji ids and times may be negative.
  • HTML parsing: a self-closing tag opens no span.

    <b/>x   0.2: "x" bold      0.3: "x" plain
    
  • Text::to_markdown writes some text differently:

    • ) and \ in a URL are written \) and \\, not %29. Markdown that already holds %29 parses as before.
      Text::new().link("l", "https://x.y/(a)").to_markdown() // 0.2: "[l](https://x.y/(a%29)"   0.3: "[l](https://x.y/(a\\))"
    • ] in the text of a link, mention, custom emoji or date is written \]; 0.2 dropped the span.
      Text::new().link("a]b", "u").to_markdown() // 0.2: "a\\]b"   0.3: "[a\\]b](u)"
    • A quote right after another quote starts with **>; 0.2 dropped the second quote.
      Text::new().blockquote("a", false).plain("\n").blockquote("b", false).to_markdown() // 0.2: "> a\nb"   0.3: "> a\n**> b"
    • A span with exactly the range of a code span or block is written around it (**abc**); 0.2 dropped the bold. A quote inside or across another quote is dropped.
  • Client::send_ephemeral_message, Message::reply_ephemeral and EphemeralMessage::reply take impl Into<MessageText> instead of impl Into<Text>. A &str, String, &String or Text argument still compiles; code generic over T: Into<Text> must convert first.

    msg.reply_ephemeral(t)                      // 0.2, t: impl Into<Text>
    msg.reply_ephemeral(Into::<Text>::into(t))  // 0.3

    Message::reply_ephemeral on a message in a forum topic now sends into that topic, as Message::reply does.

  • verify_chat(..).custom_description takes impl Into<Text> instead of impl Into<String>, and reads the description in the client's parse mode, as message text is read. With the default parse mode (Disabled) nothing changes. With another one, set Disabled on the request to send the text as typed.

    app.verify_chat(chat).custom_description(desc).await?                                 // 0.2: sent as typed
    app.verify_chat(chat).custom_description(desc).parse_mode(ParseMode::Disabled).await? // 0.3: the same
  • A message carrying a widget has Media::Widget and MediaKind::Widget; 0.2 gave Media::Unknown. Both enums are #[non_exhaustive], so matches still compile, but a widget no longer reaches an Unknown arm.

    Some(Media::Unknown { .. }) // 0.2, for a widget
    Some(Media::Widget(w))      // 0.3
  • Dc has a new public field media, and Dc::wire_id is negative when it is set. Dc is #[non_exhaustive], so code written for 0.2 still compiles; a Dc with media set is not equal to the same Dc without it.

    Dc::new(2, addr).wire_id()             // 2 in 0.2 and 0.3
    Dc::new(2, addr).media(true).wire_id() // -2, new in 0.3
  • Deprecated, not removed: StoryAreaPosition is renamed AreaPosition, the Bot API 10.4 name, which TON wallet cards use too. The old name is a deprecated alias.

    StoryAreaPosition::new(50.0, 50.0, 10.0, 10.0) // 0.2, warns in 0.3
    AreaPosition::new(50.0, 50.0, 10.0, 10.0)      // 0.3
  • A bot's upload or download of 8 MiB or more whose size is known behaves differently by default: 8 connections (4 before) and a pace of 20 MiB/s down and 19.5 MiB/s up. A download by file id, whose size vaegram does not know, goes as in 0.2. Nothing in the API changes; pass None to max_download_bytes_per_sec / max_upload_bytes_per_sec to remove the pace. A large bot upload holds 8 MiB of parts in memory at a time (4 MiB before), a large bot download 4 MiB (2 MiB before).

  • RetryContext has new public fields file_part and flood_count, and ClientEvent a new variant ConnectFailed { dc, error }. Both types are #[non_exhaustive].

Added

  • Widgets (MTProto layer 233): send_widget, send_shared_widget, Message::answer_widget / reply_widget / answer_shared_widget / reply_shared_widget, answer_widget_query, request_widget_query_answer, edit_message_widget_data, edit_inline_widget_data, edit_shared_widget_data, get_widget_public_data, the handlers on_widget_query, on_shared_widget_query and on_widget_data_updated, filters::widget(), Message::widget() and InputMessageContent::Widget. copy_message copies a shared widget by its id. Only bots can send widgets; vaegram does not render HTML.
  • TON wallet cards: ChatFullInfo::ton_wallet_card (TonWalletCard) and set_ton_wallet_card_stickers.
  • ReplyParameters::quote_rich_texts: quote part of the replied message as rich text.
  • verify_chat(..).custom_description takes formatted text, with .html(), .markdown() and .parse_mode(..).
  • CallbackQuery::answer_ephemeral(text): an ephemeral message to the user who pressed the button. A button on an inline message gives Error::InvalidArgument.
  • Bot API 10.4 names for calls that already existed: get_unique_gift(name) (as get_upgraded_gift) and delete_business_account_gift(business_connection_id, owned_gift_id) (as delete_gift(..).business_connection_id(..)).
  • Ephemeral messages take a rich message as well as text: send_ephemeral_message, Message::reply_ephemeral, EphemeralMessage::reply and CallbackQuery::answer_ephemeral. A rich message with media is refused with InvalidArgument before anything is sent. SendEphemeralMessage::message_thread_id(id) without a reply is sent as a reply to the topic's root message; together with reply_to_ephemeral it is refused. impl From<&String> for MessageText.
  • Guest messages: GuestMessage::reference_messages (service and empty messages left out), GuestMessage::answer_text and answer_rich (an article with no title), and InputMessageContent::link_preview_options, which changes text content and returns other content unchanged.
  • HandlerError::message_id, user_id and query_id, filled from the update the failed handler was given.
  • Error::Other, Error::other(e) and Error::downcast_ref::<T>(): a handler can return its own error type through vaegram::Result and read it back in on_error. A string is boxed as std's string error; read it with to_string.
  • Client::run_until_signal(): like run_until_ctrl_c, and also stops on SIGTERM on Unix. SIGTERM is watched before the run starts.
  • Client::stream_rich_message(chat_id): pushed chunks append Markdown (or HTML with .html()) source, each draft carries all of it, and finish sends one rich message. A draft Telegram refuses is skipped; only finish can fail. No local length limit. A dropped stream sends no final message.
  • ClientEvent::PacketDropped has a new reason, "msg_container holds a newer msg_id", and a packet dropped by the replay check is reported too.
  • SendEphemeralMessage::replace_callback_query_message(): the ephemeral message replaces the message the pressed button is on. The press is the one named by callback_query_id(..), which CallbackQuery::answer_ephemeral sets; without one, awaiting fails with InvalidArgument. Refused locally together with reply_to or a topic id other than 1; Telegram's own error comes back unchanged when the tap is too old.
  • SkipReason::NoPts: an updateChannelTooLong for a channel the client has no pts for and that carries none.
  • Transfer options on the builder: download_connections(n), upload_connections(n) (1 to 8), max_download_bytes_per_sec(rate) and max_upload_bytes_per_sec(rate) (None removes the pace, a value paces any account kind, 0 is refused). They apply to transfers of 8 MiB or more whose size is known.
  • RetryPolicy::retries_file_parts(): whether a policy's "no retries" also means none for the parts of a file. NoRetries sends each part once.
  • ClientEvent::ConnectFailed { dc, error }: a transfer connection that could not be opened; the transfer goes on with the ones that did.

Fixed

  • Session:
    • When Telegram sent msg_new_detailed_info for an answer whose ack had already gone out, the answer was asked for again, and msgs_state_req reported it as unknown.
    • A container holding a message id not below its own was accepted; it is now dropped whole.
    • Once the set of seen message ids was full, the messages inside a container could be dropped.
  • Transport:
    • Downloads from a data center's media-only address failed with transport error 444: the DC id was not sent negative. The permanent key and the authorization import now go over the DC's main address. Over WebSocket, and with a pinned address, the main address and the plain id are kept.
    • A WebSocket PING on an idle connection went unanswered until the client next sent something. Up to 8 pongs are queued and sent without an MTProto message.
    • A WebSocket connection through a SOCKS5 or HTTP proxy resolved Telegram's host name locally. The proxy now gets the name; SOCKS5 refuses a name over 255 bytes; a proxy that refuses the name fails the connect. SOCKS4 has no name form and still resolves locally, taking the first IPv4 address.
    • The DNS-over-HTTPS fallback accepted a chunk not followed by CRLF, a body over its size cap, and a \u escape with fewer than four hex digits.
  • Dispatcher:
    • The update state's date could move back when an older batch was applied.
    • Updates held while waiting for a gap had no limit. At most 1024 are held per key; past that a pts, qts or channel key drops what it holds and fetches the difference, reporting the dropped range in UpdatesSkipped if the fetch is given up.
    • ask lost an answer that arrived while the question was still being sent. Messages matching the question's listener that arrive during the send (at most 64) reach handlers once the question is out.
  • Parsers:
    • A file id with a zero run of length zero, or a run marker at its end, was decoded instead of refused.
    • A Markdown quote whose last line ended in inline code or a fenced block ended in the wrong place, and a span opened in a quote could run past its end.
    • In Markdown, a link whose ] started a line let the quote it was in swallow the next line (> [a\n](u) b).
    • Markdown with many unmatched [ was scanned again from each one.
    • to_markdown output that parsed back differently: the cases listed under Breaking, and a spoiler starting at the line break of an expandable quote.
  • update_status failed with Error::Unexpected when Telegram answered false, which it does when it takes no action.
  • Transfers:
    • A bot's large upload ran into FLOOD_WAIT dozens of times, and its download used half the connections Telegram allows. Large bot transfers are now paced just under what Telegram allows a bot (see Verified live).
    • A transfer connection that died silently was not noticed: a part waited 128 s three times on the same dead connection and the transfer failed after 384 s. A transfer connection with requests waiting is now probed after 10 s of silence and dropped about 20 s later, and a part that times out restarts its connection before it goes again.
    • A write or flush to a peer that takes nothing never returned. It now fails 20 s after the bytes could have left at 8 KiB/s, on every connection.
    • A part answered with a server error (500 or a negative code, such as -503 Timeout) failed the transfer. It is sent again after 1, 2, 4, 8, 16 and 32 s. Parts of a file get four times the flood waits of other requests.
    • A transfer whose connections did not all open failed; it continues with those that did.
    • A download asked upload.getFile twice for a chunk when two answers shared a container.
  • Updates:
    • A batch Telegram re-sent was dropped whole, including updates in it that had not been seen; only the updates that do not move their own counter are dropped now.
    • After PERSISTENT_TIMESTAMP_INVALID the client kept asking with the refused pts. It forgets the pts (a channel starts again from the next update; the account asks updates.getState at once) and backs off from 0.5 s to 8 s.
    • An updateChannelTooLong for a channel the client kept no pts for was ignored, so the channel's updates were never fetched; in a run without catch_up the first difference dropped it as undated. It is now fetched from the pts it carries, or reported as UpdatesSkipped with SkipReason::NoPts.
    • A channel whose too-long difference named no position kept its old pts and fetched the same messages at every gap; it is reported and started again.

Storage

  • PostgresStorage::new(config, tls, name), behind the new default-off feature postgres (tokio-postgres 0.7). config is a tokio_postgres::Config; tls is your TLS connector, which must be Clone, or tokio_postgres::NoTls; vaegram adds no TLS dependency.
    let config: tokio_postgres::Config = std::env::var("DATABASE_URL")?.parse()?;
    let app = Client::builder("my_bot", api_id, api_hash)
        .storage_engine(PostgresStorage::new(config, tokio_postgres::NoTls, "my_bot"))
        .bot_token(token)
        .start()
        .await?;
  • name is the PostgreSQL schema that holds this session's tables, so several sessions can share one database and none touches the application's own tables. It must be a lowercase identifier of at most 63 characters, not starting with pg_ and not public or information_schema. The role needs CREATE on the database, or must own a schema of that name.
  • Every statement names the schema and no connection setting is used, so the engine can run behind a transaction-mode pooler (PgBouncer, pooled Supabase or Neon endpoints). This follows from the design; no pooler was tested.
  • Connecting and reconnecting happen on the storage thread, not on your runtime. The connect and every operation are limited to 10 seconds. Each write transaction also sets statement_timeout and lock_timeout; an operation that runs out of time cancels its statement and drops its connection.
  • Same session fields as the other engines; the session format has no new tag.
  • The engine's tests (conformance, reconnect, timeout, two sessions in one database) run in CI against postgres:16; the storage-services job passed on #139.

Verified live

  • Transfer speed, one bot from one server (61 ms from DC 5) to DC 5, MB/s (1 MB = 1,000,000 bytes). "0.2 shape" is this code with 4 connections and no pace.

    File Setting Down Up FLOOD_WAITs down / up
    500 MiB, 3 rounds 0.3 default 19.74 to 20.23 20.06 to 20.15 0 to 2 / 6 to 7
    500 MiB, 3 rounds 0.2 shape 14.03 to 19.39 16.81 to 17.93 0 / 32 to 46
    2,000 MiB, 1 run 0.3 default 20.36 20.33 2 / 6

    Nothing is measured for other data centers, for user or Premium accounts (unchanged in this release), or for several processes sharing one bot token.

  • replace_callback_query_message: a plain, a rich and a document message were replaced. Not tried: a forum topic, an ephemeral message's own button, another receiver.

  • Media-only DC: tests/live_media_dc.rs against Telegram. Without the fix, three of its tests failed with Transport(444) downloading from DC 2's media-only address; with it, all three passed. The MTProxy test skipped itself (no proxy configured).

  • With the live tests of #140, run against main:

    • on_pinned_messages sees a pin and an unpin. Telegram did not push the update every time; when it did not, recover_gaps fetched it.
    • on_reactions and on_chat_action.
    • remove_chat_member followed by a rejoin, send_bot_requested_peer, and a poll with allows_revoting.
    • copy_message to a broadcast channel, and thumbnail file ids.
    • A bot's button with a custom emoji icon: the outgoing request carries the icon id (recorded byte for byte). Custom emoji in a bot's rich message: Telegram delivered the emoji's glyph for each of blocks, HTML and Markdown.

Not verified live

  • Widgets. In the last runs Telegram answered INPUT_MEDIA_WIDGET_NOT_ALLOWED to every widget the test bot sent, after accepting and rendering them earlier the same day and then showing "Widget unavailable". The recorded messages.sendMedia carries the widget exactly as given; why Telegram refuses is not known. No widget send, query or edit flow is verified.
  • ban_chat_member(..).revoke_messages(true) in a basic group: after it, the removed bot could still read the group's message by id. Whether Telegram keeps a removed bot's history or the flag does not do what its doc says is not known.
  • Custom emoji sent by a bot: vaegram sends the id, and Telegram dropped it for the test bot (the button icon is missing from both copies; rich text arrives as the glyph). Not shown: whether a bot owned by a Premium account gets it through.
  • The media-DC fix through an MTProxy and on Telegram's test servers.
  • Guest answers with an untitled article, and half-written Markdown in a rich draft: what the server does with them was not observed. Rich and topic ephemeral messages are tested on the request bytes only.
  • on_read_history, on_pending_join_requests, on_stars_balance and quick replies: their live tests skipped themselves (they need a second user account, a person reading a message, spending Stars, or Premium).
  • Not in this release: editing a guest answer (the code is on a branch; the test bot has no Guest Mode to prove it with).
  • The transfer connection watch, the write stall bound and the restart of a timed-out part: tested in memory only. In the 2,000 MiB run three parts were answered Timeout, went again and the upload finished.

Known issues

  • The live test user_posts_and_edits_a_story fails: Telegram answers PREMIUM_ACCOUNT_REQUIRED to a story post from the non-Premium test account that it accepted a day earlier. TDLib treats this error as "Premium needed", apart from the story limits. Not caused by this release's code; the other 39 tests of the live suite pass.
  • Widgets unverified live (above). Strict live tests and a request test are on the local branch test/live-widgets, not on main.
  • upload.saveFilePart returned false ended two uploads of a measuring prototype that paused every part on one FLOOD_WAIT. It was not seen with the released code and is not explained.
  • Channel updates that carry no message may not be pushed to a user account that isn't viewing the chat. Only on_pinned_messages says so; on_deleted_messages, on_read_history and on_widget_data_updated route updates of the same kind and were not tested for this.
  • Updates dropped at the cap of 1024 held updates come back with the difference the cap starts. If that difference is given up, dropped qts updates come only with the account's next fetch, and nothing schedules one. A batch held for a gap when the client stops comes back at the next start only with catch_up.
  • When a channel is started again after a too-long difference without a position, an update for it inside a batch still held for a seq gap is delivered later and may repeat a message.
  • A write on a silent transfer connection (a resent bind, with PFS and a request_timeout under 20 s) puts off its drop until the part's own timeout. A part queued before its connection dropped for another reason needs one more timeout to restart the new one.
  • CallbackQuery::answer_ephemeral returns Result<SendEphemeralMessage>, while Message::reply_ephemeral returns the request and fails when awaited: two ways to report a missing receiver.
  • SentGuestMessage's doc names only GuestMessage::answer, not answer_text / answer_rich.
  • A widget whose payload is over 1 MiB or not UTF-8 after inflating becomes Media::Unknown with no sign why.
  • Bot API 10.4's "receiver and chat optional with callback_query_id" is not offered: answer_ephemeral always fills both.
  • StoryAreaPosition stays as a deprecated alias until it is removed.

Install

vaegram = "0.3"   # or vae = "0.3", the same crate under a shorter name

Requires Rust 1.85+. Documentation: https://docs.rs/vaegram