Skip to content

v2.1.0

Choose a tag to compare

@github-actions github-actions released this 25 Oct 16:57
· 68 commits to main since this release
9fb8243

Requests to t.me now go out with one browser's TLS, HTTP/2 and headers. Nodes
on a private network can share their work without Redis (optional). Redis
connections hold up at a distance. Every release is gated by smoke tests, a
canary and fleet rollback. The /v1 and GraphQL contracts are unchanged.

Added

  • Browser-consistent HTTP client for t.me (TELEGRAM_CLIENT=browser, the
    new default). It is native: wreq on BoringSSL, in telegram-me-native. One
    profile (TELEGRAM_BROWSER_PROFILE=chrome_149 on
    TELEGRAM_BROWSER_PLATFORM=android) sets the TLS handshake, the HTTP/2
    settings and the headers, so they all describe the same browser.
    • JA4 and the Akamai HTTP/2 fingerprint change from rustls/hyper
      (t13d1011h2_61a7ad8aa9b6_…, m,s,a,p) to Chrome's
      (t13d1516h2_8daaf6152771_…, 52d84b11…); see
      docs/internals/http-client.md.
    • The "load more" call goes out as the page's XHR (sec-fetch-mode: cors,
      origin, referer, Chromium's header order), not as a navigation.
    • TELEGRAM_CLIENT=reqwest keeps pyreqwest as a fallback.
    • scripts/fingerprint.py measures both clients against a TLS echo service.
  • Peer layer (ENABLE_PEERS, off by default): nodes on a private network
    share their work directly, groupcache-style, with or without Redis. See
    Deployment → Peer layer.
    • Each channel's keys have one owner among the live nodes (rendezvous
      hashing). Only the owner fetches and renders them; on a miss the other
      nodes ask it and keep its answer for the remaining TTL. With peers up this
      replaces the Redis lease.
    • The owner answers cache reads natively, without Python: 0.18 ms for a
      56 KB page, against 95 ms through a Redis 90 ms away.
    • Membership is SWIM gossip (foca) over UDP; requests use one multiplexed
      TCP connection per peer. Nodes find each other through PEERS
      (addresses or DNS names) and the Redis heartbeat markers, which now carry
      "peer".
    • Every frame is authenticated (HMAC-SHA256 with PEER_SECRET, timestamp,
      nonce against replays) and optionally encrypted (PEER_ENCRYPTION,
      XChaCha20-Poly1305); peers outside PEER_ALLOW_CIDR are refused.
    • A slow or unreachable peer never fails a request: after PEER_TIMEOUT
      (50 ms) the node does the work itself.
    • Without Redis, nodes share rate-limit counts and feed popularity, and the
      live node with the lowest address runs the feed refresher.
    • New settings: ENABLE_PEERS, PEERS, PEER_PORT, PEER_ADVERTISE_ADDR,
      PEER_ALLOW_CIDR, PEER_SECRET, PEER_ENCRYPTION, PEER_TIMEOUT.
    • deploy/bootstrap-server.sh opens the peer ports to the private network
      (PEER_CIDR); the deploy workflow enables the layer with the
      ENABLE_PEERS variable and the PEER_SECRET secret.
    • The standalone feed worker runs with ENABLE_PEERS instead of
      REDIS_URL as well.
  • Stricter checks on every commit and pull request (make lint, make audit, make image-scan, all in make check): clippy restriction lints
    and no panicking slices in the parser; cargo-deny and cargo-machete; more
    mypy error codes and ruff rule families; import-linter layer contracts;
    vulture; uv audit; gitleaks; zizmor; hadolint; typos; trivy on the built
    image; dependency review on pull requests; OpenSSF Scorecard.
  • REDIS_CONNECT_TIMEOUT (default 2 s): seconds to open a Redis connection,
    TLS handshake included.
  • Deployment gates. Every server runs the smoke test (status codes and
    content) inside the new container after its health check and goes back to
    the previous image when it fails. Server 1 is a canary: it serves real
    traffic for CANARY_SOAK seconds and is verified again before the other
    servers follow. The rollout ends with the smoke test through the public URL
    (PUBLIC_URL or https://PUBLIC_HOST, with CF_BYPASS_KEY), and a
    failure after the first server rolls back every server that took the
    release (deploy/rollout.sh, remote-deploy.sh --verify and --rollback).
    Manual deploys can skip the smoke test while t.me is down.
  • CI runs the built image as two servers with two workers each and a real
    Valkey (tests/integration): shared cache, cross-process fetch lease,
    shared rate limit, proof of work, a rolling upgrade from the release in
    production and a Valkey outage. deploy/remote-deploy.sh is tested with
    real Docker Compose (upgrade, unhealthy and broken releases, rollback, the
    Caddy proxy) and the fleet logic with stub servers (tests/deploy).
  • Guards workflow: breaking changes of the REST (OpenAPI) and GraphQL
    contract against the base of a pull request fail it unless it is labelled
    breaking-change (scripts/api_contract.py, make contract); coverage
    floors may only go up (scripts/coverage_floor.py).
  • Coverage floors in CI: Python 92.5 % (fail_under), Rust 96 % lines
    (cargo llvm-cov).
  • Property-based tests (hypothesis): no 500 whatever t.me returns, the
    protection middleware on any input, feed invariants. Hostile-HTML tests of
    the Rust parsers (mutated pages, deep nesting, linear parse time) and a
    cargo fuzz target.
  • Nightly workflow: the production image against live t.me, production
    through the public URL, 10 minutes of fuzzing and long property runs; a
    failure opens an issue labelled nightly.
  • The smoke matrix checks the content of the main responses, not only status
    codes, and uses recorded pages only, so it also runs against a fake t.me.
  • TELEGRAM_BASE_URL: where t.me pages are fetched from (a mirror, or a fake
    t.me in tests).
  • Redis heartbeat (REDIS_HEARTBEAT_INTERVAL, default 5 s). Every worker
    writes its marker tme:v2:node:<host>:<pid> with a TTL of three intervals.
    • It keeps a connection to Redis open.
    • It notices outages within one interval.
    • It shows which processes are running.
    • It probes while the circuit breaker is open: the breaker closes as soon as
      Redis answers again, instead of after the full 30 s cooldown.

Changed

  • Request headers to t.me come from the browser profile for both clients,
    not from a hand-written Chrome 141 set: Chrome 149 values, Chrome's header
    order and a quoted sec-ch-ua-platform. cache-control and dnt, which
    Chrome does not send on a first visit, are gone.
  • The Docker build stage needs cmake, clang and git for BoringSSL. A cold
    build takes about 75 s longer on native arm64, and about 11 min longer for
    arm64 under QEMU, as the release workflow builds it. The runtime image grows
    by 3–7 MB.
  • The image runs as USER 10001:10001 (the same app user, by number).
  • The image's health check probes every 2 s during the first 30 s, so a new
    container is healthy (and a deployment done) within seconds instead of 30 s.
  • The Redis circuit breaker opens after three failures in a row within
    10 seconds
    instead of after the first one. Any command that reaches Redis
    resets the count. A single slow TLS connect to a distant Redis, or one lost
    packet, no longer takes the shared cache, the shared rate limit and the
    cross-server lease offline for 30 seconds. Failures below the threshold are
    logged at INFO.
  • Release images build each platform on a runner of its own architecture
    (ubuntu-24.04-arm for linux/arm64) instead of arm64 under QEMU, and
    merge the per-platform digests into one manifest list with the same tags,
    provenance and SBOM. Each platform has its own build cache.

Fixed

  • Redis no longer keeps dropping out when it is far away. Opening a
    connection used the 0.5 s command timeout, and a TLS connection to a Redis
    90 ms away takes 0.3–0.7 s. After any error, every reconnect timed out and
    reopened the circuit breaker, which bypassed Redis for 30 s every 30–60 s in
    production. Connections now get REDIS_CONNECT_TIMEOUT (default 2 s), and
    each worker opens its first connection at startup.
  • Deeply nested markup (tens of thousands of levels) no longer overflows the
    stack while a post is serialized, which aborted the worker process.
  • The nightly fuzzing job builds for the host target. The prebuilt
    cargo-fuzz defaulted to musl, which the nightly toolchain lacks (issue #217).

Security

  • ctutils 0.4.5 (RUSTSEC-2026-0385, non-constant-time comparisons under the
    proof-of-work HMAC). The crate declared Rust 1.85, which kept the resolver
    on the vulnerable release; it now declares 1.99, what it is built with.
  • The image no longer contains pip, whose vendored urllib3, msgpack and
    setuptools had fixable high-severity advisories; it was never used at run
    time.
  • The deploy workflow passes only the secrets it needs to the reusable
    deployment, reads repository variables from the environment instead of
    expanding them into the script, and keeps no git credentials after checkout.
  • The Dockerfile's base images (uv, Rust, Python) are pinned by digest as well
    as by tag; Dependabot updates both. OpenSSF Scorecard flagged the tag-only
    FROM lines (Pinned-Dependencies).