Chat Systems 4.0.0

Upgrading from 3.7.0

• Set SECRETS_KEYS before you upgrade: echo "k1:$(openssl rand -base64 32)", on the
  api and the worker. Both refuse to start without it. Keep JWT_SECRET unchanged for
  this start: the api re-seals what 3.7.0 sealed with it (see the RUNBOOK upgrade note).
• Set JWT_SIGNING_KEYS and JWT_PUBLIC_KEYS before you upgrade (the commands are in
  .env.example and the RUNBOOK): the private keys on the api and the worker, the public
  halves on realtime. Upgrade all three together. Everybody is signed out once, because
  the HS256 sessions 3.7.0 issued are refused. Remove JWT_SECRET once the api has logged
  that every stored secret is on a SECRETS_KEYS key.
• Migration 82 adds users.manual_away; migration 83 adds a key id beside every sealed
  secret; migration 84 adds command_invocations; migration 85 adds users.is_bot,
  bot_apps and hooks.app_id; migration 86 adds the event settings to bot_apps,
  app_deliveries and app_sockets; migration 87 adds huddle_sessions.livekit_room.
• For WebSocket clients: presence.batch and presence.changed carry away as well as
  online and offline, and presence.changed now also reaches the person it is about.
  For API clients: PUT /users/me/presence with { "away": true } sets yourself away.
• For WebSocket clients: a command.response frame carries a slash command's late answer to
  whoever ran it. For slash-command apps: the payload has request_id and response_url,
  the request an X-Chat-Request-Id header, and an empty 2xx body now means answering
  later instead of an error.
• For API clients: MemberWithUser has is_bot, Hook has app_id, and the routes a bot
  token may call carry the bot tag and security scheme in /api/openapi.json.
• For API clients: GET /workspaces/{id}/app-directory lists what is connected and
  POST /workspaces/{id}/app-requests asks the admins for an app (10 an hour per person).
• If something other than the bundled nginx sits in front of realtime, route /ws/apps to it
  as you route /ws: socket-mode apps connect there. For WebSocket clients: channel.updated
  is now in the replay log, and a channel.created event is published when a channel is made.
• Nothing to do for the huddle SFU unless you want it: LiveKit is a separate, optional compose
  file, and without LIVEKIT_URL every huddle stays on the mesh as before (see Huddle SFU
  (LiveKit) in the RUNBOOK). To turn it on, set LIVEKIT_URL on realtime as well as on the
  api; HUDDLE_SFU_THRESHOLD (6) and HUDDLE_SFU_MAX_PARTICIPANTS (100) are optional.
  For API clients: POST /huddles/{id}/sfu-token. For WebSocket clients: huddle.members
  carries the huddle's transport, and huddle.transport says a huddle moved to the SFU.

Added

• Away. You → Set yourself away (or /away) shows you away to everyone, in every
  workspace and on every device, until you set yourself active. Ten minutes without using the
  app shows you away too, and the next keystroke shows you active.
• Slash commands that take their time. A command's app may answer at once, or acknowledge
  and answer up to five times in the next 30 minutes through a response_url, in the channel
  or only to whoever ran it. An app that does not answer in three seconds is tried twice more
  with the same request id, and the person sees taking a moment… instead of an error.
• Apps with a bot user. An admin makes an app in Workspace settings → Apps with the
  scopes it needs and gets a token for it. With that token the app posts, replies in threads,
  reacts, uploads, reads history, opens DMs and looks people up through the same API the web
  client uses, only in the channels it has been added to. Its messages carry an APP label,
  what it changes is in the audit log under the app, and removing it revokes the token and
  its slash commands. docs/bot-api.md is the guide, and
  examples/echo-bot a bot in one file.
• Events for apps, over HTTP or a socket. An app subscribes to messages, mentions,
  reactions, people joining, and channels being made or archived, in the channels its bot is
  in. Events are posted, signed, to a request URL that has answered a challenge, and retried
  for an hour, after which the app shows as failing; or, in socket mode, an app-level token
  opens sockets to the instance that share the app's events and its slash commands, so an app
  behind a firewall needs no public URL. Delivery is at least once and in order per channel,
  and survives a restart of the worker or the gateway.
• An app directory. Workspace settings → Apps lists everything connected to the
  workspace: connected apps, the workspace's own apps, webhooks, slash commands and workflows,
  each with who added it and when, what it can do in plain words, the channels it touches and
  whether it is working. Each has a page with Reinstall, Rotate token and Remove, and the
  channel's Integrations section links to it. A new app gets its token, and a connected app is
  set up, only after an admin agrees to what it may do, and giving an app another scope asks
  again. Members see the directory read-only, without what touches channels they cannot see,
  and can ask the admins for an app.
• LiveKit beside the stack, for large huddles. docker-compose.livekit.yml runs a
  LiveKit SFU next to the app, and the API signs a ten-minute token for one huddle's room
  after the same check the realtime gateway runs on join. A huddle moves to it mid-call when
  a join takes it past six people, and nobody's audio drops on the way: each browser keeps
  the mesh playing until it hears everyone over LiveKit. On LiveKit a huddle holds up to 100.

Security

• Stored secrets have keys of their own. Connected apps' client secrets and tokens, TOTP
  secrets and the setup token are sealed with SECRETS_KEYS instead of JWT_SECRET.
  Rotating JWT_SECRET no longer breaks every connected app and second factor, and
  SECRETS_KEYS rotates on its own: add a key, put it first, restart, and the API re-seals
  everything, so nobody sets an app up again or reconnects.
• Sessions are signed with Ed25519, and the realtime gateway can no longer sign one.
  Tokens are EdDSA with a kid; the API and the worker hold the private keys and the gateway
  only the public halves, so a leaked gateway configuration mints nothing. HS256, unsigned
  tokens and unknown key ids are refused. Signing keys rotate without signing anybody out:
  add a key, promote it, retire the old one after the refresh-token lifetime.