Concurrent deliveries to one Telegram session crash the run with HookConflictError → session.failed

[Lukavyi]

Concurrent deliveries to one Telegram session crash the run with HookConflictError → session.failed

Environment

• eve@0.12.0
• Deployed on Vercel (Vercel Workflow world)
• Built-in telegramChannel() from eve/channels/telegram
• Telegram forum topics in a 1:1 chat (each topic = its own session via a thread-aware principalId)

Summary

When a second Telegram update for the same session (same chatId:threadId) arrives while the previous turn's workflow run is still active (not parked at a drain boundary), the run dies with:

HookConflictError: Hook token "telegram:<chatId>:<threadId>:" is already in use
by another workflow (run "wrun_…")

Because nothing catches it, the default session.failed handler then posts the user-facing "This session could not recover from an error… Start a new message to continue." A single too-fast follow-up therefore kills the conversation.

This is most visible on long turns (multi-minute tool work), where the window for a colliding delivery is widest.

Repro

1. Use the built-in telegramChannel() (no custom dispatch).
2. Send a message that triggers a long turn (e.g. several slow tool calls) into a topic.
3. While it's still working, send a second message into the same topic.
4. The second delivery throws HookConflictError; the run transitions to session.failed and posts the "could not recover" message.

Expected

A concurrent/duplicate delivery to a busy session should be coalesced, queued, or dropped — not escalated to a session-fatal error. At minimum, HookConflictError on inbound delivery should not surface to the end user as "session could not recover."

Actual

The conflict propagates and the session is failed.

Notes on root cause (from reading the compiled source)

• dispatchMessage (telegramChannel.js) delivers every inbound update via send(..., { continuationToken: continuationTokenFromState(state), state }), where the continuation token is telegram:<chatId>:<threadId>:<conversationId> — i.e. the same string as the workflow hook token.
• For sequential messages this correctly resumes the existing run, so normal back-and-forth works. The failure is purely a concurrency race: a delivery that lands while the run holds the hook but isn't at a drain boundary.
• The execution-model docs already acknowledge this: "When a turn is already active, the hook may accept additional deliveries, but the runtime only drains them at specific workflow boundaries… that drain is best-effort." — but when the drain doesn't happen, the result is a hard HookConflictError rather than a graceful no-op.
• defaultEvents has no handler that distinguishes this case, so it falls through to session.failed.

Suggested directions (any one would help)

1. Swallow inbound hook conflicts in the channel dispatch. When send() rejects with HookConflictError, drop or buffer the delivery instead of failing the session (the active run will still process its current turn).
2. Built-in per-session coalescing. Optionally fold a conflicting delivery into the next turn at the next drain boundary, matching the "best-effort drain" already described.
3. Don't render HookConflictError as session.failed. Even if it's left to the app to queue, an inbound-delivery conflict shouldn't read to the user as an unrecoverable session error.

Workaround (for others hitting this)

Maintain an app/channel-layer per-session queue keyed by chatId:threadId: mark the session busy on first delivery, buffer subsequent deliveries, and re-deliver after session.waiting. Requires an external store any webhook invocation can reach (defineState is session-scoped and can't coordinate the conflicting invocation). For a low-volume single-user bot, simply not sending a second message before the first turn finishes avoids it.

[Lukavyi]

Closing this — on further investigation the root cause was on my side, not eve.

My custom onMessage replaced the default dispatch gating (shouldDispatchTelegramMessage) without replicating its content check, so it dispatched Telegram service messages (forum_topic_created) that have no text/caption/attachments. Creating a forum topic then produced two near-simultaneous deliveries to the same chat:thread hook — the service message and the real first message — and the second raced the first, which is what surfaced as HookConflictError → session.failed. (The same empty service message also caused an messages: text content blocks must be non-empty 400 on the model call.)

Restoring the content gate (dropping no-content updates, as defaultOnMessage does) eliminated both. So the "concurrent deliveries to one Telegram session" here was self-inflicted, not a case of genuinely concurrent user sends.

The only residual point that may still be worth eve's consideration is whether an inbound-delivery HookConflictError should escalate to a user-facing session.failed at all, versus being coalesced/dropped — but I can no longer reproduce it independently of the service-message bug, so I don't want to leave a half-substantiated report open. Feel free to reopen if the escalation behavior is worth tracking separately.