[Bug/Feature]: Handle Slack conversations.history Rate Limiting with Retry-After Backoff

[htarrido-picart]

Summary

Parse Slack's Retry-After header on 429 responses, add exponential backoff with jitter, proactive rate budgeting, request queuing, and in-memory history caching so the agent never drops channel context to rate limits — especially critical as Slack tightens limits for non-Marketplace apps to 1 request/minute starting March 3, 2026.

Problem statement

ZeroClaw's Slack channel calls conversations.history to fetch channel context, and under normal usage this triggers Slack's rate limiter:

zeroclaw WARN zeroclaw::channels::slack: Slack history error for channel SOME_CHANNEL_ID: ratelimited

The current behavior is to log the warning and move on. This means the agent loses channel context — it either responds without history or fails to respond entirely. The problem is getting worse because Slack recently tightened rate limits:

• Marketplace and internal apps: conversations.history is Tier 3 (~50 requests/minute per workspace)
• Non-Marketplace commercially distributed apps: as of May 29, 2025, limited to 1 request per minute with a maximum of 15 objects per response. Existing installations of unlisted apps will be subject to these limits starting March 3, 2026.

When Slack returns HTTP 429, the response includes a Retry-After header specifying the exact number of seconds to wait before retrying. ZeroClaw currently ignores this header.

Proposed solution

• Maintain a per-workspace sliding window counter of conversations.history calls
• Before making a request, check if the budget allows it based on the configured tier
• If at or near the limit, delay the request preemptively rather than burning a call that will be rejected
• Tier detection: if the first conversations.history call returns a limit of 15 objects, the app is on the restricted non-Marketplace tier — adjust the budget to 1 request/minute automatically

Configuration:

[channels_config.slack]
# Existing fields
bot_token = "enc2:..."
app_token = "enc2:..."
allowed_users = ["U12345678"]

# Rate limit handling (new)
rate_limit_max_retries = 3              # max retries on 429 (default: 3)
rate_limit_budget_per_minute = 50       # proactive request budget (default: 50, auto-adjusts to 1 for restricted apps)
rate_limit_jitter_ms = 500              # max random jitter added to Retry-After (default: 500)
history_fetch_limit = 200               # messages per conversations.history call (default: 200, capped at 1000)

3. Request queuing for burst scenarios

When multiple channels or threads need history simultaneously (e.g., daemon startup, multi-channel deployment):

• Queue all conversations.history requests through a single rate-aware dispatcher
• Dispatcher spaces requests to stay within the per-workspace budget
• Requests are processed FIFO with priority for the channel that triggered the current agent interaction
• Queue depth limit: 20 pending requests (configurable). If exceeded, oldest non-priority requests are dropped with a warning.

4. History caching

Reduce the need for conversations.history calls by caching recent messages:

• Cache the last N messages per channel in memory (default 50, configurable)
• On inbound message, append to cache — no API call needed for recent context
• Only call conversations.history when the cache is cold (daemon restart, first interaction in a channel) or when the agent explicitly requests older context
• Cache invalidated on daemon restart (not persisted — message history belongs to Slack, not ZeroClaw)

5. Improved logging

Replace the current single-line WARN with structured, actionable log output:

WARN  zeroclaw::channels::slack: Rate limited on conversations.history for channel C0SOME_ID. 
      Retry-After: 30s. Attempt 1/3. Next retry at 2026-02-25T14:30:30Z.

ERROR zeroclaw::channels::slack: Rate limit retries exhausted for conversations.history on channel C0SOME_ID. 
      Total wait: 90s across 3 attempts. Proceeding without channel history.
      Hint: If this recurs, reduce history_fetch_limit or check if your Slack app is on the restricted non-Marketplace tier.


### Non-goals / out of scope

* No Slack Marketplace listing process changes (that is a Slack-side administrative decision, not a ZeroClaw code change)
* No rate limit handling for other Slack methods (chat.postMessage, reactions.add, etc.) in this issue — same pattern can be extended in a follow-up
* No persistent message cache (history belongs to Slack; caching across restarts introduces stale data risks)
* No multi-workspace rate budget isolation (single workspace assumed; multi-workspace support is a separate feature)



### Alternatives considered

* **Increase Slack API tier by listing on Marketplace** — solves the rate limit but is an administrative/business decision outside ZeroClaw's control. ZeroClaw should handle rate limits gracefully regardless of tier.
* **Reduce history fetch frequency** — partially helps but doesn't solve cold-start or burst scenarios. Caching is strictly better.
* **Switch to Slack Events API / Socket Mode for real-time messages** — ZeroClaw may already use this for inbound messages, but `conversations.history` is still needed for context window population on daemon restart or when replying to threads with prior context. The two approaches are complementary.
* **Ignore the error and respond without history** — current behavior. Degrades agent quality because the LLM lacks conversation context.

### Acceptance criteria

**Retry-After backoff:**
* HTTP 429 response triggers wait for `Retry-After` seconds + jitter, then retry
* Exponential backoff on repeated 429s (Retry-After * 2^attempt, capped at 120s)
* Max retries configurable, default 3
* After max retries, log `ERROR` with channel ID and total wait time, proceed without history
* Non-429 errors (auth, channel_not_found) are not retried

**Proactive rate budget:**
* Sliding window counter tracks per-workspace `conversations.history` calls
* Requests delayed preemptively when near budget limit
* Auto-detects restricted tier (15 object limit) and adjusts budget to 1 req/min

**Request queuing:**
* Concurrent history requests queued through single dispatcher
* FIFO with priority for active interaction channel
* Queue depth limit enforced, excess requests dropped with warning

**History caching:**
* Recent messages cached in memory per channel
* Cache populated from inbound events (no API call)
* `conversations.history` only called on cache miss (cold start, older context)
* Cache cleared on daemon restart

**Logging:**
* Rate limit events logged with channel ID, method, Retry-After value, attempt count, and next retry timestamp
* Retries exhausted logged at ERROR with actionable hint about tier detection

**Tests:**
* Unit tests: Retry-After parsing (valid header, missing header, zero value), exponential backoff calculation (each attempt, cap at 120s), jitter range, budget counter (increment, window expiry, tier auto-detection), cache (hit, miss, invalidation, capacity)
* Integration tests: simulated 429 response triggers retry and succeeds on second attempt, three consecutive 429s exhaust retries and log ERROR, queue dispatches requests within budget, cache prevents API call for recent messages

### Architecture impact

* `src/channels/slack.rs` — Retry-After parsing, backoff loop, rate budget check before `conversations.history` calls
* `src/channels/slack_rate_limiter.rs` — new module: `SlackRateLimiter`, sliding window counter, request queue dispatcher, tier auto-detection
* `src/channels/slack_cache.rs` — new module: `SlackHistoryCache`, per-channel in-memory message cache
* Config schema: new fields in `[channels_config.slack]`


### Risk and rollback

* Risk: Low (additive behavior on existing Slack channel, no changes to message sending or event handling)
* Retry loop introduces potential for delayed responses — mitigated by max retry cap and proceeding without history after exhaustion
* In-memory cache increases memory usage — mitigated by configurable cache size and per-channel cap
* Rollback: revert commit, remove rate limiter and cache modules. Slack channel reverts to current behavior (log warning, no retry).


### Breaking change?

No

### Data hygiene checks

- [x] I removed personal/sensitive data from examples, payloads, and logs.
- [x] I used neutral, project-focused wording and placeholders.

[theonlyhennygod]

Implemented an initial reliability patch for Slack history rate limits.

Root cause

• conversations.history polling treated Slack rate-limit responses as generic failures and continued without honoring Retry-After.

What changed

• Added Retry-After parsing for Slack responses.
• Added bounded exponential backoff + jitter for 429 and ratelimited responses.
• Added structured retry/exhaustion logs with attempt and next retry timestamp.
• Added focused unit tests for retry-after parsing and delay computation.

Why this resolves the issue

• History polling now backs off according to Slack guidance instead of immediately dropping context on first limit hit.

Validation

• cargo fmt --all
• cargo test channels::slack::tests:: -- --nocapture

PR

• fix(slack): honor Retry-After for conversations.history rate limits #1873

[theonlyhennygod]

Closed via merged PR #1873 to main (Slack conversations.history Retry-After backoff + retry handling).