Skip to content

[Fix] Cap Lark/Feishu inbound media downloads#81044

Closed
samzong wants to merge 1 commit into
openclaw:mainfrom
samzong:fix/feishu-inbound-media-streaming-cap
Closed

[Fix] Cap Lark/Feishu inbound media downloads#81044
samzong wants to merge 1 commit into
openclaw:mainfrom
samzong:fix/feishu-inbound-media-streaming-cap

Conversation

@samzong
Copy link
Copy Markdown
Contributor

@samzong samzong commented May 12, 2026

Summary

Describe the problem and fix in 2-5 bullets:

If this PR fixes a plugin beta-release blocker, title it fix(<plugin-id>): beta blocker - <summary> and link the matching Beta blocker: <plugin-name> - <summary> issue labeled beta-blocker. Contributors cannot label PRs, so the title is the PR-side signal for maintainers and automation.

  • Problem: Feishu inbound media downloads could fully buffer or read back over-limit resources before saveMediaBuffer rejected them.
  • Why it matters: large inbound media could consume memory or read large temp files even when the configured Feishu media cap should reject the payload.
  • What changed: Feishu response reads now accept maxBytes, reject over-limit Buffer, ArrayBuffer, data, readable stream, async iterable, and writeFile temp-file responses, and inbound message handling passes the same cap into the download helper.
  • What did NOT change (scope boundary): no WhatsApp changes, no provider SDK rewrite, no shared media-store redesign, and no Feishu outbound send behavior changes.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

  • Closes N/A
  • Related N/A
  • This PR fixes a bug or regression

Real behavior proof (required for external PRs)

External contributors must show after-fix evidence from a real OpenClaw setup. Unit tests, mocks, lint, typechecks, snapshots, and CI are supplemental only. Screenshots are encouraged even for CLI, console, text, or log changes; terminal screenshots and copied live output count. Be mindful of private information like IP addresses, API keys, phone numbers, non-public endpoints, or other private details when providing evidence.

  • Behavior or issue addressed: over-limit Feishu inbound media must be rejected while reading the SDK response or before reading a temp file back into memory.
  • Real environment tested: local OpenClaw source checkout on Darwin 25.4.0 arm64, Node v24.14.0 for CLI commands, managed Gateway runtime on Node v25.9.0, pnpm 11.0.8, Feishu default channel connected over WebSocket with real app credentials and channels.feishu.mediaMaxMb = 1.
  • Exact steps or command run after this patch:
    1. Installed/restarted the managed Gateway from this checkout.
    2. Verified the LaunchAgent command points at this PR checkout with pnpm openclaw gateway status --deep.
    3. Verified Feishu is running with pnpm openclaw channels status --deep.
    4. Sent an over-limit document to the configured Feishu bot in a real p2p DM.
    5. Watched /tmp/openclaw/openclaw-2026-05-12.log for Feishu/media/download lines.
  • Evidence after fix (screenshot, recording, terminal capture, console output, redacted runtime log, linked artifact, or copied live output):
$ pnpm openclaw gateway status --deep
Command: /opt/homebrew/bin/node /Users/x/git/openclaw--fix--feishu-inbound-media-streaming-cap/dist/index.js gateway --port 18789
Runtime: running (pid 94522, state active)
Connectivity probe: ok

$ pnpm openclaw channels status --deep
Gateway reachable.
- Feishu default: enabled, configured, running

Live Feishu p2p DM, redacted open_id/chat_id, channels.feishu.mediaMaxMb = 1
2026-05-12T14:18:42.982Z info channels/feishu feishu[default]: received message from ou_[redacted] in oc_[redacted] (p2p)
2026-05-12T14:18:42.997Z info channels/feishu feishu[default]: Feishu[default] DM from ou_[redacted]: <media:document> (ChatGPT Image Apr 23, 2026, 09_06_58 PM.png)
2026-05-12T14:18:43.854Z info channels/feishu feishu: failed to download file media: Error: Feishu message resource download failed: payload exceeds maxBytes 1048576

Supplemental targeted regression proof:

$ node scripts/test-projects.mjs extensions/feishu/src/media.test.ts extensions/feishu/src/bot.test.ts -- --reporter=verbose
[test] starting test/vitest/vitest.extension-feishu.config.ts

RUN  v4.1.5 /Users/x/git/openclaw--fix--feishu-inbound-media-streaming-cap

✓ extensions/feishu/src/media.test.ts > downloadMessageResourceFeishu > rejects buffer-like message resources that exceed maxBytes
✓ extensions/feishu/src/media.test.ts > downloadMessageResourceFeishu > rejects readable message resources when chunks exceed maxBytes
✓ extensions/feishu/src/media.test.ts > downloadMessageResourceFeishu > rejects async-iterable message resources when chunks exceed maxBytes
✓ extensions/feishu/src/media.test.ts > downloadMessageResourceFeishu > rejects over-limit writeFile downloads before reading the temp file
✓ extensions/feishu/src/bot.test.ts > handleFeishuMessage command authorization > uses video file_key (not thumbnail image_key) for inbound video download

Test Files  2 passed (2)
Tests  108 passed (108)
[test] passed 1 Vitest shard in 4.48s
  • Observed result after fix: the live Feishu document hit the bounded message-resource download path and failed with Feishu message resource download failed: payload exceeds maxBytes 1048576, before the older final persistence cap error Media exceeds 1MB limit.
  • What was not tested: live under-limit Feishu media success in this proof pass; the existing tests cover successful bounded reads for in-limit fixtures.
  • Before evidence (optional but encouraged): before switching the managed Gateway to this PR checkout, the same real Feishu over-limit flow failed later with Media exceeds 1MB limit. The targeted command also failed before the production fix with 5 failures: the bot download request had maxBytes: undefined, and the buffer-like, readable, async-iterable, and writeFile over-limit fixtures resolved instead of rejecting.

Root Cause (if applicable)

For bug fixes or regressions, explain why this happened, not just what changed. Otherwise write N/A. If the cause is unclear, write Unknown.

  • Root cause: downloadMessageResourceFeishu did not accept the inbound media cap, and readFeishuResponseBuffer returned fully materialized buffers for Feishu SDK response shapes before any size check.
  • Missing detection / guardrail: Feishu media tests covered response shapes and metadata, but not over-limit streaming, async-iterator, buffer-like, or temp-file read-back paths.
  • Contributing context (if known): saveMediaBuffer had the final persistence cap, which hid the fact that download/read-back had already materialized the payload.

Regression Test Plan (if applicable)

For bug fixes or regressions, name the smallest reliable test coverage that should catch this. Otherwise write N/A.

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file: extensions/feishu/src/media.test.ts, extensions/feishu/src/bot.test.ts.
  • Scenario the test should lock in: Feishu download rejects over-limit buffer-like, stream, async-iterable, and writeFile responses, and inbound media download receives the configured maxBytes before persistence.
  • Why this is the smallest reliable guardrail: the behavior lives at the Feishu SDK response-reader seam and the inbound bot handoff into that seam.
  • Existing test that already covers this (if any): none before this PR.
  • If no new test is added, why not: N/A; new regression tests are included.

User-visible / Behavior Changes

Over-limit Feishu inbound media is rejected earlier during download/read-back instead of only at final media-store persistence.

Diagram (if applicable)

For UI changes or non-trivial logic flows, include a small ASCII diagram reviewers can scan quickly. Otherwise write N/A.

Before:
Feishu SDK response -> full buffer/readFile -> saveMediaBuffer(maxBytes) -> reject

After:
Feishu SDK response -> bounded read/stat(maxBytes) -> reject or buffer -> saveMediaBuffer(maxBytes)

Security Impact (required)

  • New permissions/capabilities? (Yes/No) No
  • Secrets/tokens handling changed? (Yes/No) No
  • New/changed network calls? (Yes/No) No
  • Command/tool execution surface changed? (Yes/No) No
  • Data access scope changed? (Yes/No) No
  • If any Yes, explain risk + mitigation: N/A

Repro + Verification

Environment

  • OS: Darwin 25.4.0 arm64
  • Runtime/container: local source checkout, Node v24.14.0, pnpm 11.0.8
  • Model/provider: N/A
  • Integration/channel (if any): Feishu plugin
  • Relevant config (redacted): test fixture channels.feishu.mediaMaxMb = 1; Feishu app credentials configured locally and redacted; channels.feishu.mediaMaxMb = 1

Steps

  1. Run pnpm test extensions/feishu/src/media.test.ts extensions/feishu/src/bot.test.ts -- --reporter=verbose.
  2. Observe the over-limit Feishu response-shape tests and inbound bot cap-forwarding test.
  3. Run touched-surface lint/format checks for the four changed Feishu files.

Expected

  • Over-limit Feishu SDK response shapes reject before returning a buffer.
  • Inbound bot media download forwards the configured byte cap to downloadMessageResourceFeishu.
  • Touched Feishu files pass lint and format checks.

Actual

  • Targeted Feishu tests passed: 2 files, 108 tests.
  • Touched Feishu files passed oxlint and oxfmt checks.
  • pnpm check:changed passed conflict-marker, changelog attribution, wildcard re-export, duplicate coverage, extension typecheck, and extension-test typecheck gates, then failed on an unrelated pre-existing browser lint issue: extensions/browser/src/cli/browser-cli-state.option-collisions.test.ts:41 no-control-regex.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios: targeted Feishu media/bot tests, changed-lane discovery, touched-file oxlint, touched-file oxfmt, git diff --check, and pre-ship review scan.
  • Edge cases checked: exact cap boundary rejection for buffer-like responses, chunked readable stream overflow, async-iterable overflow, temp-file size check before readFile, and inbound configured mediaMaxMb propagation.
  • What you did not verify: live under-limit Feishu media success in this proof pass.

AI-assisted disclosure

  • This PR was implemented with Codex assistance.
  • I reviewed the diff and ran the verification above from my local checkout.
  • No CHANGELOG.md changes are included.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

If a bot review conversation is addressed by this PR, resolve that conversation yourself. Do not leave bot review conversation cleanup for maintainers.

No existing bot review conversations for this new PR.

Compatibility / Migration

  • Backward compatible? (Yes/No) Yes
  • Config/env changes? (Yes/No) No
  • Migration needed? (Yes/No) No
  • If yes, exact upgrade steps: N/A

Risks and Mitigations

List only real risks for this PR. Add/remove entries as needed. If none, write None.

  • Risk: The Feishu SDK writeFile path still downloads the resource into a temp file before OpenClaw can inspect file size.
    • Mitigation: this PR rejects before reading that temp file back into memory and keeps saveMediaBuffer as the final cap. A deeper SDK-level streaming/cancel path would be a separate change if Feishu exposes a suitable API.

@openclaw-barnacle openclaw-barnacle Bot added channel: feishu Channel integration: feishu size: M proof: supplied External PR includes structured after-fix real behavior proof. labels May 12, 2026
@clawsweeper
Copy link
Copy Markdown
Contributor

clawsweeper Bot commented May 12, 2026

Codex review: needs maintainer review before merge.

Summary
The PR threads the Feishu media byte cap into inbound message-resource downloads and adds bounded-read regression tests for supported SDK response shapes.

Reproducibility: yes. Current main source shows the configured maxBytes is enforced only after Feishu SDK responses are materialized, and the PR body supplies before/after targeted test output plus live over-limit Feishu p2p runtime logs.

Real behavior proof
Sufficient (live_output): The PR body includes copied live output from a real Feishu p2p DM and redacted runtime logs showing an over-limit document rejected by the new maxBytes path after the fix.

Next step before merge
No repair lane is needed because the patch is focused, real behavior proof is supplied, and there are no actionable review findings.

Security
Cleared: The diff adds no dependencies, workflows, permissions, secret handling, or new network calls, and it reduces Feishu inbound resource-consumption exposure.

Review details

Best possible solution:

Merge the focused Feishu plugin fix after required CI and maintainer review pass, keeping saveMediaBuffer as defense-in-depth and leaving SDK-level temp-file download cancellation to a separate follow-up if the SDK exposes it.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main source shows the configured maxBytes is enforced only after Feishu SDK responses are materialized, and the PR body supplies before/after targeted test output plus live over-limit Feishu p2p runtime logs.

Is this the best way to solve the issue?

Yes. Passing the existing cap into the Feishu download seam and checking each supported SDK response shape is the narrow maintainable fix without changing config, shared media-store ownership, or outbound behavior.

What I checked:

  • Current main buffers before the configured cap reaches persistence: Current main reads Buffer, ArrayBuffer, readable stream, async iterable, and writeFile temp-file Feishu SDK responses into memory before saveMediaBuffer enforces maxBytes. (extensions/feishu/src/media.ts:245, 186de9daa0ed)
  • Current inbound caller passes the cap only to final media persistence: resolveFeishuMediaList receives maxBytes and passes it to saveMediaBuffer, but current main does not pass it into downloadMessageResourceFeishu before the download/readback step. (extensions/feishu/src/bot-content.ts:382, 186de9daa0ed)
  • Existing media store cap confirms the requested behavior is not a new config surface: saveMediaBuffer already rejects buffers over maxBytes, so the PR moves an existing Feishu media cap earlier rather than adding a new policy knob. (src/media/store.ts:449, 186de9daa0ed)
  • PR head applies maxBytes at the Feishu response-reader seam: The PR adds checked buffer reads, bounded readable/async-iterable accumulation, writeFile stat checking before readFile, and maxBytes propagation through message-resource fallback downloads. (extensions/feishu/src/media.ts:245, c699b9e03856)
  • PR head forwards the inbound cap into all message-resource calls: The patched bot-content path passes maxBytes for embedded post images, embedded post media, and single inbound media-resource downloads. (extensions/feishu/src/bot-content.ts:382, c699b9e03856)
  • Regression coverage targets the changed seam: The PR adds tests for over-limit buffer-like, stream, async-iterable, and writeFile responses plus cap propagation from inbound video handling. (extensions/feishu/src/media.test.ts:717, c699b9e03856)

Likely related people:

  • steipete: Recent history shows repeated Feishu media, media-store, and test maintenance on the exact files and adjacent media safety paths. (role: recent area contributor; confidence: high; commits: 827b0de0ce74, 252a76d25c93, 5d8f4d87676e; files: extensions/feishu/src/media.ts, extensions/feishu/src/media.test.ts, extensions/feishu/src/bot-content.ts)
  • vincentkoc: Recent Feishu commits touched media filename recovery, voice behavior, and bot-content-adjacent behavior in the same plugin area. (role: adjacent Feishu contributor; confidence: medium; commits: 14312ff5704f, 4c72e605cdc7, b642ebece9b0; files: extensions/feishu/src/media.ts, extensions/feishu/src/bot-content.ts, extensions/feishu/src/media.test.ts)
  • obviyus: Recent history includes Feishu runtime payload seam typing and related refactor work near the inbound media handoff surface. (role: adjacent Feishu refactor contributor; confidence: medium; commits: f248fc8f86c1; files: extensions/feishu/src/bot-content.ts)

Remaining risk / open question:

  • Required CI checks for head c699b9e were still queued or pending at inspection time, so merge should wait for the required checks to finish.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 186de9daa0ed.

@openclaw-barnacle openclaw-barnacle Bot added triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. and removed proof: supplied External PR includes structured after-fix real behavior proof. labels May 12, 2026
@samzong
Copy link
Copy Markdown
Contributor Author

samzong commented May 12, 2026

@clawsweeper re-review

@samzong
Copy link
Copy Markdown
Contributor Author

samzong commented May 12, 2026

@codex review

@clawsweeper
Copy link
Copy Markdown
Contributor

clawsweeper Bot commented May 12, 2026

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 12, 2026
@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 12, 2026
Signed-off-by: samzong <samzong.lu@gmail.com>
@samzong samzong force-pushed the fix/feishu-inbound-media-streaming-cap branch from 1a7827c to c699b9e Compare May 12, 2026 14:48
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 12, 2026
@samzong samzong changed the title [Fix] Cap Feishu inbound media downloads [Fix] Cap Lark/Feishu inbound media downloads May 12, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 12, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 12, 2026
@steipete
Copy link
Copy Markdown
Contributor

Thanks @samzong. I landed this as a broader core media-download refactor on main instead of updating this PR branch, because the right fix ended up being shared Plugin SDK/runtime helpers rather than another Feishu-only cap.

Landed commit: 53d007b
Source PR head: c699b9e

What landed:

  • shared streamed save/read helpers with max-byte enforcement and retry support
  • migrated Feishu, WhatsApp, Line, Slack, Teams, Discord, Telegram, and other plugin media paths onto the shared helpers
  • regression coverage for overflow cancellation, MIME/filename preservation, Slack audio MIME overrides, and Office/generic ZIP detection
  • changelog credit and commit co-author trailer for @samzong

Verification:

  • Blacksmith Testbox check:changed passed: https://github.com/openclaw/openclaw/actions/runs/25803936030
  • pnpm test src/media/fetch.test.ts src/media/store.test.ts extensions/line/src/download.test.ts extensions/slack/src/monitor/media.test.ts -- --reporter=verbose
  • pnpm tsgo:all
  • pnpm plugin-sdk:api:check
  • pnpm check:media-download-helpers
  • git diff --check

Closing this PR because the generalized fix is now on main with your contribution credited.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: feishu Channel integration: feishu proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants