Repository navigation
S3 presigned URLs in messages.attachments / messages.files / toolcalls.attachments expire on read (images 403 after S3_URL_EXPIRY_SECONDS) #13762
Replies: 4 comments
|
Adding cross-links to prior user reports of this same bug for searchability:
Adjacent (fixed already): #11445 — that one was the agent-avatar-only slice and was resolved by the existing |
|
I decided rather than making LibreChat an S3 refresh proxy, which adds significant plumbing for one storage provider, to leverage CloudFront, which is more appropriate given the context of LibreChat (web app, SPA). |
|
Appreciated, thanks for the quick steer — the CloudFront path is clearly the right shape for the AWS context. I'll close #13752. One follow-up worth your take: for Azure deployments where CloudFront isn't applicable, the gap is the same shape (private blob, presigned URL TTL, attachments rendered from message snapshots) but there's no equivalent upstream-shipped CDN strategy today. Is the recommended direction also a CDN-with-signed-cookies model (Azure Front Door / Azure CDN), or is SAS-based read flow still the expected approach for Azure users? Asking because we're considering whether to invest in an Azure Front Door equivalent or stay on SAS for the long term. Either answer is useful — just want to know your thinking before we plan. |
|
CloudFront fixes this for many AWS deployments, but some S3 deployments can't use it for private files:
That last point applies to me, since I'm running a deployment that must comply with GDPR, and all data must stay in EU regions. For these, the S3 strategy still breaks images in messages and agent file lists after I've proposed a fix without refresh-on-read and without extensive S3-specific code in #16912: LibreChat can serve files itself through stable links that never expire. What do you think? |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
What happened?
S3 presigned URLs embedded in persisted message and tool-call snapshots are never refreshed on read. Once a presigned URL expires (default
S3_URL_EXPIRY_SECONDS=120, i.e. 2 minutes), any image or file rendered from those snapshots returns403 SignatureDoesNotMatchand shows as a broken tile in the UI.User-facing symptom: a user generates an image (or uploads one) in a conversation, leaves it, and on return — or on initial load of any older conversation — the image is gone.
The bug affects three persistence sites that all snapshot the full file record (including the SAS-signed
filepath) into denormalized[Mixed]arrays that are never updated after write:messages.attachments[]— tool outputs fromimage_gen_oai, DALL·E, Flux, Stable Diffusion, code execution, etc. (written byapi/server/controllers/agents/callbacks.jsTools_End handler viasaveBase64Image).messages.files[]— user-uploaded files referenced in a message (written bybuildMessageFilesinpackages/api/src/utils/message.ts).toolcalls.attachments[]— code-interpreter /execute_codeartifacts (packages/data-schemas/src/schema/toolCall.ts:10,40), rehydrated by the client throughuseToolCallsMapon conversation open.These snapshots are served verbatim by the read endpoints below. The existing
refreshS3FileUrlschain (packages/api/src/storage/s3/crud.ts:936) is wired only intoGET /api/filesand the avatar dispatchers — it never touches messages or tool calls.Root cause: the following endpoints return persisted documents without re-signing the embedded URLs:
routes/messages.js—GET /(cursor + search branches),GET /:conversationId,GET /:conversationId/:messageId(3 sites).routes/share.js—GET /:shareId(getSharedMessagesresponse).controllers/tools.js—GET /api/agents/tools/calls(getToolCalls).I have a fix prepared — a small refresh-on-read shim that walks
attachments[]/files[]on the response payload and dispatches to the existingrefreshS3Urlper entry. In-memory only (no Mongo writes), per-call memoization keyed onfilepath, errors retain the original URL. Happy to open a PR againstdevif maintainers agree on the approach.Version Information
Reproducible on current🅰️ feat: Native Anthropic Provider for Custom Endpoints").
mainanddev(verified againstdevat commit9efe4878e, "The bug exists in any release with
fileStrategy: "s3"sincerefreshS3FileUrlswas introduced; nothing has wired it into the message or toolcall read paths. Reproducible in self-hosted Docker, source builds, and managed deployments alike.Steps to Reproduce
.envwithfileStrategy=s3, valid S3 creds, andS3_URL_EXPIRY_SECONDS=30(any short TTL; default 120 also reproduces, just takes longer).image_gen_oaiavailable, or use any default chat with image-gen enabled.> S3_URL_EXPIRY_SECONDS(e.g. 35 seconds with the setting above).403 SignatureDoesNotMatchfroms3.amazonaws.com(or your S3-compatible endpoint).Same repro applies to:
messages.files[](upload an image to a message, send, wait, reload).toolcalls.attachments[](run anexecute_codetool that produces a chart, wait, reload — the chart 403s whenuseToolCallsMaprehydrates it).anonymizeMessagesinpackages/data-schemas/src/methods/share.ts:60copies the snapshot at share creation, thenGET /api/share/:shareIdreturns it verbatim).What browsers are you seeing the problem on?
Chrome, Firefox, Safari, Edge — backend issue, browser-independent.
Relevant log output
Screenshots
n/a — broken image tile in conversation history. Happy to attach if useful.
Code of Conduct
Additional context
Why this matters at default settings:
DEFAULT_EXPIRY_SECONDS = 2 * 60inpackages/api/src/storage/s3/s3Config.ts. Any user opening a conversation older than two minutes hits this on every image in that conversation.Adjacent open work: #13740 (Snapshot Files for Shared-Link Attachments) introduces a
fileSnapshots[]model onSharedLinkand applies a read-time URL rewrite ingetSharedMessagesfor share-scoped serving. That work targets a different concern (auth boundaries for anonymous viewers), but it shares the "rewrite URLs on the response" pattern — so coordinating onroutes/share.jswould avoid conflicts when both land.All reactions