Skip to content

Use public Blob URLs for large source ingestion#28

Merged
suguanYang merged 1 commit into
stagingfrom
fix/public-blob-source-ingest
May 10, 2026
Merged

Use public Blob URLs for large source ingestion#28
suguanYang merged 1 commit into
stagingfrom
fix/public-blob-source-ingest

Conversation

@suguanYang
Copy link
Copy Markdown
Contributor

Summary

  • switch large source staging uploads back to the current public Vercel Blob store so the browser multipart upload matches the deployed Blob token/store access
  • keep staged Blob paths unguessable with per-upload UUID paths plus Vercel's random suffix, and avoid original file names in the Blob pathname
  • send the public Blob URL to Knowhere as a URL parse job instead of downloading/re-uploading through the Next.js server
  • persist staged Blob metadata on parsing sources, then delete and clear it once reconciliation sees the Knowhere job complete or fail
  • add immediate best-effort Blob cleanup when URL job creation or metadata handoff fails

Verification

  • pnpm test
  • pnpm exec tsc --noEmit
  • pnpm lint
  • pnpm build
  • git diff --check

@vercel
Copy link
Copy Markdown

vercel Bot commented May 10, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
knowhere-notebook-staging Ready Ready Preview, Comment May 10, 2026 11:23am

Request Review

@suguanYang suguanYang merged commit 0729450 into staging May 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant