Skip to content

fix(files): index generated docs without compiling and fix the docx sandbox bundle - #7386

Merged
icecrasher321 merged 2 commits into
stagingfrom
fix/search-index-doc-compile
Sep 2, 2026
Merged

fix(files): index generated docs without compiling and fix the docx sandbox bundle#7386
icecrasher321 merged 2 commits into
stagingfrom
fix/search-index-doc-compile

Conversation

@icecrasher321

@icecrasher321 icecrasher321 commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Workspace file search indexing no longer compiles generated documents. The indexer reads the compiled artifact when one exists and otherwise indexes the generation source as text, through the same read-only resolver the public share route uses. It previously went through the compile-or-load download path, which inside a Trigger.dev worker took the isolated-vm fallback and tried to run Python and Node-style document sources as sandbox JavaScript.
  • Parser failures in the indexer follow the file tool's policy: a structured parser rejecting UTF-8 bytes (a .json file wrapped in a markdown code fence, for example) falls back to raw text, and rejecting binary bytes marks the revision skipped instead of retrying three times and failing it.
  • The docx sandbox bundle threw ReferenceError: __require is not defined on load since the docx 9.7.x bump: docx inlines JSZip's UMD together with esbuild's __require helper, and Bun's browser/iife build rewrites the bare require references inside it to a runtime helper it never emits. _polyfills.ts defines the stub, build.ts evaluates every bundle in a bare node:vm context before writing it, and a test loads the checked-in bundles the same way. All three bundles are regenerated.
  • trigger.config.ts syncs the doc-sandbox template variables and the copilot artifact bucket, so workers read generated documents through the compiler the app authored them for and can load the compiled artifacts. The values still have to exist in the Trigger.dev environment.
  • A cancelled workspace-file download now surfaces as the abort instead of being rewrapped as a transport failure.

Type of Change

  • Bug fix

Testing

  • bun run test for the new extract.test.ts, verify.test.ts, and workspace-file-manager-download.test.ts, plus the uploads, doc-compile, serve-route, and workspace-files suites
  • Bare node:vm smoke: real docx, pdf, and pptx documents generated through the rebuilt bundles
  • type-check, lint, check:audits, boundary and egress checks

Revisions that already failed are terminal (the dispatcher only re-claims pending), so once the fixed worker is live and the worker environment has the doc template and copilot bucket, re-drive them:

UPDATE workspace_file_search_index
SET status = 'pending', dispatched_at = NULL, failure_reason = NULL, updated_at = now()
WHERE status = 'failed' AND failure_reason = 'indexing_error';

INSERT INTO workspace_file_search_dispatch_queue (workspace_id, enqueued_at, updated_at)
SELECT DISTINCT workspace_id, now(), now()
FROM workspace_file_search_index
WHERE status = 'pending' AND dispatched_at IS NULL
ON CONFLICT (workspace_id) DO UPDATE SET updated_at = now();

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

🤖 Generated with Claude Code

@vercel

vercel Bot commented Sep 2, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Sep 2, 2026 5:07am UTC

Request Review

@greptile-apps

greptile-apps Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR changes workspace-file indexing to resolve generated documents without compiling them, adds parser fallback and cancellation handling, validates regenerated document sandbox bundles, and synchronizes document-runtime storage configuration.

  • Introduces a read-only extraction path for stored files, generated-document artifacts, and generation source text.
  • Falls back to valid UTF-8 text when structured parsing fails while skipping rejected binary content.
  • Adds bare-context validation for the checked-in DOCX, PDF, and PPTX sandbox bundles.
  • Propagates download cancellation and synchronizes document-sandbox environment variables into Trigger.dev workers.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains in the eligible follow-up scope.

No blocking failure remains.

Important Files Changed

Filename Overview
apps/sim/lib/workspace-files/search/extract.ts Adds bounded, cancellation-aware resolution and text extraction without compiling generated documents.
apps/sim/lib/workspace-files/search/indexing.ts Replaces the compile-or-load indexing path with the new read-only extraction helpers.
apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts Forwards abort signals to storage downloads and preserves cancellation errors.
apps/sim/lib/execution/sandbox/bundles/_polyfills.ts Defines the missing sandbox bundle runtime helper with a fail-closed dynamic-require implementation.
apps/sim/lib/execution/sandbox/bundles/build.ts Evaluates each generated bundle in a bare VM context before writing it.
apps/sim/lib/execution/sandbox/bundles/verify.ts Adds reusable bare-context bundle evaluation and export-registration validation.
apps/sim/trigger.config.ts Synchronizes document-sandbox templates and copilot artifact storage settings to workers.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  I[Workspace file indexer] --> R[Read stored bytes]
  R --> D{Generated document?}
  D -->|Compiled artifact exists| A[Read artifact]
  D -->|Artifact unavailable| S[Index generation source as text]
  D -->|Ordinary file| P[Parse stored bytes]
  A --> P
  P -->|Parsed content| X[Bound and index text]
  P -->|Parser rejects UTF-8| T[Index raw text]
  P -->|Parser rejects binary| K[Mark revision skipped]
Loading

Reviews (2): Last reviewed commit: "fix(files): surface download aborts and ..." | Re-trigger Greptile

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 12 files

Not reviewed (too large): apps/sim/lib/execution/sandbox/bundles/pptxgenjs.cjs (~125 lines), apps/sim/lib/execution/sandbox/bundles/docx.cjs (~42 lines), apps/sim/lib/execution/sandbox/bundles/pdf-lib.cjs (~42 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts
Comment thread apps/sim/lib/workspace-files/search/indexing.ts
@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@greptile

@icecrasher321

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@icecrasher321 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 13 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Not reviewed (too large): apps/sim/lib/execution/sandbox/bundles/pptxgenjs.cjs (~125 lines), apps/sim/lib/execution/sandbox/bundles/docx.cjs (~42 lines), apps/sim/lib/execution/sandbox/bundles/pdf-lib.cjs (~42 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.

Re-trigger cubic

@icecrasher321
icecrasher321 merged commit b967a3b into staging Sep 2, 2026
29 checks passed
@icecrasher321
icecrasher321 deleted the fix/search-index-doc-compile branch September 2, 2026 05:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant