Skip to content

v0.7.44: nextjs upgrade, CSV imports to tables fixes, tombstone for KB docs, mcp fixes#5903

Merged
waleedlatif1 merged 14 commits into
mainfrom
staging
Jul 23, 2026
Merged

v0.7.44: nextjs upgrade, CSV imports to tables fixes, tombstone for KB docs, mcp fixes#5903
waleedlatif1 merged 14 commits into
mainfrom
staging

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

icecrasher321 and others added 14 commits July 22, 2026 23:43
…rise pages (#5887)

* improvement(landing): keyword-forward SEO copy on the home and enterprise pages

* fix(landing): sync homepage JSON-LD title and restore visible open-source claim
* chore(deps): bump next to 16.2.11 to clear security advisories

Patches SSRF, cache confusion, DoS, and middleware-bypass advisories
(GHSA-89xv-2m56-2m9x et al.) affecting next < 16.2.11 across apps/sim,
apps/docs, and packages/emcn. Excludes next/@next/env from the
minimum-release-age gate until the 7-day window elapses on 2026-07-28.

* chore(deps): drop aged-out typescript entries from release-age excludes

typescript and @typescript/typescript6 passed the 7-day minimum-release-age
gate (aged out 2026-07-15 and 2026-07-13), so their exclusions are no longer
needed. Keeps @typescript/native-preview (permanent nightly builds) and the
Pi packages (age out 2026-07-24).
…ssage (#5891)

Shopify custom-app token verification faithfully surfaces a real 401 from
Shopify, but the generic 'double-check it in Shopify' copy didn't tell users
what to check. The #1 real cause is pasting the wrong secret (API key / API
secret key) instead of the shpat_ Admin API access token, or using a token
bound to a different store.

- Add an optional per-provider invalidCredentialsHelp override on the token
  service-account descriptor; set it for Shopify to name the exact fix.
- Move the error-code to message mapping out of the shared connect modal into
  the descriptor module (getTokenServiceAccountErrorMessage) so provider copy
  is inherited from the definition rather than hard-coded in the modal.
- Add unit tests for the mapper (override, fallback, all codes).
…mport (#5888)

* fix(tables): sniff CSV delimiter and capture the real header row on import

Table CSV import derived the delimiter purely from the file extension, so
semicolon-delimited exports (European-locale Excel) parsed as a single column.
Header derivation also read Object.keys(rows[0]), so with relax_column_count a
ragged/sparse first data row dropped its trailing columns from schema inference.

- Add detectCsvDelimiter: trial-parse the head against ,/;/tab/| and pick the
  candidate with the most columns (tie-break on row-width consistency), quote-aware
  so separators inside quoted cells don't skew the guess; extension is now only the
  fallback
- Add sniffCsvDelimiterFromStream: memory-bounded (64KB) peek-then-replay for the
  streaming import paths so multi-GB files sniff without buffering
- Capture the true header row via the csv-parse columns callback on every path,
  fixing dropped columns when the first row is short
- Wire into the create route, append/replace route, background runner, client
  dialog preview, and parseFileRows (copilot)

* fix(tables): rank CSV delimiter by row consistency and bound the peek buffer

Addresses review findings on the delimiter sniffer:

- Rank candidates by row-width consistency first (modal column count), then column
  count. Ranking on column count alone let a comma that appears in a semicolon
  file's unquoted header win over the real separator; the wide-but-ragged split
  now loses to the uniform one.
- Move the partial-trailing-line trim into detectCsvDelimiter so every path
  (stream, client preview, parseFileRows) prepares the sniff sample identically
  and can't disagree on the delimiter.
- Cap the stream peeker's detection copy at the sniff window via Buffer.concat's
  length arg and replay the buffered chunks by reference, so an oversized source
  chunk can't break the bounded-memory contract.

* fix(tables): score CSV delimiter by width*consistency and dedupe headers

Addresses round-2 review findings:

- Score each delimiter candidate by modalWidth * consistency instead of
  consistency alone. Consistency-only let a separator that appears uniformly
  inside values (e.g. a pipe once per row) beat a real delimiter whose rows are
  legitimately ragged; the product rewards a split that is both wide and uniform,
  with width breaking ties. Genuinely symmetric files (two valid columns under
  either separator) fall back to the global-default candidate order.
- Dedupe the captured header row to match the parser's record keys. csv-parse
  collapses duplicate column names onto one key (last value wins), so reporting
  the raw duplicates made schema inference invent a phantom empty column and
  could fail mapping validation. dedupeHeaders keeps first-occurrence order,
  case-sensitively, matching the emitted keys.

* fix(tables): keep the final row when sniffing a complete CSV

detectCsvDelimiter always trimmed to the last newline, so a complete file with
no trailing newline lost its final data row from scoring — leaving the header
alone and letting a header-widening separator beat the real one. It now takes a
`complete` flag: the trim runs only for truncated prefixes, where a mid-record
cut must be dropped. The stream sniffer passes it from `exhausted`, and the
buffered callers (client preview, parseFileRows) pass it when the sample covers
the whole file.

* fix(tables): keep the double-cast annotation adjacent to its cast

Hoist the csv-parse options out of the multi-line parse() call so the
`// double-cast-allowed` annotation sits directly above the `as unknown as`
token — the strict boundary audit only matches an annotation within three lines
of the cast. Also simplify the stream sniffer's completeness check to `exhausted`
(the loop only ends early on end-of-stream, so it already means the whole file
fit the sniff window).

* fix(tables): observe EOF at the exact sniff-window boundary

The stream sniffer's read loop stopped as soon as the buffered size reached the
sniff window, so a file whose size is exactly the window never triggered the
extra read that observes end-of-stream — it was judged a truncated prefix and
dropped its final newline-less row, disagreeing with the buffered callers (which
mark that size complete). Read while size <= window so the boundary case sees EOF
and `exhausted` (hence `complete`) is set correctly. Regression test added.

* test(tables): use sleep helper instead of raw setTimeout promise
…v2 block (#5892)

* feat(slack): native Sim app trigger mode behind the preview-gated slack_v2 block

Restore the native Sim Slack app mode for the slack_oauth trigger using a
single-credential-picker design. The trigger is only reachable through the
preview-gated slack_v2 block, so the mode inherits that gate — no separate env
flag.

- Re-add SIM_SUBSCRIBED_EVENTS / SLACK_SIM_EVENT_OPTIONS derived exports.
- Merge the trigger credential picker (credentialKind: 'any') so it lists Sim
  OAuth accounts and reusable custom bots together, mirroring the slack_v2 block.
- Event dropdown narrows to the Sim app's subscribed events when an OAuth
  account is selected, all events for a custom bot (resolved client-side from
  the warmed credential list).
- Deploy branch discriminates by the RESOLVED credential, not a UI field: a bot
  credential routes by credential id (custom app); otherwise validate the event
  against SIM_SUBSCRIBED_EVENTS, resolve the credential owner's token, derive
  routingKey from Slack team_id (auth.test), and route on the shared Sim app.
- The ingest endpoint and team_id fan-out routing already existed on staging.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018asmKsWQ5Vi7T7wD9uHofz

* fix(slack): address review — set credentialId, scope OAuth path to workspace, drop event narrowing

Review round 1 (Greptile 4/5 + Cursor Bugbot):

- deploy.ts: the native Sim-app (OAuth) branch now sets providerConfig.credentialId
  (runtime token resolution in the slack provider and credential-disconnect cleanup
  both key slack_app rows on it — without it, file downloads / reaction text fail and
  disconnect leaves the webhook active).
- deploy.ts: resolve the OAuth credential through resolveTriggerCredentialId
  (workspace- and oauth-scoped) so a pasted foreign/other-tenant credential id can't
  bind to the workflow; use the resolved canonical id for token owner lookup + routing.
- deploy.ts: a deleted/secretless custom bot credential (getSlackBotCredential → null
  but a service_account row exists) now returns the "reconnect the bot" error instead
  of the misleading "connected Slack account" message.
- oauth.ts: drop the credential-based event-option narrowing. The shared dropdown
  framework doesn't revalidate a stored value when its dependency changes, so switching
  a custom bot → Sim account left an orphaned event that failed deploy with a 400. The
  event picker now offers all events; the deploy path is the authoritative gate.
- tests: add broken-bot and workspace-not-resolvable cases; assert credentialId is set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018asmKsWQ5Vi7T7wD9uHofz

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ombstone (#5884)

* fix(knowledge): replace deletion-safety heuristics with a two-phase tombstone

PR #5883 merged an interim fix (sourceConfirmedEmpty bypassing two
static safety heuristics) before this follow-up redesign was ready.
This supersedes that approach with a properly general fix, matching
how production sync systems (Entra Connect, SCIM/Entra ID
deprovisioning, Cassandra/Couchbase tombstones) handle this exact
problem: never let a single observation trigger an irreversible mass
deletion, no matter how confident the signal looks.

A document missing from a normal sync's listing is now soft-deleted
(marked pending-removal) rather than hard-deleted immediately. It's
only actually purged once a *later* sync confirms it's still absent.
If it reappears in between, it's resurrected automatically — this
self-heals a transient outage or a bad API response without needing
to distinguish 'real' emptiness from 'ambiguous' emptiness at all,
which is what the removed heuristics were trying (and failing) to do
from a single observation.

This removes shouldSkipEmptyListing, exceedsDeletionSafetyThreshold,
and sourceConfirmedEmpty entirely — Google Sheets no longer needs a
connector-specific bypass flag, since a genuinely trashed spreadsheet
now reconciles through the exact same general path as every other
connector, with no special-casing and no new misuse surface for
future connectors. A forced fullSync still purges everything absent
in one pass, preserving the existing 'trigger a full sync to force
cleanup' escape hatch.

Uses the existing (previously unused for individual documents)
document.deletedAt column as the tombstone marker — no schema
migration required. shouldReconcileDeletions (the isIncremental /
listingCapped / listingTruncated gate) is unchanged; it still governs
whether reconciliation may run at all. Resurrection runs
unconditionally even when that gate is closed, since presence is
trustworthy evidence regardless of whether the listing was complete.

* fix(knowledge): resurrect atomically on content update, sweep tombstoned docs on connector teardown

Two real gaps found by review, both stemming from the same root
cause: other code paths assumed deletedAt IS NULL means 'the only
real rows' and were never updated for the new tombstone semantics.

- updateDocument's guard required deletedAt IS NULL, so a tombstoned
  document reappearing with CHANGED content failed its content
  update (rejected by the guard) while the separate resurrect step
  still cleared deletedAt regardless — the document became active
  again but kept serving stale pre-tombstone content. Fixed by
  clearing deletedAt as part of the same update statement and
  dropping the guard, so content and resurrection land atomically.
- Both connector-teardown cleanup paths (the ConnectorDeletedException
  handler in sync-engine.ts, and the connector DELETE API route) only
  swept documents with deletedAt IS NULL, so pending-removal documents
  escaped cleanup entirely and were orphaned once their connector was
  gone. Fixed by including tombstoned docs in both sweeps — there's no
  future sync left to confirm or resurrect them once the connector
  itself is deleted.

* fix(knowledge): don't resurrect a tombstoned document whose content refresh failed

Critical race found by adversarial audit: resurrectIds was derived
purely from 'externalId was seen in the listing', independent of
whether the paired content update actually succeeded. If updateDocument
threw (hydration failure, storage upload failure, or any other
transient error) or the deferred-hydration fetch itself failed, the
document's row was never touched — yet the separate unconditional
resurrect step still cleared deletedAt for it anyway, reproducing the
exact bug this PR fixes (visible again, serving stale pre-tombstone
content), just triggered by a failed write instead of a gated one.

Track externalIds whose refresh attempt failed (hydration rejection or
write rejection) and exclude them from partitionSyncReconciliation's
resurrectIds. A failed refresh leaves the document tombstoned as-is —
not soft-deleted, not hard-deleted, not resurrected — so a later sync
gets a clean retry instead of the row landing in an inconsistent state
either way.

* fix(knowledge): exclude fulfilled-but-unverified hydration outcomes from resurrection too

Both bots independently found a third instance of the same bug class:
when deferred hydration for an update fulfills but has no usable
content (skipped as oversized, or an empty re-fetch), the code falls
back to keeping the stored content as last-known-good and counts it
unchanged — correct for an already-visible document, but for a
tombstoned one it means content was never actually verified as
current. That fallback wasn't added to failedExternalIds, so
reconciliation still resurrected it with stale pre-tombstone content
despite hydration never actually confirming anything.

Fixed by adding both fallback branches to failedExternalIds, same as
the rejected-promise cases. Verified across every connector's
skippedReason call site that this only ever happens inside
getDocument (the deferred hydration path already covered here) —
no connector sets skippedReason directly in listDocuments, so there's
no equivalent listing-time gap to fix.

* fix(knowledge): force a full listing when pending-removal documents exist

A subtler instance of the same bug class, this time architectural
rather than a code-path gap: an incremental listing only includes
documents whose content changed since the last sync. A tombstoned
document that's still genuinely present at the source but unchanged
would never appear in an incremental delta at all, so it could never
be resurrected — and on a connector that runs incrementally from here
on (its normal syncMode), it would stay tombstoned indefinitely with
no self-correcting path, only a manual full resync.

Added shouldRunIncrementalSync (extracted as a testable pure function,
matching shouldReconcileDeletions' existing pattern) and a cheap
existence check for any pending-removal document on this connector.
Whenever one exists, this sync forces a full listing instead of an
incremental one, guaranteeing every tombstoned document gets a real
resurrect-or-confirm decision. This only affects which listing mode
runs — it doesn't touch options.fullSync, so the deletion-safety grace
period for other, unrelated documents in the same sync is unaffected.

* fix(knowledge): bound tombstone-forced full syncs, resurrect kept docs on connector delete

Two more real findings from this review round:

- A document whose refresh keeps failing every sync (e.g. permanently
  oversized) never resurrects and never hard-deletes (it's present in
  the listing, just unreadable) — correct on its own, but it also never
  stops being counted by hasTombstonedDocs, so it would force a full
  listing for this connector forever, permanently disabling incremental
  sync on account of one stuck document. Bounded the check to the same
  RETRY_WINDOW_DAYS already used for the stuck-document retry sweep
  below: past the window, this connector stops forcing full syncs on
  the stuck document's account. The document itself is unaffected —
  it stays tombstoned either way, matching the existing 'last-known-good
  forever' tolerance this codebase already accepts for any document
  whose hydration keeps failing.

- The connector-DELETE route's deleteDocuments=false path (kept docs)
  counted tombstoned documents but never resolved them one way or the
  other. With the connector gone, there's no future sync left to ever
  confirm or resurrect them, so they'd become permanent invisible
  orphans holding storage forever. Since 'kept' documents become normal
  standalone KB entries once detached from their connector, resurrect
  any pending-removal ones as part of that transition — consistent
  with what happens to their non-tombstoned sibling documents.

* fix(knowledge): serialize reconciliation writes against a concurrent connector delete

An independent adversarial audit (not just Greptile/Cursor) found the
one genuinely critical gap 6 rounds of bot review missed: resurrect/
soft-delete/hard-delete writes applied raw document IDs snapshotted at
the top of the sync, with no re-check immediately before the write. A
connector-DELETE request choosing to keep documents detaches them
(connectorId set to NULL) via the exact same FOR UPDATE lock on the
connector row that this fix now also takes before applying any
reconciliation write — serializing the two: whichever transaction
commits first wins, and the loser's re-check sees the up-to-date
connectorId and skips any document the other request already claimed.
Without this, a sync racing a 'delete connector, keep documents'
request could silently resurrect-then-strand or soft/hard-delete a
document the user explicitly chose to keep, with a secondary effect of
misclassifying it for storage-billing decrement (which keys off
whether connectorId is still set).

Also tightened the excludedDocs query: it previously required
deletedAt IS NULL, so a document that was both userExcluded and
tombstoned (reachable via excludeConnectorDocuments, which has no
deletedAt filter) fell out of the exclusion set and could be silently
un-excluded and re-indexed on reappearing. Dropped that requirement so
userExcluded is honored regardless of tombstone state, consistent with
how existingDocs/tombstonedDocs are already merged for classification.

Documented (not code-changed) the remaining lower-severity finding: a
document that outlives the 7-day hasTombstonedDocs bound on a
persistently-incremental connector can stay unresolved indefinitely.
Deliberately not hard-deleting it after the window expires — that
would delete a document with no positive evidence it's actually gone,
reintroducing the exact risk this whole design exists to avoid. It's
already fully excluded from search/billing/listings either way, so
this is an accepted, bounded, orphaned-row trade-off, not a
correctness or security issue.

* fix(knowledge): close remaining hard-delete race window, guard listing-time skip/drop resurrection

Two more real findings, both closing gaps in the previous round's fixes:

- The FOR UPDATE lock protected resurrect/soft-delete (applied inside
  the same transaction) but hardDeleteDocuments still ran after that
  transaction committed, using IDs snapshotted under the lock. A
  concurrent 'delete connector, keep documents' request could still
  detach those same documents in the gap between commit and the
  hardDeleteDocuments call. Added an optional expectedConnectorId
  parameter to hardDeleteDocuments/hardDeleteDocumentBatch — when
  provided, it re-verifies connectorId at the moment of the actual
  delete query, not just the caller's earlier snapshot. Every other
  caller is unaffected (parameter is optional, defaults to no filter).

- Two more listing-time paths could resurrect a tombstoned document
  without ever verifying its content: a listing-time skippedReason
  short-circuits classification straight to 'unchanged' before the
  hash comparison ever runs, and empty non-deferred content classifies
  as 'drop' unconditionally regardless of hash. Both are now added to
  failedExternalIds when reappearing on an existing (possibly
  tombstoned) document, same treatment as the deferred-hydration
  equivalents from the prior round.

* refactor(knowledge): dedupe retry-cutoff computation, extract ownership-filter helper

/simplify pass: hoist the shared RETRY_WINDOW_DAYS cutoff into one computation
reused by both the tombstone-retry bound and the stuck-document retry query,
and pull the FOR-UPDATE-lock reconciliation's ID filtering into a pure,
directly-unit-tested filterStillOwnedReconciliationIds function matching this
file's existing convention for its other decision logic.

* fix(knowledge): re-verify connectorId at the actual hard-delete, fix docsDeleted count

Cursor findings: expectedConnectorId was only checked on the pre-transaction
SELECT in hardDeleteDocumentBatch, not on the DELETE itself — the billing
lookups and KB locking in between are async and can span a concurrent
"delete connector, keep documents" request, so the delete (and its embedding
cleanup) now re-verifies against a fresh in-transaction snapshot instead of
the stale existingIds. Also fixed docsDeleted to use hardDeleteDocuments'
actual returned count instead of the pre-filter candidate count, so a sync
log no longer overreports deletions that expectedConnectorId skipped.

/cleanup: dropped two comments that only restated the line below them.

* fix(knowledge): re-verify connectorId in updateDocument's atomic write

Greptile P1: updateDocument's content-update/resurrect write only checked
document.id and archivedAt, never connectorId — despite connectorId being
a parameter — so a document a concurrent "delete connector, keep documents"
request already detached could still be matched, resurrected, and
overwritten with connector-sourced content after the connector was deleted.
Adds the same connectorId ownership check already used by the reconciliation
transaction and hardDeleteDocuments in this PR.

* fix(knowledge): close connectorId race in the stuck-document retry path

Final independent audit: the stuck-document retry block selected candidate
IDs filtered by connectorId, then reset their processing state and deleted
their embeddings using that stale ID set with no re-check — the same
SELECT-then-write race already patched in updateDocument and
hardDeleteDocumentBatch. A concurrent "delete connector, keep documents"
request could null out connectorId in between, so this now re-verifies
ownership immediately before the embedding delete/document update/re-enqueue
and only acts on documents still owned by this connector.

* fix(knowledge): lock the connector row for stuck-doc retry ownership too

Cursor + Greptile (same root cause, two reports): the previous round's fix
re-checked connectorId via a separate SELECT before the embedding delete and
processing-state reset, but a bare re-SELECT only narrows a TOCTOU window, it
never closes it — a concurrent "delete connector, keep documents" request
could still commit its detach in between. Wraps the ownership re-check and
both writes in a transaction that takes the same knowledge_connector FOR
UPDATE lock the DELETE route takes before nulling connectorId, so the two
requests serialize instead of racing, matching the pattern already used by
the reconciliation transaction elsewhere in this PR.
…kspaces (#5893)

* feat(sidebar): search the workspace switcher when a user has many workspaces

Shows a search input in the workspace dropdown once the list exceeds
WORKSPACE_SEARCH_THRESHOLD (3). ArrowUp/Down move through results, Enter
switches, and the query resets on close. All the search machinery is gated on
showSearch so users with few workspaces get no extra re-renders. The input
reuses the emcn ChipInput chrome (icon prop) rather than hand-rolling the field.

The highlight tracks the highlighted workspace by identity (a stored id, not a
numeric position). An effect keeps that id pinned to a workspace that is actually
in the current results — seeding the first result on open and re-seeding when a
query filters the current one out — so even the default highlight is identity-
stable. A live list change while the menu is open (shrink, grow, or reorder from
a membership change or background refetch) therefore carries the highlight and
Enter along with the same workspace instead of stranding them on whatever now
sits at that row. `activeIndex` derives from the id and is the single source of
truth for Enter, the visual highlight, and the scroll target.

Search and highlight reset via an effect keyed on the menu-open state, so closing
by any path (selecting a workspace, Escape, click-away) clears them — not only
the Radix-driven close that routes through onOpenChange. Hover-highlight is wired
to mousemove rather than mouseenter so a keyboard-driven scrollIntoView can't fire
a synthetic enter that hijacks the keyboard selection, and composition keys are
ignored while an IME is active so confirming a candidate can't switch workspace.

* fix(settings): pad settings-sidebar scroll list so the last item's hover isn't clipped

The scrollable settings-nav container had `pt-1.5` but no bottom padding, so
the last item's rounded `--surface-active` hover pill was clipped against the
container's overflow edge (visible on "Custom blocks", the final Enterprise
item). Give it symmetric vertical padding (`py-1.5`) so the last row gets
equal breathing room. Padding sits on the scroll container, not between
items, so item spacing is unchanged.
…onstant (#5895)

* improvement(shopify): pin Admin API to supported 2025-10 via shared constant

Every Shopify tool and the credential validator hardcoded the retired 2024-10
Admin API version. Retired versions still work (Shopify forward-falls to the
oldest supported version) but the served version drifts silently and emits
deprecation signals.

- Add apps/sim/tools/shopify/constants.ts exporting SHOPIFY_API_VERSION as the
  single source of truth; bump to the supported 2025-10.
- Wire all 21 Shopify tools, the token-service-account validator, and the OAuth
  store route to the shared constant (no more per-file inline version).
- Derive the validator test's expected URL from the constant so it can't drift.

Our operations already use modern 2024-10+ shapes (ProductCreateInput /
ProductUpdateInput new product model, orderCancel new signature, CustomerInput,
InventoryAdjustQuantitiesInput, fulfillmentCreate), none of the removed inline
ProductInput.variants pattern — so the bump is a small, explicit step forward
from the version Shopify already forward-serves.

* chore(shopify): remove dead ShopifySetInventoryParams type

Surfaced by /validate-integration: unexported, unused (no set_inventory tool).
… advertised on v2 only (#5898)

Slack app review has since approved `app_mentions:read`, `assistant:write`, and
`im:history` — they are live in the prod app manifest's `oauth_config.scopes.bot`
— but the repo still had them commented out behind a stale "re-add once approved"
TODO. Sim was therefore requesting a narrower grant than the app is entitled to,
so newly connected accounts got tokens missing exactly the scopes backing three
`simSubscribed` events on the native Sim app trigger: `app_mention`
(app_mentions:read), assistant threads (assistant:write), and DMs (im:history).

The requested scope set now matches the manifest's 20 approved bot scopes exactly.

Scopes are per-credential, not per-block (one Slack app -> one `slack` provider ->
one shared token, requested server-side via getCanonicalScopesForProvider), so the
grant itself cannot be scoped to v2. What is per-block is what each picker
advertises and treats as missing, so:

- slack_v2 + the slack_oauth trigger advertise the full set (they host the native
  Sim app trigger that needs it).
- The legacy v1 block stays pinned to the pre-expansion 17 scopes. It has no
  feature needing the new three, and advertising them there would flag every
  existing Slack credential as missing scopes and prompt a needless reconnect.


Claude-Session: https://claude.ai/code/session_018asmKsWQ5Vi7T7wD9uHofz

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…5896)

* fix(confluence): preserve panel/callout macro semantics through sync

Confluence's rendered view HTML wraps Info/Note/Warning/Tip and custom Panel
macros in divs whose class/color convey meaning that the shared
htmlToPlainText tag-stripper discards along with the tags — a red "do not
use" warning panel becomes indistinguishable from a plain paragraph once
flattened, so RAG has no signal that a bullet under it is an exclusion rule
rather than a normal one.

Adds preserveConfluenceCallouts, a Confluence-specific pre-pass that rewrites
each detected panel into a single bracketed label (e.g. "[WARNING] Do NOT use
this form for: GitLab") before the generic plain-text conversion runs, so the
callout semantic survives both the tag strip and htmlToPlainText's trailing
whitespace collapse. Bumps the connector's content-representation marker so
already-synced pages get one automatic re-hydration under the new extraction,
rather than silently keeping their stale flattened content until their next
edit.

* fix(confluence): preserve word boundaries when extracting callout body text

Greptile P1: cheerio's .text() concatenates every descendant text node with
no separator, so pulling a macro body's text in one call fused adjacent
blocks together (e.g. a paragraph ending in "for:" immediately followed by a
list item "GitLab" became "for:GitLab"), corrupting the exact word boundaries
RAG chunking and keyword matching depend on.

extractBlockJoinedText now extracts each paragraph/list-item/heading/cell/quote
individually and joins them with a single space, keeping every block's text
intact and properly separated, matching how htmlToPlainText already treats
the rest of the page.

* fix(confluence): fix nested-block duplication in callout text extraction

Greptile P1: filtering the found blocks to only top-level ones still wasn't
enough — a nested block (an outer <li> containing its own nested <ul><li>, a
<td> containing a <blockquote>) matched the selector once, but its .text()
call recurses into and flattens its own matched descendants with no
separator, reproducing the exact word-fusion bug one level deeper (and any
duplicate-selection would have double-counted the same text).

Replaces the block-selector approach with a recursive text-node walk:
extractBlockJoinedText now visits every text node individually and joins them
all with a single space, so word boundaries are preserved at every nesting
depth with no double-counting, matching the pattern html-parser.ts already
uses elsewhere in this codebase for the same class of problem.

* fix(confluence): apply the same word-boundary-safe extraction to panel headers

Greptile: panelHeader text extraction was left on the plain .text() call
while panel/macro body extraction was already fixed to use
extractBlockJoinedText, so a rich multi-node header (e.g. <b>Warning:</b>
followed by a sibling <span>) could still fuse into "Warning:Do not use"
with no space. Panel headers now go through the same recursive text-node
walk as bodies, for consistency across every text extraction in this file.

* fix(confluence): distinguish inline formatting from block boundaries in extraction

Greptile: the recursive text-node walk unconditionally inserted a space
between every text node, which fixed block-boundary fusion but broke
genuinely inline-formatted text — "un<b>believe</b>able" became
"un believe able" and "Hello<b>!</b>" became "Hello !", corrupting valid
callout content on its way into the index.

Adds an INLINE_FORMATTING_TAGS allowlist (b, strong, i, em, span, a, etc.):
text flowing through those tags accumulates with no artificial separator,
preserving exact source adjacency, while every other tag boundary (p, li,
td, headings, br, ...) still flushes to a new segment — a block always
implies a break even with no literal whitespace in the source, but an inline
tag never does. Fixed one test that had encoded the old, incorrect
expectation for two genuinely adjacent inline tags with no source whitespace
between them, and added regression tests for mid-word inline formatting,
punctuation attached to an inline tag, and a header with real source spacing.

* fix(confluence): process nested panels/macros innermost-first

Cursor: processing matches in document order (outermost first) read a
nested, not-yet-converted panel/macro as plain body text before it ever got
its own bracketed label, silently dropping the inner callout's type. Worse,
an untitled outer panel's `.find('.panelHeader')` could reach past its own
missing header into a nested panel's header and adopt it as its own title.

Replaces the two independent .each() passes with a loop that converts only
"leaf" macros (no remaining nested macro/panel inside them) and repeats
until none are left. This processes innermost-first, so a nested macro is
already its own bracketed <p> by the time its parent's body/header text is
read, and an untitled outer panel's .find() can no longer reach a header
that isn't its own, since a leaf by definition has no nested panel left to
reach into.

* test(confluence): add explicit regression for the exact reported <br> repro

Formalizes an explicit test for the exact <br>-separated string Greptile's
review cited as broken (verified manually not to reproduce, but wasn't
directly asserted in the suite before this).
…s work under Bun (#5897)

* fix(mcp): stream guarded transport via undici.request so SSE responses work under Bun

undici's fetch exposes its response body as a WHATWG ReadableStream whose
bridge is broken under the Bun runtime the standalone server runs on: headers
arrive but response.body never yields data, hanging every MCP streamable-HTTP
(text/event-stream) tools/list read to its 30s timeout. undici's lower-level
request() returns a Node Readable, which Bun streams natively.

createSsrfGuardedFetchWithDispatcher (the MCP transport builder) now routes
through undiciRequestAsResponse: same guarded Agent + connect.lookup (SSRF
unchanged), request() instead of fetch(), and a hand-rolled Node->Web body
bridge (not Readable.toWeb, which throws ERR_INVALID_STATE on the redirect
body.cancel()). Buffered reads, followRedirectsGuarded, maxResponseSize, and
abort all preserved and verified on both Node and Bun.

* fix(mcp): use a single cast for header-iterable detection to satisfy boundary audit

* fix(mcp): handle URLSearchParams OAuth bodies and copy stream chunks

- toUndiciRequestBody serializes a URLSearchParams body (the MCP SDK's OAuth
  token/refresh exchange sends one); undici.request rejects it otherwise.
- Default content-type application/x-www-form-urlencoded when the caller didn't
  set one (fetch parity).
- nodeReadableToWebStream enqueues a copy (new Uint8Array(chunk)) not a view, so
  undici recycling the pooled source buffer can't corrupt queued chunks.
- Tests for both.

* fix(mcp): decode Content-Encoding on the undici.request transport (fetch parity)

undici.request returns raw bytes; fetch auto-decompresses. Restore that: pipe a
gzip/deflate/br response body through the matching zlib decoder before the WHATWG
bridge and strip content-encoding/content-length. maxResponseSize still caps the
compressed wire bytes; errors forward into the decoder and the source is torn down
when the decoded stream ends or is cancelled. Adds a gzip decode test.

* fix(mcp): guard decoder errors and null-body drain against unhandled crashes

- Attach the stream bridge's error listener before piping into the zlib decoder, so
  a synchronous zlib error (server mislabeling a non-gzip body as gzip) rejects the
  reader instead of crashing the process.
- Attach an error listener before draining a null-body response, and wrap Response
  construction in try/catch that destroys the source (no socket leak) on an
  out-of-range status. Adds an invalid-gzip regression test.
@waleedlatif1
waleedlatif1 requested a review from a team as a code owner July 23, 2026 21:55
@greptile-apps

greptile-apps Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Too many files changed for review. (103 files found, 100 file limit)

Bypass the limit by tagging @greptile-apps to review.

@vercel

vercel Bot commented Jul 23, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 23, 2026 9:55pm

Request Review

@cursor

cursor Bot commented Jul 23, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Changes SSRF-guarded outbound fetch and knowledge sync deletion semantics, both of which can affect production data and external integrations; MCP and connector behavior need careful regression testing.

Overview
Release v0.7.44 bundles infrastructure fixes, knowledge-base behavior changes, and marketing/docs polish.

MCP / outbound HTTP: SSRF-guarded fetch now uses undici.request (with a custom Node→Web stream bridge, gzip decoding, and redirect parity) instead of undici.fetch, so SSE/streaming bodies work on Bun—fixing hung MCP tools/list and similar transports.

Knowledge connectors: Sync drops empty-listing and bulk-deletion safety heuristics in favor of a two-phase tombstone: missing docs are soft-deleted first, hard-deleted on a later sync if still absent, with resurrect when they reappear. Connector deletion clears deletedAt on kept docs (including pending-removal). Confluence adds preserveConfluenceCallouts and bumps content representation to view-callouts for re-hydration. Google Sheets stops setting sourceConfirmedEmpty.

Tables: CSV/TSV import sniffs delimiters from the file head (server routes + import dialog preview) and captures headers via the parser callback.

Product UX: Custom block image icons size via parent [&_img]:size-full wrappers instead of forcing size-full on the <img>. Workspace switcher gains search + keyboard navigation when there are more than three workspaces. Webhook naming: outgoing block is Webhook; generic trigger is Webhook Trigger (blocks, toolbar, docs).

Integrations: Shopify Admin API version is centralized in SHOPIFY_API_VERSION (2025-10); Shopify service-account errors get invalidCredentialsHelp. Slack v2 advertises full trigger scopes; legacy Slack block omits v2-only scopes from required reconnect prompts.

Landing / SEO: Shared HOME_PAGE_TITLE, optional hero definition paragraphs, and refreshed home/enterprise/module copy; new library article on customer-support automation agents.

Deps: Docs app Next.js 16.2.11.

Reviewed by Cursor Bugbot for commit a5e1030. Configure here.

@waleedlatif1
waleedlatif1 merged commit 41bf1c4 into main Jul 23, 2026
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants