feat(knowledge): add Live sync option to KB connectors + fix embedding billing

[waleedlatif1]

Summary

• Adds a "Live" sync option (every 5 minutes) to KB connector modals, gated to Max and Enterprise plan users
• Disabled with a "Max" badge for free users — same badge pattern as Sim Mailer in settings
• Server-side 403 guard on both POST and PATCH if a sub-hourly interval is submitted without a Max plan
• Self-hosted and dev environments bypass the gate
• Fixes missing embedding cost billing: `generateEmbeddings` now returns `totalTokens`, `isBYOK`, and `modelName`; `processDocumentAsync` calls `recordUsage` after successful embedding insertion
• BYOK workspaces are excluded from billing; Azure (system-level key) is billed correctly
• Uses `calculateCost` + `getCostMultiplier` consistent with how billing works across the rest of the platform

Type of Change

• New feature
• Bug fix

Testing

Tested manually

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[cursor]

PR Summary

Medium Risk
Adds subscription-gated behavior to connector creation/updates and introduces new billing writes based on embedding token usage, which can affect customer entitlements and cost accounting if incorrect.

Overview
Adds a "Live" (5-minute) sync frequency option for knowledge base connectors, shown in the add/edit connector modals and disabled for non-Max plans with a MaxBadge.

Enforces the same gate server-side by rejecting connector POST/PATCH requests that set a sub-hourly syncIntervalMinutes unless hasLiveSyncAccess allows it (self-hosted bypass).

Fixes embedding billing by extending generateEmbeddings to return { embeddings, totalTokens, isBYOK, modelName }, updating callers accordingly, and recording knowledge-base model usage after successful document embedding insertion (skipping BYOK), including a DB enum migration for the new usage-log source.

^{Reviewed by Cursor Bugbot for commit 80e97aa. Configure here.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Apr 3, 2026 9:51pm

[greptile-apps]

Greptile Summary

This PR adds a "Live" (every 5 minutes) sync option to Knowledge Base connector modals, gated behind Max/Enterprise plan access both client-side (disabled button + MaxBadge) and server-side (403 guard in POST and PATCH handlers). It also fixes a billing gap where embedding costs were never recorded: generateEmbeddings now returns totalTokens, isBYOK, and modelName, and processDocumentAsync calls recordUsage + checkAndBillOverageThreshold after a successful embedding insert.

Key changes:

• SYNC_INTERVALS extracted to a shared consts.ts and the "Live" (5 min) entry added with requiresMax: true; MaxBadge component extracted to max-badge.tsx (resolves previous review comment)
• hasLiveSyncAccess added to subscription.ts, mirroring hasInboxAccess logic but bypassing on !isHosted (self-hosted environments) rather than !isProd
• generateEmbeddings return type changed from number[][] to GenerateEmbeddingsResult; all call sites updated (chunks service, docs-chunker, document service)
• Billing in processDocumentAsync: tokens accumulated across batches, isBYOK/modelName captured once on the first batch; billing skipped for BYOK workspaces; a logger.warn is emitted when cost resolves to zero
• DB migration 0185 adds knowledge-base to the usage_log_source Postgres enum
• Tests updated to include usage in mock API responses via the shared mockNextFetchResponse helper

Confidence Score: 5/5

Safe to merge — no critical or blocking issues found; all three previous P1 concerns have been addressed.

All three P1 findings from the prior review round (billing metadata discarded in DocsChunker, isBYOK/modelName overwritten per batch, silent billing miss for Azure custom deployment names) have been resolved. The billing flow is well-structured, the server-side plan gate is consistent between POST and PATCH, the DB migration is correct, and tests were updated to cover the new usage field in mock responses.

No files require special attention.

Important Files Changed

Filename	Overview
apps/sim/lib/knowledge/embeddings.ts	Refactored generateEmbeddings to return a GenerateEmbeddingsResult struct; callEmbeddingAPI now returns { embeddings, totalTokens } from data.usage.total_tokens; all callers updated.
apps/sim/lib/knowledge/documents/service.ts	Adds embedding-cost billing to processDocumentAsync: accumulates totalEmbeddingTokens across batches, calls recordUsage and checkAndBillOverageThreshold after successful processing; BYOK workspaces excluded; missing pricing entries emit a logger.warn.
apps/sim/lib/billing/core/subscription.ts	Adds hasLiveSyncAccess gating sub-hourly sync to Max/Enterprise plans; also parallelises getHighestPrioritySubscription + getEffectiveBillingStatus calls inside hasInboxAccess.
apps/sim/app/api/knowledge/[id]/connectors/route.ts	POST handler enforces live-sync gate: if syncIntervalMinutes is 1–59 and user lacks Max/Enterprise, returns 403.
apps/sim/app/api/knowledge/[id]/connectors/[connectorId]/route.ts	PATCH handler adds the same live-sync gate, guarded by parsed.data.syncIntervalMinutes !== undefined before the range check.
apps/sim/app/workspace/[workspaceId]/knowledge/[id]/components/consts.ts	New shared SYNC_INTERVALS constant extracted from both modal files; adds Live (5 min) with requiresMax: true.
apps/sim/app/workspace/[workspaceId]/knowledge/[id]/components/max-badge.tsx	New shared MaxBadge component extracted to eliminate markup duplication between add and edit connector modals.
packages/db/migrations/0185_new_gravity.sql	Migration adds knowledge-base to the usage_log_source Postgres enum via standard ADD VALUE.

Sequence Diagram

sequenceDiagram
    participant UI as Connector Modal
    participant API as POST /connectors
    participant Sub as hasLiveSyncAccess
    participant DB as Database
    participant Embed as generateEmbeddings
    participant Billing as recordUsage

    UI->>API: POST { syncIntervalMinutes: 5 }
    API->>Sub: hasLiveSyncAccess(userId)
    Sub-->>API: true/false
    alt not Max/Enterprise
        API-->>UI: 403 Live sync requires Max
    else allowed
        API->>DB: Insert connector
        API-->>UI: 201 Created
    end

    Note over DB,Billing: Background: processDocumentAsync
    DB->>Embed: generateEmbeddings(texts, model, workspaceId)
    Embed-->>DB: { embeddings, totalTokens, isBYOK, modelName }
    DB->>DB: Insert embeddings + mark completed (transaction)
    alt !isBYOK && totalTokens > 0 && userId
        DB->>Billing: recordUsage({ cost, tokens })
        Billing->>Billing: checkAndBillOverageThreshold(userId)
    end

Loading

_{Reviews (4): Last reviewed commit: "fix(knowledge): call checkAndBillOverage..." | Re-trigger Greptile}

[waleedlatif1]

@greptile

[waleedlatif1]

@cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 144b056. Configure here.}

[waleedlatif1]

@greptile

[waleedlatif1]

@cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit e92e3c9. Configure here.}

[waleedlatif1]

@greptile

[waleedlatif1]

@cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 80e97aa. Configure here.}