feat: migrate integrations from database to file-based architecture

[larryro]

Summary

• Split integration storage into file-based config (config.json + connector.ts) and a slim integrationCredentials DB table for encrypted secrets and runtime state
• Add unified loader, credential CRUD, file I/O actions, shared Zod schema, and update all consumers (agent tools, workflow engine, frontend, OAuth flows, RLS)
• Add organization authorization checks, audit logging, and schema fixes for the new credential model

Test plan

• Verify integrations load correctly from file-based configs
• Test credential CRUD operations (create, update, delete) with org authorization
• Confirm OAuth2 flows work end-to-end with the new credential storage
• Verify agent tools and workflow engine use the new dual-source model
• Check audit logging captures credential mutations with sensitive field redaction
• Run existing integration and workflow tests

Summary by CodeRabbit

Release Notes

• New Features

  • File-based integration configuration system with improved organization and seeding capabilities
  • Enhanced credential management system for better security and separation of concerns

• Improvements

  • Updated integration installation and management workflow for clearer user experience
  • Refined integration terminology: "Install" and "Uninstall" replace "Create" and "Delete" for better clarity

[greptile-apps]

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This pull request refactors the integration system from a database-only architecture to a hybrid file-based and credential-backed model. Integration definitions are now stored as config.json files in the filesystem (under INTEGRATIONS_DIR), with per-credential runtime data (authentication, connection config, sync stats) stored in a new integrationCredentials table. All example integration configs are updated to remove the name field and add an installed boolean. New modules (file_actions.ts, file_utils.ts, load_integration.ts, credential_queries.ts, credential_mutations.ts) provide file I/O and credential management operations. Throughout the codebase, integration lookups migrate from database-only queries (getByName on integrations table) to a hybrid approach (getBySlug on integrationCredentials + file config via loadIntegration). Type references are updated from Doc<'integrations'> to new abstractions like LoadedIntegration and IntegrationWithCredentials. UI components update to use credential records and display installed flags instead of isActive. Environment setup adds INTEGRATIONS_DIR configuration.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related PRs

• refactor(convex): DRY up backend, separate internals, and harden authorization #397: Directly related—both introduce file-based integration actions, credential tables, and unified loadIntegration action; core architectural alignment.
• feat(audit-logs): implement comprehensive audit logging system #272: Related—both refactor integration mutations and action handlers, updating how integrations are created, updated, and deleted.
• feat(platform): integration package upload with OAuth2 and connection testing #432: Related—both modify the Circuly example integration's config.json and connector.ts (main PR adjusts config keys and removes TypeScript types).

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 22.22% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat: migrate integrations from database to file-based architecture' directly and clearly summarizes the primary architectural change made throughout the pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/file-based-integrations

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}