Skip to content

feat(web): customisable AI code review agent configs #1143

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
fatmcgav wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat(web): customisable AI code review agent configs #1143

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
18304b0
feat(web): customisable AI code review agent configs
Apr 21, 2026
561ee96
Address a number of the CodeRabbit findings.
Apr 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
656 changes: 656 additions & 0 deletions docs/api-reference/sourcebot-public.openapi.json
View file
Open in desktop

Large diffs are not rendered by default.

33 changes: 29 additions & 4 deletions docs/docs/features/agents/review-agent.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,11 +137,36 @@ By default, the agent does not review PRs and MRs automatically. To enable autom

You can also trigger a review manually by commenting `/review` on any PR or MR. To use a different command, set `REVIEW_AGENT_REVIEW_COMMAND` to your preferred value (without the leading slash).

# Agent configs

Agent configs let you customise how the review agent behaves per repository, connection, or your whole org. You can override the model, review command, auto-review behaviour, custom prompt, and context files — all without changing environment variables.

Configs are managed on the **Agents** page in the Sourcebot UI. Each config has a **scope**:

- **Repo** — applies to specific repositories (highest priority)
- **Connection** — applies to all repos in a specific connection
- **Org** — applies to all repositories in your org (lowest priority, catch-all)

When a PR or MR arrives, Sourcebot selects the most specific matching config. If no config exists, the agent falls back to global environment variable defaults.

## Custom prompt

Each config can include a custom prompt. Two modes are available:

- **Append** (default) — your instructions are added after the built-in review rules.
- **Replace** — your instructions entirely replace the built-in rules. Use this when you want full control over what the agent looks for.

## Context files

You can configure one or more repository files to be fetched at review time and injected as additional context for the model. This is useful for encoding project-specific conventions that the model should be aware of when reviewing diffs — for example, preferred error handling patterns, style rules, or areas of the codebase that need extra scrutiny.

Set **Context files** in the agent config form to a comma or space separated list of paths relative to the repository root (e.g. `AGENTS.md .sourcebot/review.md`). Files that do not exist in the repository are silently ignored. The files are fetched once per PR from the head commit and included in the context for every diff hunk.

# Environment variable reference

| Variable | Default | Description |
|---|---|---|
| `REVIEW_AGENT_AUTO_REVIEW_ENABLED` | `false` | Automatically review new and updated PRs/MRs |
| `REVIEW_AGENT_REVIEW_COMMAND` | `review` | Comment command that triggers a manual review (without the `/`) |
| `REVIEW_AGENT_MODEL` | first configured model | `displayName` of the language model to use for reviews |
| `REVIEW_AGENT_LOGGING_ENABLED` | unset | Write prompt and response logs to disk for debugging |
| `REVIEW_AGENT_AUTO_REVIEW_ENABLED` | `false` | Automatically review new and updated PRs/MRs. Can be overridden per agent config. |
| `REVIEW_AGENT_REVIEW_COMMAND` | `review` | Comment command that triggers a manual review (without the `/`). Can be overridden per agent config. |
| `REVIEW_AGENT_MODEL` | first configured model | `displayName` of the language model to use for reviews. Can be overridden per agent config. |
| `REVIEW_AGENT_LOGGING_ENABLED` | unset | Write prompt and response logs to disk for debugging. |
63 changes: 63 additions & 0 deletions packages/db/prisma/migrations/20260421203635_add_agent_config/migration.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
-- CreateEnum
CREATE TYPE "AgentType" AS ENUM ('CODE_REVIEW');

-- CreateEnum
CREATE TYPE "AgentScope" AS ENUM ('ORG', 'CONNECTION', 'REPO');

-- CreateEnum
CREATE TYPE "PromptMode" AS ENUM ('REPLACE', 'APPEND');

-- CreateTable
CREATE TABLE "AgentConfig" (
"id" TEXT NOT NULL,
"orgId" INTEGER NOT NULL,
"name" TEXT NOT NULL,
"description" TEXT,
"type" "AgentType" NOT NULL,
"enabled" BOOLEAN NOT NULL DEFAULT true,
"prompt" TEXT,
"promptMode" "PromptMode" NOT NULL DEFAULT 'APPEND',
"scope" "AgentScope" NOT NULL,
"settings" JSONB NOT NULL DEFAULT '{}',
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" TIMESTAMP(3) NOT NULL,

CONSTRAINT "AgentConfig_pkey" PRIMARY KEY ("id")
);

-- CreateTable
CREATE TABLE "AgentConfigToRepo" (
"agentConfigId" TEXT NOT NULL,
"repoId" INTEGER NOT NULL,

CONSTRAINT "AgentConfigToRepo_pkey" PRIMARY KEY ("agentConfigId","repoId")
);

-- CreateTable
CREATE TABLE "AgentConfigToConnection" (
"agentConfigId" TEXT NOT NULL,
"connectionId" INTEGER NOT NULL,

CONSTRAINT "AgentConfigToConnection_pkey" PRIMARY KEY ("agentConfigId","connectionId")
);

-- CreateIndex
CREATE INDEX "AgentConfig_orgId_type_enabled_idx" ON "AgentConfig"("orgId", "type", "enabled");

-- CreateIndex
CREATE UNIQUE INDEX "AgentConfig_orgId_name_key" ON "AgentConfig"("orgId", "name");

-- AddForeignKey
ALTER TABLE "AgentConfig" ADD CONSTRAINT "AgentConfig_orgId_fkey" FOREIGN KEY ("orgId") REFERENCES "Org"("id") ON DELETE CASCADE ON UPDATE CASCADE;

-- AddForeignKey
ALTER TABLE "AgentConfigToRepo" ADD CONSTRAINT "AgentConfigToRepo_agentConfigId_fkey" FOREIGN KEY ("agentConfigId") REFERENCES "AgentConfig"("id") ON DELETE CASCADE ON UPDATE CASCADE;

-- AddForeignKey
ALTER TABLE "AgentConfigToRepo" ADD CONSTRAINT "AgentConfigToRepo_repoId_fkey" FOREIGN KEY ("repoId") REFERENCES "Repo"("id") ON DELETE CASCADE ON UPDATE CASCADE;

-- AddForeignKey
ALTER TABLE "AgentConfigToConnection" ADD CONSTRAINT "AgentConfigToConnection_agentConfigId_fkey" FOREIGN KEY ("agentConfigId") REFERENCES "AgentConfig"("id") ON DELETE CASCADE ON UPDATE CASCADE;

-- AddForeignKey
ALTER TABLE "AgentConfigToConnection" ADD CONSTRAINT "AgentConfigToConnection_connectionId_fkey" FOREIGN KEY ("connectionId") REFERENCES "Connection"("id") ON DELETE CASCADE ON UPDATE CASCADE;
89 changes: 89 additions & 0 deletions packages/db/prisma/schema.prisma
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,28 @@ enum CodeHostType {
azuredevops
}

enum AgentType {
CODE_REVIEW
}

/// Determines which repositories an AgentConfig applies to.
enum AgentScope {
/// Applies to all repositories in the organization.
ORG
/// Applies to all repositories under specific connections.
CONNECTION
/// Applies to specific repositories only.
REPO
}

/// Controls how a custom prompt is combined with the built-in system rules.
enum PromptMode {
/// Custom prompt replaces the built-in rules entirely.
REPLACE
/// Custom prompt is appended after the built-in rules.
APPEND
}

model Repo {
id Int @id @default(autoincrement())
name String /// Full repo name, including the vcs hostname (ex. github.com/sourcebot-dev/sourcebot)
Expand Down Expand Up @@ -75,6 +97,8 @@ model Repo {

searchContexts SearchContext[]

agentConfigMappings AgentConfigToRepo[]

@@unique([external_id, external_codeHostUrl, orgId])
@@index([orgId])
@@index([indexedAt])
Expand Down Expand Up @@ -170,6 +194,8 @@ model Connection {
/// When the connection was last synced successfully.
syncedAt DateTime?

agentConfigMappings AgentConfigToConnection[]

/// Controls whether repository permissions are enforced for this connection.
/// When `PERMISSION_SYNC_ENABLED` is false, this setting has no effect.
/// Defaults to the value of `PERMISSION_SYNC_ENABLED`.
Expand Down Expand Up @@ -291,6 +317,8 @@ model Org {
searchContexts SearchContext[]

chats Chat[]

agentConfigs AgentConfig[]
}

enum OrgRole {
Expand Down Expand Up @@ -569,3 +597,64 @@ model OAuthToken {
createdAt DateTime @default(now())
lastUsedAt DateTime?
}

/// Configures a customisable AI agent (e.g. code review) scoped to an org, connection, or specific repos.
model AgentConfig {
id String @id @default(cuid())

org Org @relation(fields: [orgId], references: [id], onDelete: Cascade)
orgId Int

name String
description String?

type AgentType
enabled Boolean @default(true)

/// Custom prompt instructions. Null means the agent uses its built-in rules only.
prompt String?

/// Controls whether the custom prompt replaces or appends to the built-in rules.
promptMode PromptMode @default(APPEND)

/// Determines what this config is scoped to: the whole org, specific connections, or specific repos.
scope AgentScope

/// Repo-level scope mappings (populated when scope = REPO).
repos AgentConfigToRepo[]

/// Connection-level scope mappings (populated when scope = CONNECTION).
connections AgentConfigToConnection[]

/// Extensible per-agent settings stored as JSON.
/// Shape: { autoReviewEnabled?: boolean, reviewCommand?: string, model?: string, contextFiles?: string }
settings Json @default("{}")

createdAt DateTime @default(now())
updatedAt DateTime @updatedAt

@@unique([orgId, name])
@@index([orgId, type, enabled])
}

/// Maps an AgentConfig to specific repositories (used when scope = REPO).
model AgentConfigToRepo {
agentConfig AgentConfig @relation(fields: [agentConfigId], references: [id], onDelete: Cascade)
agentConfigId String

repo Repo @relation(fields: [repoId], references: [id], onDelete: Cascade)
repoId Int

@@id([agentConfigId, repoId])
}

/// Maps an AgentConfig to specific connections (used when scope = CONNECTION).
model AgentConfigToConnection {
agentConfig AgentConfig @relation(fields: [agentConfigId], references: [id], onDelete: Cascade)
agentConfigId String

connection Connection @relation(fields: [connectionId], references: [id], onDelete: Cascade)
connectionId Int

@@id([agentConfigId, connectionId])
}
Comment on lines +640 to +660
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Check whether the migration adds reverse indexes or tenant-safe constraints for AgentConfig mappings.
# Expected: indexes on repoId/connectionId and/or constraints preventing cross-org mappings.

rg -n -C3 \
  'AgentConfigToRepo|AgentConfigToConnection|repoId|connectionId|CREATE INDEX|CREATE UNIQUE INDEX|FOREIGN KEY|REFERENCES' \
  --glob '!**/node_modules/**' \
  --glob '!**/dist/**' \
  --glob '!**/build/**'

Repository: sourcebot-dev/sourcebot

Length of output: 50380

🏁 Script executed:

#!/bin/bash
# Find migration files related to AgentConfig mappings
find packages/db/prisma/migrations -type f -name "*.sql" 2>/dev/null | head -20

Repository: sourcebot-dev/sourcebot

Length of output: 1716

🏁 Script executed:

#!/bin/bash
# Check the current schema.prisma for AgentConfigToRepo and AgentConfigToConnection definitions
sed -n '640,680p' packages/db/prisma/schema.prisma

Repository: sourcebot-dev/sourcebot

Length of output: 836

🏁 Script executed:

#!/bin/bash
# Search for any indexes or constraints on these models in migrations
find packages/db/prisma/migrations -type f -name "*.sql" -exec grep -l "AgentConfigToRepo\|AgentConfigToConnection\|agentConfigId.*repoId\|agentConfigId.*connectionId" {} \; 2>/dev/null

Repository: sourcebot-dev/sourcebot

Length of output: 142

🏁 Script executed:

#!/bin/bash
# Examine the migration that added AgentConfig models
cat packages/db/prisma/migrations/20260421203635_add_agent_config/migration.sql

Repository: sourcebot-dev/sourcebot

Length of output: 2432

Add tenant constraints and reverse lookup indexes to AgentConfig mapping tables.

The migration does not include the recommended protections. The junction tables (AgentConfigToRepo and AgentConfigToConnection) lack orgId columns, so the database cannot enforce that a mapped repo or connection belongs to the same organization as the AgentConfig. While the API validates this at the application layer, the invariant should be protected at the schema layer to prevent data corruption from regressions or bypass.

Additionally, the composite primary keys are agentConfigId-first, but typical query patterns filter by repoId or connectionId first. Without reverse indexes, these lookups will perform full table scans or inefficient index usage.

Add orgId column with tenant-safe constraints and create reverse indexes such as @@index([repoId, agentConfigId]) and @@index([connectionId, agentConfigId]) to both tables.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@packages/db/prisma/schema.prisma` around lines 640 - 660, Add a tenant-scoped
orgId column to both AgentConfigToRepo and AgentConfigToConnection and make the
PK include orgId (e.g. @@id([orgId, agentConfigId, repoId]) and @@id([orgId,
agentConfigId, connectionId])) so the DB enforces tenant isolation; also add
reverse lookup indexes @@index([repoId, agentConfigId]) and
@@index([connectionId, agentConfigId]) respectively to speed queries. Ensure the
new orgId field type matches your org identifier (e.g. String) and
populate/migrate it from AgentConfig/Repo/Connection so the FK/PK invariants
hold.

63 changes: 63 additions & 0 deletions packages/web/src/app/(app)/agents/configs/[agentId]/page.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
import { authenticatedPage } from "@/middleware/authenticatedPage";
import { NavigationMenu } from "@/app/(app)/components/navigationMenu";
import { AgentConfigForm } from "../agentConfigForm";
import { notFound } from "next/navigation";
import { OrgRole } from "@sourcebot/db";

type Props = {
params: Promise<{ agentId: string }>;
};

export default authenticatedPage(async ({ prisma, org }, { params }: Props) => {
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Gate config editing to org owners.

This page exposes the edit form and config details with plain authentication. Add the owner-role option to authenticatedPage so non-owners cannot access agent configuration management.

Proposed fix 
+import { OrgRole } from "@sourcebot/db";
...
-export default authenticatedPage(async ({ prisma, org }, { params }: Props) => {
+export default authenticatedPage(async ({ prisma, org }, { params }: Props) => {
     const { agentId } = await params;
...
-});
+}, { minRole: OrgRole.OWNER, redirectTo: "/settings" });

Based on learnings: Use authenticatedPage with { minRole: OrgRole.OWNER, redirectTo: '/settings' } option to gate pages by role in the (app) route group.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
export default authenticatedPage(async ({ prisma, org }, { params }: Props) => {
import { OrgRole } from "@sourcebot/db";
// ... other imports ...
export default authenticatedPage(async ({ prisma, org }, { params }: Props) => {
const { agentId } = await params;
// ... rest of the function body ...
}, { minRole: OrgRole.OWNER, redirectTo: "/settings" });
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/app/`(app)/agents/configs/[agentId]/page.tsx at line 10, The
page currently allows any authenticated user to edit agent configs; update the
authenticatedPage call to require org owners by passing the options object {
minRole: OrgRole.OWNER, redirectTo: '/settings' } to authenticatedPage (i.e.,
change authenticatedPage(async (...) => ...) to authenticatedPage({ minRole:
OrgRole.OWNER, redirectTo: '/settings' }, async (...) => ...)); also ensure
OrgRole is imported where used (add import for OrgRole if missing) so non-owners
are redirected to /settings.

const { agentId } = await params;

const [config, connections, repos] = await Promise.all([
prisma.agentConfig.findFirst({
where: { id: agentId, orgId: org.id },
include: {
repos: { select: { repoId: true } },
connections: { select: { connectionId: true } },
},
}),
prisma.connection.findMany({
where: { orgId: org.id },
select: { id: true, name: true, connectionType: true },
orderBy: { name: "asc" },
}),
prisma.repo.findMany({
where: { orgId: org.id },
select: { id: true, displayName: true, external_id: true, external_codeHostType: true },
orderBy: { displayName: "asc" },
}),
]);

if (!config) {
notFound();
}

return (
<div className="flex flex-col items-center overflow-hidden min-h-screen">
<NavigationMenu />
<div className="w-full max-w-3xl px-4 mt-12 mb-24">
<h1 className="text-2xl font-semibold text-foreground mb-8">Edit agent config</h1>
<AgentConfigForm
initialValues={{
id: config.id,
name: config.name,
description: config.description ?? "",
type: config.type,
enabled: config.enabled,
prompt: config.prompt ?? "",
promptMode: config.promptMode,
scope: config.scope,
repoIds: config.repos.map((r) => r.repoId),
connectionIds: config.connections.map((c) => c.connectionId),
settings: config.settings as Record<string, unknown>,
}}
connections={connections}
repos={repos}
/>
</div>
</div>
);
}, { minRole: OrgRole.OWNER, redirectTo: '/agents' });
Loading
Loading