Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docs-web/architecture/external-chat-providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,5 +72,12 @@ Dashboard settings use `src/server/chat-provider-routes.ts` to manage chat provi
| `DELETE /api/chat-providers/channel-bindings/:bindingId` | Deletes a channel binding. |
| `GET /api/chat-providers/connections/:connectionId/delivery-status` | Lists recent outbound delivery records for one provider connection. |
| `GET /api/chat-providers/channel-bindings/:bindingId/delivery-status` | Lists recent outbound delivery records for one channel binding. |
| `POST /api/chat-providers/ingress/:providerConnectionId` | Accepts authenticated inbound bridge messages, normalizes provider payloads, deduplicates external message IDs, and posts routed text to dashboard chat threads. |

These endpoints only manage configuration and status records. Inbound message processing, provider polling, and outbound provider delivery are implemented by later runtime adapters.
The ingress endpoint supports OpenClaw, webhook, and native bridge payloads for WhatsApp, iMessage, Telegram, Slack, Microsoft Teams, and Discord. OpenClaw and native bridges authenticate with bearer tokens from the configured bridge secret. Webhook bridges require a configured signing secret and a valid HMAC signature; they do not accept bearer-only fallback. All ingress requests require a fresh timestamp, and signed requests or requests with explicit nonces are replay-checked before processing.

Inbound messages normalize to provider connection id, provider kind, external channel id/name, external sender id/name, text, external message id, timestamp, and redacted raw metadata. The repository idempotency lookup runs before chat posting; duplicate external messages return the existing delivery record without creating another conversation message.

Channel resolution only considers enabled bindings with inbound enabled for the provider connection and external channel. If multiple projects share a channel, routing hints such as `projectSelectorPrefix`, `projectSelector`, `projectAlias`, `aliases`, or payload-level project selectors are applied first. If no hint selects exactly one binding, the runtime records a `disambiguation_needed` inbound delivery state and returns a conflict response instead of guessing a project.

Routed inbound text is posted through `ChatThreadRuntimeService.postMessage` with metadata marking `source: "chat_provider"`, provider kind, external channel id, external sender, inbound delivery id, and `suppressRichWidgets: true`. Outbound provider replies and widget stripping remain separate follow-up work.
Original file line number Diff line number Diff line change
Expand Up @@ -72,5 +72,12 @@ Dashboard settings use `src/server/chat-provider-routes.ts` to manage chat provi
| `DELETE /api/chat-providers/channel-bindings/:bindingId` | Deletes a channel binding. |
| `GET /api/chat-providers/connections/:connectionId/delivery-status` | Lists recent outbound delivery records for one provider connection. |
| `GET /api/chat-providers/channel-bindings/:bindingId/delivery-status` | Lists recent outbound delivery records for one channel binding. |
| `POST /api/chat-providers/ingress/:providerConnectionId` | Accepts authenticated inbound bridge messages, normalizes provider payloads, deduplicates external message IDs, and posts routed text to dashboard chat threads. |

These endpoints only manage configuration and status records. Inbound message processing, provider polling, and outbound provider delivery are implemented by later runtime adapters.
The ingress endpoint supports OpenClaw, webhook, and native bridge payloads for WhatsApp, iMessage, Telegram, Slack, Microsoft Teams, and Discord. OpenClaw and native bridges authenticate with bearer tokens from the configured bridge secret. Webhook bridges require a configured signing secret and a valid HMAC signature; they do not accept bearer-only fallback. All ingress requests require a fresh timestamp, and signed requests or requests with explicit nonces are replay-checked before processing.

Inbound messages normalize to provider connection id, provider kind, external channel id/name, external sender id/name, text, external message id, timestamp, and redacted raw metadata. The repository idempotency lookup runs before chat posting; duplicate external messages return the existing delivery record without creating another conversation message.

Channel resolution only considers enabled bindings with inbound enabled for the provider connection and external channel. If multiple projects share a channel, routing hints such as `projectSelectorPrefix`, `projectSelector`, `projectAlias`, `aliases`, or payload-level project selectors are applied first. If no hint selects exactly one binding, the runtime records a `disambiguation_needed` inbound delivery state and returns a conflict response instead of guessing a project.

Routed inbound text is posted through `ChatThreadRuntimeService.postMessage` with metadata marking `source: "chat_provider"`, provider kind, external channel id, external sender, inbound delivery id, and `suppressRichWidgets: true`. Outbound provider replies and widget stripping remain separate follow-up work.
9 changes: 8 additions & 1 deletion docs/architecture/external-chat-providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,5 +72,12 @@ Dashboard settings use `src/server/chat-provider-routes.ts` to manage chat provi
| `DELETE /api/chat-providers/channel-bindings/:bindingId` | Deletes a channel binding. |
| `GET /api/chat-providers/connections/:connectionId/delivery-status` | Lists recent outbound delivery records for one provider connection. |
| `GET /api/chat-providers/channel-bindings/:bindingId/delivery-status` | Lists recent outbound delivery records for one channel binding. |
| `POST /api/chat-providers/ingress/:providerConnectionId` | Accepts authenticated inbound bridge messages, normalizes provider payloads, deduplicates external message IDs, and posts routed text to dashboard chat threads. |

These endpoints only manage configuration and status records. Inbound message processing, provider polling, and outbound provider delivery are implemented by later runtime adapters.
The ingress endpoint supports OpenClaw, webhook, and native bridge payloads for WhatsApp, iMessage, Telegram, Slack, Microsoft Teams, and Discord. OpenClaw and native bridges authenticate with bearer tokens from the configured bridge secret. Webhook bridges require a configured signing secret and a valid HMAC signature; they do not accept bearer-only fallback. All ingress requests require a fresh timestamp, and signed requests or requests with explicit nonces are replay-checked before processing.

Inbound messages normalize to provider connection id, provider kind, external channel id/name, external sender id/name, text, external message id, timestamp, and redacted raw metadata. The repository idempotency lookup runs before chat posting; duplicate external messages return the existing delivery record without creating another conversation message.

Channel resolution only considers enabled bindings with inbound enabled for the provider connection and external channel. If multiple projects share a channel, routing hints such as `projectSelectorPrefix`, `projectSelector`, `projectAlias`, `aliases`, or payload-level project selectors are applied first. If no hint selects exactly one binding, the runtime records a `disambiguation_needed` inbound delivery state and returns a conflict response instead of guessing a project.

Routed inbound text is posted through `ChatThreadRuntimeService.postMessage` with metadata marking `source: "chat_provider"`, provider kind, external channel id, external sender, inbound delivery id, and `suppressRichWidgets: true`. Outbound provider replies and widget stripping remain separate follow-up work.
9 changes: 9 additions & 0 deletions src/app/dependency-factory/dashboard-factory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,12 @@ import { ProviderExecutionService } from "../../services/provider-execution-serv
import { SchedulerService } from "../../services/scheduler-service.js";
import { ExecutionInvocationControlService } from "../../services/execution-invocation-control-service.js";
import { createLateBoundDependency } from "../../shared/late-bound-dependency.js";
import { ChatProviderIngressService } from "../../services/chat-provider-ingress-service.js";

export interface DashboardDependencies {
chatThreadRuntimeService: ChatThreadRuntimeService;
chatProviderRepository: CoreDependencies["chatProviderRepository"];
chatProviderIngressService: ChatProviderIngressService;
activityCacheService: ActivityCacheService;
taskRerunService: TaskRerunService;
executionControlService: ExecutionControlService;
Expand Down Expand Up @@ -145,6 +147,12 @@ export function createDashboardDependencies(
logger: logger.child({ component: "chat-thread-runtime-service" }),
});

const chatProviderIngressService = new ChatProviderIngressService({
chatProviderRepository,
chatThreadRuntimeService,
logger: logger.child({ component: "chat-provider-ingress-service" }),
});

const activityCacheService = new ActivityCacheService(
{
getSubtasks: () => projectRuntimeRepository.getSelectedProjectLiveStatus().subtasks,
Expand Down Expand Up @@ -453,6 +461,7 @@ export function createDashboardDependencies(
return {
chatProviderRepository,
chatThreadRuntimeService,
chatProviderIngressService,
activityCacheService,
taskRerunService,
executionControlService,
Expand Down
3 changes: 3 additions & 0 deletions src/app/lifecycle/dashboard-lifecycle-service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ import type { ChatThreadRuntimeService } from "../../services/chat-thread-runtim
import type { QuicksprintService } from "../../services/quicksprint-service.js";
import type { ProjectSetupService } from "../../services/project-setup-service.js";
import type { SchedulerService } from "../../services/scheduler-service.js";
import type { ChatProviderIngressService } from "../../services/chat-provider-ingress-service.js";
import type { MemoryService } from "../../services/memory-service.js";
import type { KnowledgeService } from "../../services/knowledge-service.js";
import type { MemoryPromotionService } from "../../services/memory-promotion-service.js";
Expand Down Expand Up @@ -109,6 +110,7 @@ export interface BootDashboardDeps {
schedulerService: SchedulerService;
sprintIssueService: SprintIssueService;
chatThreadRuntimeService: ChatThreadRuntimeService;
chatProviderIngressService: ChatProviderIngressService;
dashboardRealtimeService: DashboardRealtimeService;
logger: Logger;
getLiveActivitiesForActiveTasks: () => Promise<Record<string, JulesActivity[]>>;
Expand Down Expand Up @@ -415,6 +417,7 @@ export async function bootDashboard(deps: BootDashboardDeps): Promise<DashboardS
knowledgeService: deps.knowledgeService,
agentPresetRepository: deps.agentPresetRepository,
chatProviderRepository: deps.chatProviderRepository,
chatProviderIngressService: deps.chatProviderIngressService,
projectManagementRepository: deps.projectManagementRepository,
executionRepository: deps.executionRepository,
getLiveSnapshot: (projectIdHint) => getProjectLiveSnapshot({
Expand Down
3 changes: 3 additions & 0 deletions src/contracts/chat-provider-types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,9 @@ export interface UpdateChatProviderDeliveryStateInput {
attemptCount?: number;
lastError?: string | null;
externalMessageId?: string | null;
conversationThreadId?: string | null;
conversationMessageId?: string | null;
payload?: Record<string, unknown> | null;
}

export interface ChatProviderMessageDeliveryRecord {
Expand Down
6 changes: 6 additions & 0 deletions src/repositories/chat-provider-repository.ts
Original file line number Diff line number Diff line change
Expand Up @@ -522,13 +522,19 @@ export class ChatProviderRepository {
attempt_count = ?,
last_error = ?,
external_message_id = ?,
conversation_thread_id = ?,
conversation_message_id = ?,
payload_json = ?,
updated_at = ?
WHERE id = ?
`).run(
this.requireDeliveryStatus(input.status),
input.attemptCount !== undefined ? this.requireNonNegativeInteger(input.attemptCount, "attemptCount") : existing.attemptCount,
input.lastError !== undefined ? input.lastError : existing.lastError,
input.externalMessageId !== undefined ? input.externalMessageId : existing.externalMessageId,
input.conversationThreadId !== undefined ? input.conversationThreadId : existing.conversationThreadId,
input.conversationMessageId !== undefined ? input.conversationMessageId : existing.conversationMessageId,
input.payload !== undefined ? this.stringifyNullableJson(input.payload) : this.stringifyNullableJson(existing.payload),
now,
deliveryId,
);
Expand Down
83 changes: 83 additions & 0 deletions src/server/chat-provider-ingress-routes.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
import type { Express, Request } from "express";
import type { DashboardDependencies } from "./dashboard-server.js";
import { asyncRoute } from "./route-utils.js";
import { HttpRouteError } from "./http-errors.js";
import { requireTrimmedString } from "./request-parsers.js";
import { ChatProviderIngressSecurity, ChatProviderIngressSecurityError } from "../services/chat-provider-security.js";

const defaultSecurityVerifier = new ChatProviderIngressSecurity();

export function registerChatProviderIngressRoutes(router: Express, deps: DashboardDependencies): void {
if (!deps.chatProviderRepository || !deps.chatProviderIngressService) {
return;
}

const handler = asyncRoute(async (req, res) => {
const providerConnectionId = requireTrimmedString(
req.params.providerConnectionId ?? req.params.connectionId,
"providerConnectionId",
);
const connection = deps.chatProviderRepository!.getConnectionInternal(providerConnectionId);
if (!connection) {
throw new HttpRouteError(404, "Chat provider connection not found.");
}

try {
defaultSecurityVerifier.verify(connection, {
headers: req.headers,
rawBody: buildRequestBodyForSignature(req),
});
} catch (error) {
if (error instanceof ChatProviderIngressSecurityError) {
deps.logger?.warn("Rejected chat provider ingress authentication", {
logPurpose: "security",
providerConnectionId,
providerKind: connection.providerKind,
reason: error.code,
statusCode: error.status,
});
throw new HttpRouteError(error.status, error.message);
}
throw error;
}

const result = await deps.chatProviderIngressService!.processInbound({
providerConnectionId,
payload: req.body,
});

const statusCode = statusCodeForIngressResult(result.status);
res.status(statusCode).json(result);
});

router.post("/api/chat-providers/ingress/:providerConnectionId", handler);
router.post("/api/chat-providers/connections/:connectionId/ingress", handler);
}

function buildRequestBodyForSignature(req: Request): string {
const rawBody = (req as Request & { rawBody?: unknown }).rawBody;
if (typeof rawBody === "string") {
return rawBody;
}
if (Buffer.isBuffer(rawBody)) {
return rawBody.toString("utf8");
}
return JSON.stringify(req.body ?? {});
}

function statusCodeForIngressResult(status: string): number {
switch (status) {
case "accepted":
return 202;
case "duplicate":
return 200;
case "ambiguous":
return 409;
case "unbound":
return 404;
case "rejected":
return 404;
default:
return 500;
}
}
4 changes: 2 additions & 2 deletions src/server/chat-provider-routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ function decorateSetupDefinition(req: Request, schema: ChatProviderSetupSchema):
} {
return {
...schema,
ingressUrlTemplate: `${getRequestOrigin(req)}/api/chat-providers/connections/{connectionId}/ingress`,
ingressUrlTemplate: `${getRequestOrigin(req)}/api/chat-providers/ingress/{connectionId}`,
bridgeModes: schema.bridgeModes.map((bridgeMode) => ({
...bridgeMode,
setupHints: buildSetupHints(schema.kind, bridgeMode.mode),
Expand All @@ -197,7 +197,7 @@ function decorateConnection(
}

function buildIngressUrl(req: Request, connectionId: string): string {
return `${getRequestOrigin(req)}/api/chat-providers/connections/${encodeURIComponent(connectionId)}/ingress`;
return `${getRequestOrigin(req)}/api/chat-providers/ingress/${encodeURIComponent(connectionId)}`;
}

function buildSetupHints(providerKind: ChatProviderKind, bridgeMode: ChatProviderBridgeMode): ChatProviderSetupHints {
Expand Down
4 changes: 4 additions & 0 deletions src/server/code-ux-server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ import { workspaceVolumeHelperPool } from "../infrastructure/providers/cli/works
import { disposeCommandSpawner, shutdownGitHelperPool } from "../shared/subprocess/command-runner.js";
import { LocalMcpCliConfigService } from "../services/local-mcp-cli-config-service.js";
import type { McpConnectionInfo } from "../contracts/mcp-connection-types.js";
import type { ChatProviderIngressService } from "../services/chat-provider-ingress-service.js";

function detectMergeConflictMessage(message: string | null | undefined): boolean {
const normalized = String(message || "").trim().toLowerCase();
Expand Down Expand Up @@ -187,6 +188,7 @@ export class CodeUxServer {
private projectSetupService: import("../services/project-setup-service.js").ProjectSetupService;
private schedulerService: import("../services/scheduler-service.js").SchedulerService;
private chatThreadRuntimeService: import("../services/chat-thread-runtime-service.js").ChatThreadRuntimeService;
private chatProviderIngressService: ChatProviderIngressService;
private runtimeCleanupService: RuntimeCleanupService;
private runtimeStartupRecoveryService: RuntimeStartupRecoveryService;
private dashboardRealtimeService: DashboardRealtimeService;
Expand Down Expand Up @@ -266,6 +268,7 @@ export class CodeUxServer {
this.projectSetupService = deps.projectSetupService;
this.schedulerService = deps.schedulerService;
this.chatThreadRuntimeService = deps.chatThreadRuntimeService;
this.chatProviderIngressService = deps.chatProviderIngressService;
this.runtimeCleanupService = deps.runtimeCleanupService;
this.runtimeStartupRecoveryService = new RuntimeStartupRecoveryService({
sessionTracking: this.sessionTracking,
Expand Down Expand Up @@ -1340,6 +1343,7 @@ export class CodeUxServer {
projectSetupService: this.projectSetupService,
schedulerService: this.schedulerService,
chatThreadRuntimeService: this.chatThreadRuntimeService,
chatProviderIngressService: this.chatProviderIngressService,
dashboardRealtimeService: this.dashboardRealtimeService,
logger: this.logger,
getLiveActivitiesForActiveTasks: () => this.getLiveActivitiesForActiveTasks(),
Expand Down
6 changes: 6 additions & 0 deletions src/server/dashboard-middleware.ts
Original file line number Diff line number Diff line change
Expand Up @@ -106,14 +106,20 @@ export const applyDashboardPreRouteMiddleware = (
app.use(express.json({
limit: DASHBOARD_LARGE_SETTINGS_JSON_BODY_LIMIT,
type: (req) => shouldParseDashboardJsonBody(req, "large"),
verify: captureRawJsonBody,
}));
app.use(express.json({
limit: DASHBOARD_DEFAULT_JSON_BODY_LIMIT,
type: (req) => shouldParseDashboardJsonBody(req, "default"),
verify: captureRawJsonBody,
}));
app.use(createDashboardJsonBodyErrorHandler(dashboardLogger));
};

function captureRawJsonBody(req: IncomingMessage, _res: unknown, buf: Buffer): void {
(req as IncomingMessage & { rawBody?: string }).rawBody = buf.toString("utf8");
}

function createDashboardJsonContentTypeGuard(dashboardLogger: Logger): RequestHandler {
return (req, res, next) => {
const pathname = getRequestPathname(req);
Expand Down
2 changes: 2 additions & 0 deletions src/server/dashboard-route-registration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import { registerMemoryRoutes } from "./memory-routes.js";
import { registerKnowledgeRoutes } from "./knowledge-routes.js";
import { registerDocsWebRoutes } from "./docs-web-routes.js";
import { registerChatProviderRoutes } from "./chat-provider-routes.js";
import { registerChatProviderIngressRoutes } from "./chat-provider-ingress-routes.js";

export interface DashboardRouteRegistrationOptions {
app: Express;
Expand Down Expand Up @@ -101,6 +102,7 @@ const registerPreviewRouteGroup = (app: Express, deps: DashboardDependencies): v
const registerSettingsRouteGroup = (app: Express, deps: DashboardDependencies, liveActivityCacheMs: number): void => {
registerSettingsRoutes(app, deps, liveActivityCacheMs);
registerChatProviderRoutes(app, deps);
registerChatProviderIngressRoutes(app, deps);
};

const registerProjectConfigurationRouteGroup = (app: Express, deps: DashboardDependencies): void => {
Expand Down
Loading