[pull] main from activepieces:main

.agents/features/ee-projects.md

@@ -29,6 +29,7 @@ The EE Projects module adds team collaboration, role-based access control (RBAC)
 - **Community (CE)**: Single-user projects only. No project members, no roles, no releases, no git sync.
 - **Enterprise (EE, self-hosted)**: Full feature set behind `projectRolesEnabled` and `environmentsEnabled` plan flags.
 - **Cloud**: Members and invitations available on paid plans. Custom roles behind `customRolesEnabled`. Git sync and releases behind `environmentsEnabled`.
+- **`workerGroupId` assignment / per-project worker routing**: Enterprise feature gated behind `platform_plan.isolatedWorkersEnabled`. Both setting a project's `workerGroupId` (via the project update endpoint) and listing project worker groups (`GET /v1/projects/worker-groups`) require the flag.
 
 ## Domain Terms
 
@@ -42,6 +43,7 @@ The EE Projects module adds team collaboration, role-based access control (RBAC)
 - **Git Sync**: Configuration of an SSH-backed git repo + branch used as a release source or push target.
 - **RBAC**: Role-Based Access Control — enforced per-request via `rbacService.assertPrincipalAccessToProject()`.
 - **Release Type**: GIT_BRANCH (from git), MANUAL (from another project), ROLLBACK (revert to a previous release).
+- **workerGroupId**: Optional pool label on a project (bare, e.g. `1cpu_machine`). When set (and `isolatedWorkersEnabled` is on for the platform), the project's `EXECUTE_FLOW`/`EXECUTE_WEBHOOK` jobs are routed to `project-<label>-jobs`; other job types are unaffected. The matching worker advertises `AP_WORKER_GROUP_ID=<label>` with `AP_PROJECT_WORKER=true` (scope comes from the flag, not a prefix). Set via `POST /v1/projects/:id`. See the Workers feature doc for the unified worker-group mechanics.
 
 ## Project Members
 
@@ -106,3 +108,7 @@ The EE Projects module adds team collaboration, role-based access control (RBAC)
 - Platform admins: see all projects
 - Operators: see all projects except others' personal
 - Regular users: see own personal + team projects where member
+
+**Endpoints** (`platform-project-controller.ts`):
+- `POST /v1/projects/:id` — update; body `UpdateProjectPlatformRequest` accepts `workerGroupId` (validated against `^[a-z0-9_-]+$`, applied in `platformProjectService.update()` only when `platform_plan.isolatedWorkersEnabled` is on).
+- `GET /v1/projects/worker-groups` — platform-admin only; returns `{ groups: [{ label, slots }], sharedSlots }` from online project-scope workers (those started with `AP_PROJECT_WORKER=true`) via `machineService.listProjectWorkerGroups()` for the assignment UI; returns 402 `FEATURE_DISABLED` when `isolatedWorkersEnabled` is off.

.agents/features/workers.md

@@ -11,13 +11,14 @@ Workers are separate Node processes that poll the app for jobs and execute flows
 - `packages/server/worker/src/lib/worker.ts` — worker lifecycle (`worker.start/stop`), `pollAndExecute` loop (with version gate, tracks `inFlightJobs`), `getWorkerProps`; builds the `Runtime` via `createSandboxRuntime` and a per-job `Resolver` via `createResolver`. `stop()` drains in-flight jobs (`drainInFlightJobs`) before teardown; an unexpected disconnect aborts the runtime (`abortInFlightRuntime`)
 - `packages/server/worker/src/lib/runtime/sandbox-config.ts` — bridges worker settings → `SandboxSettings` + cache base path (merges the env-only `AP_REUSE_SANDBOX` override into the fetched `WorkerSettings`)
 - `packages/server/sandbox/` — standalone `@activepieces/sandbox` library holding the in-process sandbox (`createSandboxRuntime`, `sandbox-manager`, isolate/fork process makers), cache/piece installation, the worker-side `createResolver`, and the `Runtime` / `Resolver` / `ProvisionInput` type contracts (`src/lib/types.ts`)
-- `packages/server/worker/src/lib/config/configs.ts` — worker env vars incl. `AP_REUSE_SANDBOX` (`WorkerSystemProp.REUSE_SANDBOX`, reuse the engine process between jobs) and `AP_WORKER_CONCURRENCY`
+- `packages/server/worker/src/lib/config/configs.ts` — worker env vars incl. `AP_REUSE_SANDBOX` (`WorkerSystemProp.REUSE_SANDBOX`, reuse the engine process between jobs), `AP_WORKER_CONCURRENCY`, `AP_WORKER_GROUP_ID`, and `AP_PROJECT_WORKER` (`WorkerSystemProp.PROJECT_WORKER`, boolean, default `false` — flags the group id as project-scoped)
 - `packages/server/worker/src/lib/config/worker-settings.ts` — caches the `WorkerSettingsResponse` fetched on connect
 - `packages/server/utils/src/ap-version.ts` — `apVersionUtil.getCurrentRelease()`; both sides read the deploy-root `package.json` version
 - `packages/core/shared/src/lib/automation/workers/index.ts` — `WorkerProps`, `MachineInformation`, `WorkerSettingsResponse`, `WorkerToApiContract` contracts
 
 ## Edition Availability
 - Community / Enterprise / Cloud: all editions run workers; topology differs (embedded `WORKER_AND_APP` for self-host single-container vs dedicated worker fleets on Cloud).
+- **Per-project worker routing** is an enterprise feature gated behind `platform_plan.isolatedWorkersEnabled`. The routing itself (queue resolution) is edition-agnostic, but assigning a project's `workerGroupId` and listing available project groups (`GET /v1/projects/worker-groups`) require the flag; unassigned projects always use the shared/platform queue.
 
 ## Domain Terms
 
@@ -28,6 +29,10 @@ Workers are separate Node processes that poll the app for jobs and execute flows
 - **`WorkerSettingsResponse`** — runtime config the app hands a worker on connect; now includes `APP_VERSION` (the app's release).
 - **`connectionGeneration`** — worker-side counter bumped on every disconnect; in-flight poll loops exit when their captured generation goes stale, so a reconnect starts fresh loops.
 - **version gate** — both sides refuse to exchange jobs when worker release ≠ app release (see below).
+- **worker group (`AP_WORKER_GROUP_ID` + `AP_PROJECT_WORKER`)** — one unified routing primitive for both platform- and project-scoped dedicated pools. A worker advertises a **bare** id in the Socket.IO handshake auth plus a `projectWorker` boolean (from `AP_PROJECT_WORKER`, default `false`); the **flag — not a prefix — encodes scope**. There is no startup format check (the old required `pl_`/`pr_` prefix and its `assertValidWorkerGroupPrefix` guard were removed):
+  - `AP_PROJECT_WORKER=false` (default) → **platform** scope → polls `platform-<id>-jobs` (`getPlatformGroupQueueName`); `<id>` matches `platform_plan.workerGroupId`.
+  - `AP_PROJECT_WORKER=true` → **project** scope → polls `project-<label>-jobs` (`getProjectGroupQueueName`); `<label>` is an arbitrary pool label matched against `project.workerGroupId`.
+  The app parses both handshake values once with `parseWorkerGroupValue({ value, projectWorker })` (in `workers/job/index.ts`) into `{ scope, id }` (`WorkerGroupAssignment`) — scope from the flag, id used verbatim — and stores `{ workerGroupId, workerGroupScope }` on the cache entry. An empty/absent id yields no assignment (shared queue), regardless of the flag. **Project-scope groups route only `EXECUTE_FLOW`/`EXECUTE_WEBHOOK`** (`PROJECT_GROUP_ROUTABLE_JOB_TYPES`); all other job types use the platform group / shared queue. **Platform-scope groups route all job types** for the platform. Producer routing reads bare ids from the DB (scope implied by table), and the worker env is now bare too — scope travels beside the id, not inside it. A worker with any group must run a process sandbox mode and set `AP_REUSE_SANDBOX`.
 
 ## Connection & Poll Flow
 1. Worker connects → emits `FETCH_WORKER_SETTINGS`; app's `machineService.onConnection` returns `WorkerSettingsResponse` (incl. `APP_VERSION`) and registers `createHandlers` for the socket.

docs/install/configure-operate/worker-groups.mdx

@@ -19,10 +19,18 @@ Use both together to guarantee capacity for busy projects and cap the rest.
 
 ## How it works
 
-A worker joins a group by setting its group ID at startup:
+A worker joins a group by setting its group ID at startup. `AP_PROJECT_WORKER` selects the group's scope:
+
+- **Project** group — set `AP_PROJECT_WORKER=true`. Assign individual projects to it from the platform's Workers → Assigqnments page.
+- **Platform** group — leave `AP_PROJECT_WORKER=false` (the default). Serves the whole platform's runs; used for dedicated/canary fleets.
 
 ```bash
-AP_WORKER_GROUP_ID=<group-name>
+# Project group, e.g. 1cpu_machine
+AP_WORKER_GROUP_ID=1cpu_machine
+AP_PROJECT_WORKER=true
+
+# Platform group (AP_PROJECT_WORKER defaults to false)
+AP_WORKER_GROUP_ID=canary
 ```
 
 Grouped workers run in a process-isolated execution mode, so set:
@@ -32,4 +40,24 @@ AP_EXECUTION_MODE=SANDBOX_PROCESS   # or SANDBOX_CODE_AND_PROCESS
 AP_REUSE_SANDBOX=true               # or false, must be set explicitly
 ```
 
-Once a project is assigned to a group, its runs are routed to that group's dedicated queue and picked up only by workers carrying the matching `AP_WORKER_GROUP_ID`. Projects without a group keep drawing from the shared queue.
+Once a project is assigned to a group, its flow and webhook runs are routed to that group's dedicated queue (`project-<group-name>-jobs`) and picked up only by workers carrying the matching `AP_WORKER_GROUP_ID` with `AP_PROJECT_WORKER=true`. Projects without a group keep drawing from the shared queue.
+
+## Assigning projects to a group
+
+You can assign a project to a project group in two ways:
+
+- **UI** — go to **Platform → Infrastructure → Workers** (`/platform/infrastructure/workers`), open the **Assignments** tab, and pick the group for each project.
+- **API** — set `workerGroupId` on the project via the projects API. Use the bare group name (the same value as the group's `AP_WORKER_GROUP_ID`); pass `null` to unassign and return the project to the shared queue.
+
+The same request also sets `maxConcurrentJobs` — the project's **concurrency limit** (the soft-cap ceiling). It's the maximum number of flow runs the project executes at once:
+
+- **Omit it** (or send `null`) and the project uses the default: `min(plan limit, group slots)` — i.e. it can use the group's full reserved capacity, up to the plan's limit.
+- **Set a number** to cap the project below that. The value is clamped to the group's slot count (a project can't run more concurrently than its pool can serve).
+
+```bash
+curl -X POST 'https://your-instance.com/api/v1/projects/<PROJECT_ID>' \
+  -H 'Authorization: Bearer <PLATFORM_API_KEY>' \
+  -H 'Content-Type: application/json' \
+  -d '{ "workerGroupId": "1cpu_machine", "maxConcurrentJobs": 2 }'
+```
+

packages/core/execution/src/lib/workers/index.ts

@@ -13,6 +13,11 @@ export enum WorkerMachineType {
     DEDICATED = 'DEDICATED',
 }
 
+export enum WorkerGroupScope {
+    PLATFORM = 'platform',
+    PROJECT = 'project',
+}
+
 export enum NetworkMode {
     UNRESTRICTED = 'UNRESTRICTED',
     STRICT = 'STRICT',
@@ -67,6 +72,7 @@ export const WorkerMachineWithStatus = WorkerMachine.extend({
     status: z.nativeEnum(WorkerMachineStatus),
     type: z.nativeEnum(WorkerMachineType),
     workerGroupId: z.string().optional(),
+    workerGroupScope: z.nativeEnum(WorkerGroupScope).optional(),
 })
 
 export type WorkerMachineWithStatus = z.infer<typeof WorkerMachineWithStatus>

packages/core/shared/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@activepieces/shared",
-  "version": "0.107.1",
+  "version": "0.108.0",
   "type": "commonjs",
   "sideEffects": false,
   "main": "./dist/src/index.js",

packages/core/shared/src/lib/ee/billing/index.ts

@@ -83,6 +83,7 @@ export const STANDARD_CLOUD_PLAN: PlatformPlanWithOnlyLimits = {
     aiProvidersEnabled: false,
     chatEnabled: false,
     dataManipulationEnabled: false,
+    workerGroupsEnabled: false,
     globalConnectionsEnabled: false,
     customRolesEnabled: false,
     environmentsEnabled: false,
@@ -110,6 +111,7 @@ export const OPEN_SOURCE_PLAN: PlatformPlanWithOnlyLimits = {
     aiProvidersEnabled: true,
     chatEnabled: false,
     dataManipulationEnabled: false,
+    workerGroupsEnabled: false,
     globalConnectionsEnabled: false,
     customRolesEnabled: false,
     includedAiCredits: 0,

packages/core/shared/src/lib/management/platform/platform.model.ts

@@ -64,6 +64,7 @@ export const PlatformPlan = z.object({
     aiProvidersEnabled: z.boolean(),
     chatEnabled: z.boolean(),
     dataManipulationEnabled: z.boolean(),
+    workerGroupsEnabled: z.boolean(),
     managePiecesEnabled: z.boolean(),
     manageTemplatesEnabled: z.boolean(),
     customAppearanceEnabled: z.boolean(),

packages/core/shared/src/lib/management/project/project-requests.ts

@@ -14,6 +14,7 @@ export const UpdateProjectPlatformRequest = z.object({
     }).optional(),
     globalConnectionExternalIds: z.array(z.string()).optional(),
     maxConcurrentJobs: z.optional(Nullable(z.number().int().positive())),
+    workerGroupId: z.optional(Nullable(z.string())),
 })
 
 export type UpdateProjectPlatformRequest = z.infer<typeof UpdateProjectPlatformRequest>

packages/core/shared/src/lib/management/project/project.ts

@@ -59,6 +59,7 @@ export const Project = z.object({
     releasesEnabled: z.boolean(),
     metadata: Nullable(Metadata),
     poolId: Nullable(ApId),
+    workerGroupId: Nullable(z.string()),
 })
 
 const projectAnalytics = z.object({

packages/pieces/community/microsoft-onenote/src/i18n/translation.json

@@ -11,12 +11,14 @@
   "Create Page": "Create Page",
   "Create Image Note": "Create Image Note",
   "Append Note": "Append Note",
+  "Custom API Call": "Custom API Call",
   "Creates a notebook.": "Creates a notebook.",
   "Creates a new section in notebook.": "Creates a new section in notebook.",
   "Create a new note in a specific section with title and content.": "Create a new note in a specific section with title and content.",
   "Creates a page in section.": "Creates a page in section.",
   "Create a note containing an embedded image via a public image URL.": "Create a note containing an embedded image via a public image URL.",
   "Append content to the end of an existing note.": "Append content to the end of an existing note.",
+  "Make a custom API call to a specific endpoint": "Make a custom API call to a specific endpoint",
   "Notebook Name": "Notebook Name",
   "Notebook": "Notebook",
   "Section Name": "Section Name",
@@ -33,6 +35,15 @@
   "Content Type": "Content Type",
   "Content": "Content",
   "Heading Level": "Heading Level",
+  "Method": "Method",
+  "Headers": "Headers",
+  "Query Parameters": "Query Parameters",
+  "Body Type": "Body Type",
+  "Body": "Body",
+  "Response is Binary ?": "Response is Binary ?",
+  "No Error on Failure": "No Error on Failure",
+  "Timeout (in seconds)": "Timeout (in seconds)",
+  "Follow redirects": "Follow redirects",
   "The name of the notebook. Must be unique and cannot contain more than 128 characters or the following characters: ?*/:<>|'\"%~": "The name of the notebook. Must be unique and cannot contain more than 128 characters or the following characters: ?*/:<>|'\"%~",
   "The notebook to create the section in.": "The notebook to create the section in.",
   "The name of the section. Must be unique within the notebook and cannot contain more than 50 characters or the following characters: ?*/:<>|&#''%~": "The name of the section. Must be unique within the notebook and cannot contain more than 50 characters or the following characters: ?*/:<>|&#''%~",
@@ -57,6 +68,8 @@
   "The type of content to append.": "The type of content to append.",
   "The content to append to the page.": "The content to append to the page.",
   "The heading level (only for heading content type).": "The heading level (only for heading content type).",
+  "Authorization headers are injected automatically from your connection.": "Authorization headers are injected automatically from your connection.",
+  "Enable for files like PDFs, images, etc.": "Enable for files like PDFs, images, etc.",
   "Paragraph": "Paragraph",
   "List Item": "List Item",
   "Heading": "Heading",
@@ -67,6 +80,16 @@
   "H4": "H4",
   "H5": "H5",
   "H6": "H6",
+  "GET": "GET",
+  "POST": "POST",
+  "PATCH": "PATCH",
+  "PUT": "PUT",
+  "DELETE": "DELETE",
+  "HEAD": "HEAD",
+  "None": "None",
+  "JSON": "JSON",
+  "Form Data": "Form Data",
+  "Raw": "Raw",
   "New Note in Section": "New Note in Section",
   "Fires when a new note is created in a specified section.": "Fires when a new note is created in a specified section.",
   "The notebook to monitor for new notes.": "The notebook to monitor for new notes.",

packages/server/api/src/app/app.ts

@@ -110,6 +110,7 @@ import { variableModule } from './variable/variable.module'
 import { webhookModule } from './webhooks/webhook-module'
 import { engineResponseWatcher } from './workers/engine-response-watcher'
 
+import { workerCapacity } from './workers/machine/worker-capacity'
 import { migrateQueuesAndRunConsumers, workerModule } from './workers/worker-module'
 
 export const setupApp = async (app: FastifyInstance): Promise<FastifyInstance> => {
@@ -242,6 +243,7 @@ export const setupApp = async (app: FastifyInstance): Promise<FastifyInstance> =
     await app.register(alertsModule)
     await app.register(invitationModule)
     await app.register(workerModule)
+    await workerCapacity.setup()
     await app.register(oidcModule)
     await aiProviderService(app.log).setup()
     await app.register(aiProviderModule)

packages/server/api/src/app/database/migration/postgres/1797000000000-AddWorkerGroupsEnabledToPlatformPlan.ts

@@ -0,0 +1,35 @@
+import { QueryRunner } from 'typeorm'
+import { Migration } from '../../migration'
+
+export class AddWorkerGroupsEnabledToPlatformPlan1797000000000 implements Migration {
+    name = 'AddWorkerGroupsEnabledToPlatformPlan1797000000000'
+    breaking = false
+    release = '0.85.5'
+    transaction = true
+
+    public async up(queryRunner: QueryRunner): Promise<void> {
+        await queryRunner.query(`
+            ALTER TABLE "platform_plan"
+            ADD COLUMN IF NOT EXISTS "workerGroupsEnabled" boolean
+        `)
+        await queryRunner.query(`
+            UPDATE "platform_plan"
+            SET "workerGroupsEnabled" = false
+            WHERE "workerGroupsEnabled" IS NULL
+        `)
+        await queryRunner.query(`
+            ALTER TABLE "platform_plan"
+            ALTER COLUMN "workerGroupsEnabled"
+            SET DEFAULT false
+        `)
+        await queryRunner.query(`
+            ALTER TABLE "platform_plan"
+            ALTER COLUMN "workerGroupsEnabled"
+            SET NOT NULL
+        `)
+    }
+
+    public async down(queryRunner: QueryRunner): Promise<void> {
+        await queryRunner.query('ALTER TABLE "platform_plan" DROP COLUMN IF EXISTS "workerGroupsEnabled"')
+    }
+}

packages/server/api/src/app/database/migration/postgres/1798000000000-AddWorkerGroupIdToProject.ts

@@ -0,0 +1,24 @@
+import { QueryRunner } from 'typeorm'
+import { Migration } from '../../migration'
+
+export class AddWorkerGroupIdToProject1798000000000 implements Migration {
+    name = 'AddWorkerGroupIdToProject1798000000000'
+    breaking = false
+    release = '0.85.5'
+    transaction = true
+
+    public async up(queryRunner: QueryRunner): Promise<void> {
+        await queryRunner.query(`
+            ALTER TABLE "project"
+            ADD COLUMN IF NOT EXISTS "workerGroupId" character varying
+        `)
+        await queryRunner.query(`
+            CREATE INDEX IF NOT EXISTS "idx_project_worker_group" ON "project" ("workerGroupId")
+        `)
+    }
+
+    public async down(queryRunner: QueryRunner): Promise<void> {
+        await queryRunner.query('DROP INDEX IF EXISTS "idx_project_worker_group"')
+        await queryRunner.query('ALTER TABLE "project" DROP COLUMN IF EXISTS "workerGroupId"')
+    }
+}