Routines

[cryppadotta]

Plan a routines feature

A company often has regular routines. E.g. every morning at 10am, review the code changes from the past 24 hours and update the docs.

Routines can be triggered by time, schedules, or even some sort of trigger/ping like a webhook or some call from the api.

Routines have a definition of what should be done. Like an issue they have a project and an assignee. Between those things, the agent should know how to proceed with the work and deliver it.

When a routine needs to fire, it's work should be queued up and run by the agent, like already happens with an issue - things like concurrency of the agent should be respected.

Routines are sort like 'template issues' - that is, they aren't issues themselves but rather a template for what will happen.

Issues already are the tracking backbone of paperclip and it probably makes sense for routines to actualize a new issue on run - because we get so many operational benefits here - you can see the logs, comment on a particular run, auditing, work product all of it.

But maybe this creates a bit of an awkwardness around having a bunch of 'duplicate-seeming' routine copies of tasks in the feed. Possibly we should consider something like 'routine executions' as maybe an issue key or type to keep them out of some of the feeds or categorize them specially in some way.

I'll note as an aside that there are probably other types of "overloaded issues" we'll have in the future such as chats with the ceo that may also become "issue-backed" for consistency throughout the system, so this isn't a one-off idea.

Routines also need a meta-surface such as apis, paperclip skills updated and access. e.g. agents should be able to reflect on recurring work and have the tools to create and setup their own routines.

What I'd like you to do is think through more deeply about how we design this system of recurring routines. Define the timing and/or trigger system (webhooks and all the challenges around them, sharing keys and urls, giving agents the ability to make these webhook urls and share them with outside services. E.g. maybe an agent could create a webhook agent per routine like you would with discord or github and then give that url to an outside service to self-setup the routine trigger - all apis and actions etc. should support this).

And the ui as well, is there a routines entry in the sidebar? maybe under company. So we need ui for CRUD routines as well as the ability to audit the activity of routines, which again is maybe under the routines tab - then think about this idea of routine-executed issues, are then in the inbox view? issues view? recent activity? what filters do or do not set as defaults?

Give recommendations here

[cryppadotta]

Agent suggests:

---

Plan

Recommendation

Make routines a first-class company surface, but keep each execution issue-backed.

The core model should be:

• routine = durable definition/template owned by a company
• routine_trigger = schedule, webhook, or authenticated API/manual trigger attached to a routine
• routine_run = one trigger occurrence and its dispatch/audit record
• issue = the actual work item created from a routine run

That gives us the operational benefits of issues without forcing routine definitions themselves to behave like issues.

The main design recommendation is to add generic issue-origin metadata rather than a routine-only issue type. That solves the immediate routines use case and also leaves room for future issue-backed surfaces like chats, sync jobs, or other system-generated workflows.

1. Core data model

routines

A routine is a reusable work definition, scoped to a company.

Suggested fields:

• company_id
• project_id required for V1
• goal_id optional override if it differs from the project goal
• parent_issue_id optional stable parent for recurring child executions
• title
• instructions_body or description_template
• assignee_agent_id required
• priority
• status: active | paused | archived
• execution_mode: start with create_issue
• concurrency_policy: coalesce_if_active | always_enqueue | skip_if_active
• catch_up_policy: skip_missed | enqueue_missed_with_cap
• created_by_*, updated_by_*

Why project_id required for V1: recurring work still needs to trace back to a company goal chain, and forcing a project anchor keeps the routing clear.

routine_triggers

A routine can have multiple triggers.

Suggested kinds:

• schedule_cron
• webhook
• api

Suggested shared fields:

• routine_id
• kind
• enabled
• last_fired_at
• last_result
• created_by_*, updated_by_*

Schedule-specific fields:

• cron_expression using the same 5-field cron shape already used by plugin jobs
• timezone as an IANA zone
• next_run_at

Webhook-specific fields:

• public_id opaque random identifier used in the URL
• secret_ref into company secrets
• signing_mode: bearer | hmac_sha256
• replay_window_sec
• last_rotated_at

API trigger fields can stay minimal and use normal Paperclip auth rather than public secrets.

routine_runs

A routine run is the audit spine for one trigger occurrence.

Suggested fields:

• routine_id
• trigger_id
• status: received | coalesced | skipped | issue_created | dispatch_failed | completed
• triggered_at
• idempotency_key nullable
• trigger_payload jsonb nullable
• linked_issue_id nullable
• coalesced_into_run_id nullable
• failure_reason nullable

This table matters even if work is issue-backed. It gives a clean place to answer:

• did the trigger arrive?
• was it deduped or skipped?
• which issue did it create?
• what happened over time?

issues origin metadata

Do not make routines special by overloading issue status alone. Add generic origin metadata such as:

• origin_kind: manual | routine_execution | ...future
• origin_id
• origin_run_id

For routine executions, that means:

• origin_kind = routine_execution
• origin_id = routine_id
• origin_run_id = routine_run_id

This is the cleanest answer to the "future overloaded issues" problem mentioned in the task.

2. Execution lifecycle

Recommended flow:

1. A trigger fires.
2. Paperclip writes a routine_run row immediately.
3. The routine dispatcher checks the routine's concurrency policy.
4. If allowed, Paperclip creates a new issue from the routine template.
5. That issue is assigned to the routine assignee and enters normal issue/heartbeat flow.
6. The routine run links to the created issue.
7. When the issue reaches a terminal state, the routine run reflects the final outcome.

Concurrency recommendation

Default concurrency_policy should be coalesce_if_active for scheduled routines.

Reason: if a daily or hourly routine falls behind, blindly creating another identical issue will flood the system and make the feed worse. With coalescing:

• if a prior execution issue is still open, the new run is recorded
• the run is marked coalesced
• optionally the active issue gets a machine comment noting another trigger arrived

always_enqueue should exist, but mainly for webhook cases where every inbound event matters.

Missed runs / downtime

Use skip_missed as the default catch_up_policy.

If the server was down for 6 hours, Paperclip should not suddenly create 72 backfilled routine issues unless the routine explicitly opted in. For the opt-in path, cap backfill count per wake cycle.

3. Trigger system

Scheduled triggers

Use the same scheduler process already intended for heartbeats. Do not build a separate worker model for routines.

Implementation guidance:

• reuse the cron validation approach already present for plugin jobs
• store next_run_at
• evaluate in the routine timezone, not server local time
• handle DST by computing the next valid wall-clock occurrence in the declared timezone

Webhook triggers

Webhooks should be first-class routine triggers, not "webhook agents".

The outside system should call Paperclip, and Paperclip should decide whether to create an execution issue. That preserves governance, company scoping, auditing, and future policy checks.

Recommended webhook design:

• generate an opaque public URL per trigger
• generate a secret once and store only the secret reference
• support bearer token or HMAC signature verification
• support caller-supplied idempotency keys
• log delivery metadata and a sanitized payload preview
• rate limit per trigger
• support secret rotation without changing the routine itself

This is also where Paperclip should let agents help with setup:

• an agent can create a webhook trigger for a routine it owns or is assigned to manage
• the API returns the URL and secret once
• the agent can paste/share that URL with an external service
• the secret material lives in company_secrets, not in issue comments or adapter config

Authenticated API/manual triggers

Add two non-public entry points:

• Run now from the UI
• authenticated API call for internal systems or agents

These should flow through the same routine_run creation path as schedules and webhooks so all audit trails stay unified.

4. UI recommendation

Add a dedicated Routines entry under the company surface in the sidebar.

The routines area should have:

• list view: status, assignee, trigger summary, next run, last run result, active linked issue
• detail view: definition, triggers, recent runs, linked execution issues, activity
• create/edit flow: routine definition plus trigger setup
• audit surface: runs table with filters for status, trigger kind, and date

Detail page tabs

Recommended tabs:

• Overview
• Triggers
• Runs
• Execution Issues
• Activity

For webhook triggers, the detail view should show:

• endpoint URL
• masked auth info
• last delivery time
• recent failures
• rotate secret action

5. Feed and inbox behavior

Routine executions should be visible, but not noisy by default.

Recommendations:

• Inbox: do not surface routine execution issues by default just because they were system-created
• still surface them when they become relevant to a human, such as unread mentions, approvals, failures, or explicit following/touching
• Issues list: default filter hides origin_kind = routine_execution, with an explicit toggle/filter to include them
• Recent activity: show higher-level routine run events with links to the linked issue instead of treating every execution as a top-level surprise item
• Routine detail: always show the full execution issue history there

This keeps the main work surfaces readable while preserving audit depth.

6. API surface

Suggested endpoints:

• GET /api/companies/:companyId/routines
• POST /api/companies/:companyId/routines
• GET /api/routines/:id
• PATCH /api/routines/:id
• POST /api/routines/:id/triggers
• PATCH /api/routine-triggers/:id
• POST /api/routine-triggers/:id/rotate-secret
• POST /api/routines/:id/run
• GET /api/routines/:id/runs
• GET /api/routine-runs/:id
• POST /api/routine-triggers/public/:publicId/fire

Routine creation/update rules should follow the existing invariants:

• company-scoped access checks everywhere
• activity log entries for all mutations
• actor checks for board vs agent
• secret values only revealed on creation/rotation

7. Agent-facing behavior

Agents should gain routine tools, but with clear boundaries.

Recommended abilities:

• list routines in the company or relevant project
• inspect routine definitions and recent runs
• create/update routines assigned to themselves
• create webhook triggers and receive the one-time URL/secret material
• manually fire routines they own

Guardrails:

• board can manage any routine
• agents can only manage routines within company scope
• if a routine exposes a public webhook, log that as a notable governance action
• if later needed, public-webhook creation can become approval-gated without changing the model

8. Implementation shape

Recommended rollout:

Phase 1

• schema for routines, routine_triggers, routine_runs
• issue origin metadata
• scheduled triggers only
• create-issue execution path
• routines list/detail UI
• issues/inbox filters for routine executions

Phase 2

• webhook trigger support
• secret generation and rotation
• delivery logging and idempotency handling
• agent routine CRUD tools

Phase 3

• richer coalescing controls
• backfill policies
• better analytics on routine health and SLA

9. Key recommendation summary

The strongest design choice here is:

1. make routines first-class definitions
2. materialize each execution as a normal issue
3. track trigger occurrences separately in routine_runs
4. add generic issue-origin metadata so future issue-backed surfaces fit the same model
5. hide routine execution noise from default feeds while keeping full auditability

That stays aligned with Paperclip's current architecture: issues remain the work backbone, the existing scheduler remains the timing backbone, and webhook/security concerns live in company-scoped server APIs instead of being pushed down into ad hoc agent configs.

[cryppadotta]

I'm basically happy with this. It keeps the "issue" object as the core runtime primitive but gives nice ux considerations around it (not polluting the feed etc.)

[bayological]

Ship it

[phjlljp]

Looks great to me. I want it!

[gsxdsm]

Love it. Ship

[luxingxiao]

Ship it

[bshih]

My only suggestion is to add support for labeling these issues to make filtering / visual inspection in the issue list easier. I built something related where scripts sometimes will auto-create new issues for agents to work on at the end of their run time (e.g., after a long non-agent parameter optimization script runs, it creates an issue and assigns a Researcher to review and comment on the results). In my workflow, I label all these with "Auto" so it's easy to see at a glance which ones are generated by this process.