Cron lane self-deadlock: isolated agentTurn jobs never execute with default maxConcurrentRuns

[JorgeMenaDev]

Summary

Isolated agentTurn cron jobs deadlock on the cron lane when cron.maxConcurrentRuns is 1 (the default). Sessions get created in sessions.json but the LLM call never fires — transcript files are 0 bytes/missing, and jobs always hit their timeout.

Version / OS

• OpenClaw: 2026.3.8 (stable)
• OS: macOS 15.6 (arm64)
• Node: v24.13.0

Repro (minimal)

1. cron.maxConcurrentRuns at default (1), or explicitly set to 1
2. Create any isolated cron with payload.kind: "agentTurn":

   {
     "sessionTarget": "isolated",
     "payload": { "kind": "agentTurn", "message": "Reply: OK", "timeoutSeconds": 60 }
   }

3. Trigger via openclaw cron run <jobId> or wait for schedule
4. Observe:
   • Session entry created in sessions.json ✅
   • Transcript .jsonl file missing or 0 bytes ❌
   • Job times out ❌
   • Gateway logs show lane wait exceeded: lane=cron waitedMs=32586 queueAhead=0

Root cause

The cron runner queues work on the cron lane. The isolated agent turn (runEmbeddedPiAgent) is also invoked with lane: "cron". With maxConcurrentRuns: 1:

1. Cron runner takes slot 1 on the cron lane
2. Embedded agent run queues behind it on the same lane
3. Agent run cannot start — only 1 slot available
4. Cron runner cannot finish — waiting for agent run
5. Deadlock until timeout kills everything

Workaround

Set cron.maxConcurrentRuns: 2 and restart:

// ~/.openclaw/openclaw.json
{
  "cron": {
    "maxConcurrentRuns": 2
  }
}

openclaw gateway restart

After this, the same canary cron completes in 4.6 seconds.

Affects

• All embedded providers tested: Anthropic (claude-opus-4-6, claude-sonnet-4-6), GLM (glm-4.7)
• Multiple agents tested: custom agents + main agent
• Sub-agents spawned from live chat sessions work perfectly (same models, same tasks)
• Only cron-initiated isolated sessions are affected

Log snippet

[diagnostic] lane wait exceeded: lane=cron waitedMs=32586 queueAhead=0

Suggestion

The cron isolated agent runner should not enqueue the embedded run onto the same cron lane the cron service is executing on. Options:

• Use a distinct lane name for the embedded run (e.g. cron-agent)
• Avoid lane nesting entirely
• Or at minimum, set the default maxConcurrentRuns to 2+ so the default config works out of the box

[Kaze310]

We independently traced the same deadlock from source inspection and found two extra implementation details that may help narrow the fix.

1. sessionFile staying null in the session store appears to be a real code-path issue, not just fallout from the timeout.

   In the isolated cron path, the code creates/persists the session entry first, then computes a local transcript path and passes it down to the embedded runner. But unlike the normal session init flow, it does not appear to persist that sessionFile path back into the session store entry. So sessions.json can legitimately remain with sessionFile: null even before the deadlock is resolved.

2. There seems to be a second, separate transcript replay problem later in the same broad execution path.

   In failing runs we also saw repeated errors of the form:

   thinking or redacted_thinking blocks in the latest assistant message cannot be modified

   From source inspection this looks like a transcript sanitization / replay compatibility issue rather than a provider-only limitation. OpenClaw already has downgrade/sanitization helpers for reasoning blocks, but this path may still be incomplete for some responses-style models.

So at least from what we saw, there are probably two distinct isolated-cron bugs here:

• the cron-lane self-deadlock that prevents the embedded run from starting at all
• a later transcript replay / reasoning-block compatibility failure that can break runs even after startup issues are bypassed

The deadlock is enough to explain the "session entry created, no transcript, no run" mode. The thinking error looks like a second bug, not the same one.

[Kaze310]

One practical workaround we validated without changing cron.maxConcurrentRuns is to avoid the isolated path entirely.

This does not make non-default agents directly reliable as cron executors under the current constraints. Instead, it avoids the deadlocking path:

• keep the cron job on the default agent
• use sessionTarget: "main"
• use payload.kind: "systemEvent"
• give each cron job its own dedicated sessionKey
• if a cheaper model is needed, apply the model override at the session level for that dedicated session key rather than relying on job-level payload.model

That combination avoids the isolated agentTurn path entirely, so it works without raising cron lane concurrency.

So there are really two separate needs here:

1. fix isolated cron so non-default agents can run reliably without deadlocking
2. make the stable main + systemEvent path support first-class job/session model selection so users do not need session-store workarounds

[newcodebook]

You can treat this as an isolated-cron execution-path bug rather than a prompt/model issue.

Workaround (avoid the deadlocking path): move the job off sessionTarget: "isolated" + payload.kind: "agentTurn".

1. Switch the job to the main session + system event path:

openclaw cron edit <job-id> --session main --system-event "Reply: OK"

2. Give each cron job a dedicated session key (so runs don’t compete with an interactive session), then retest:

openclaw cron edit <job-id> --session-key "cron:<job-id>"
openclaw cron run <job-id>
openclaw cron runs --id <job-id> --limit 10

Why this helps: main + systemEvent routes through the normal heartbeat/session flow and avoids the isolated cron-lane + embedded agent-turn path that can deadlock when cron.maxConcurrentRuns=1.

If you must stay on isolated agentTurn, please paste openclaw cron get <job-id> plus the latest openclaw cron runs --id <job-id> --limit 20 so we can see whether you’re hitting the lane deadlock vs the later transcript replay (thinking block) failure.

Recommended reading:

• OpenClaw Cron & Heartbeat: Make Your Agent Actually Run 24/7
• Cron: isolated jobs fail with 'Field required' on tools.function

[thehawkeye]

Validation report (OpenClaw 2026.3.8, macOS, Discord):\n\n- Symptom matched cron-lane stalls for isolated agentTurn delivery in admin channel.\n- Mitigation applied: cron.maxConcurrentRuns=2\n- Post-mitigation results:\n - job afd2e49e-737b-4cdf-a14d-768f1a09e97f: status=ok, deliveryStatus=delivered, summary=CRON_STABLE_OK\n - visible Discord message id: 1481251151109292143\n- One failure was observed only during gateway restart drain window (GatewayDrainingError), not steady-state.\n\nOperationally this workaround stabilizes isolated+announce for Discord in steady state.\n

[momchilon]

Bug Report: isolated cron + announce enqueues but never materializes or delivers

Summary

On this OpenClaw instance, main cron jobs execute and record run history, but isolated cron jobs with delivery.mode = announce appear to enqueue successfully and then never materialize into a finished run record or outbound delivery.

This affects at least Telegram direct-message delivery and reproduces even with a minimal ping prompt, so it does not appear to be weather-skill-specific.

Environment

• OpenClaw gateway app version observed in status/logs: 2026.3.8
• Host: Linux
• Channel under test: Telegram direct chat
• Active agent: main
• Gateway healthy, Telegram channel healthy

What the docs say

Relevant docs indicate this should work:

• docs/automation/cron-jobs.md
• docs/cli/cron.md
• docs/concepts/session-tool.md

Key documented expectations:

• openclaw cron run <jobId> only queues the run; later outcome should appear in openclaw cron runs --id <jobId>
• Isolated jobs with delivery.mode = announce should deliver directly via channel adapters
• If delivery is omitted for isolated jobs, default is announce
• Isolated cron run sessions should appear in session store and have retention via cron.sessionRetention

Actual behavior

Main-session cron

• Runs successfully
• Produces finished run entries in cron.runs
• If no delivery was requested, deliveryStatus: not-requested
• Quiet-hours behavior also works as expected

Isolated + announce cron

• Job creation succeeds
• Manual run returns:
  • { ok: true, enqueued: true, runId: ... }
• But afterward:
  • no finished cron.runs entry
  • no ~/.openclaw/cron/runs/<jobId>.jsonl output file
  • no telegram sendMessage ok chat=<id> log line
  • often no usable transcript/session materialization beyond a registry stub

Minimal reproduction

Repro job

openclaw cron add \
  --name "cron ping explicit main" \
  --at "5m" \
  --agent main \
  --session isolated \
  --message "Reply with exactly: cron ping explicit main ok" \
  --announce \
  --channel telegram \
  --to "5761957901" \
  --model gemini-flash-lite \
  --delete-after-run

CLI returned a created job like:

• job id: 5cd7d2c8-2633-4f5f-b21b-0287a04593c5
• sessionTarget: "isolated"
• delivery.mode: "announce"
• delivery.channel: "telegram"
• delivery.to: "5761957901"

Manual run

openclaw cron run 5cd7d2c8-2633-4f5f-b21b-0287a04593c5

Observed response:

{
  "ok": true,
  "enqueued": true,
  "runId": "manual:5cd7d2c8-2633-4f5f-b21b-0287a04593c5:1773265236839:4"
}

Expected

• A finished run entry should later appear in:

  openclaw cron runs --id 5cd7d2c8-2633-4f5f-b21b-0287a04593c5

• A Telegram message should be delivered to the target direct chat
• Optionally, an isolated run session/transcript should exist per docs

Actual

After waiting, still got:

{
  "entries": [],
  "total": 0,
  "offset": 0,
  "limit": 10,
  "hasMore": false,
  "nextOffset": null
}

And no Telegram delivery occurred.

Related tests already performed

1) Weather isolated announce cron

Created isolated Telegram announce cron for a Belgrade weather forecast.

• job id: 0a2426e1-3b07-45da-898b-fa766f0f3de8
• behavior: enqueued, but no finished run record / no delivery

2) Earlier isolated weather cron

• job id: 9253bac3-9310-4f11-9953-832dae9b1095
• scheduler created cron session registry entries like:
  • agent:main:cron:9253bac3-9310-4f11-9953-832dae9b1095
• but there was no usable transcript/run completion/delivery

3) Main-session weather cron

• job id: 602847a1-d409-4989-b77f-46858b3dfa5d
• manual run initially skipped due to quiet-hours
• after extending active hours and restarting gateway, run succeeded with:
  • status: ok
  • deliveryStatus: not-requested
• so main-session cron runner itself appears functional

Evidence that Telegram itself is fine

Gateway logs show normal successful DM sends in this same chat context, e.g. many lines like:

• telegram sendMessage ok chat=5761957901 message=...

So this is not a generic Telegram channel outage.

Evidence that scheduler itself is fine

Gateway logs show normal cron lifecycle for job creation:

• cron: started
• cron: job added
• cron: job created
• timer arming / next wake timestamps

So this is not a generic scheduler-disabled problem.

Suspected failure point

The failure seems to be somewhere between:

1. manual/scheduled queue acceptance for an isolated run, and
2. first materialized isolated agent-turn execution / run persistence / announce delivery

In other words, isolated cron announce jobs appear to be accepted by the scheduler but do not complete into a durable run result.

Notable observations

• Explicit --agent main did not fix it
• Minimal prompt without skills did not fix it
• Restarting the gateway did not fix it
• Quiet-hours were a red herring for main-session tests only
• Docs do not mention an extra required config knob for isolated Telegram announce delivery in direct chats

Potential areas to inspect upstream

• Isolated cron run persistence path
• Cron run completion bookkeeping for isolated sessions
• Announce delivery pipeline for cron jobs (not subagents)
• Interaction between isolated cron sessions and session-store/transcript creation
• Whether cron.runs is only written after a code path that is never reached for isolated jobs

Nice-to-have diagnostics upstream

Helpful extra logging would be anything around:

• isolated cron run session creation success/failure
• transcript creation path failures
• announce delivery target resolution for cron jobs
• per-run state transitions before cron.runs persistence

User impact

This blocks a documented and useful pattern:

• recurring isolated background job
• direct channel announcement delivery

Example use case blocked here:

• daily 8am weather forecast to Telegram DM

[emilyorz]

Reproduced on OpenClaw 2026.3.8 with isolated agentTurn cron jobs timing out under the default cron concurrency.

We tested the workaround suggested here by setting:

{
  "cron": {
    "maxConcurrentRuns": 2
  }
}

After that change, we validated two successful scheduler-path runs (not just a manual trigger):

1. a minimal isolated canary job
2. a staging job closely matching the failing real-world pattern (isolated + agentTurn + announce + script exec + timeoutSeconds: 300)

Both completed successfully and delivered normally after the config change.

This seems to strongly support the lane/concurrency deadlock theory in this issue, at least for 2026.3.8 under the default cron.maxConcurrentRuns behavior.

[newcodebook]

This looks like the cron-lane deadlock path, not a Telegram-specific delivery problem.

1. Raise cron.maxConcurrentRuns to 2 in ~/.openclaw/openclaw.json, then restart the gateway:

   {
     "cron": {
       "maxConcurrentRuns": 2
     }
   }

   openclaw gateway restart

2. Re-run the job and check that the run materializes instead of staying enqueued:

   openclaw cron run <job-id>
   openclaw cron runs --id <job-id> --limit 10

Why this helps: with the default maxConcurrentRuns: 1, the cron service can occupy the only cron lane slot while the isolated agentTurn it spawned is queued onto that same lane, so the job waits on itself until timeout. The reports here that started succeeding after raising concurrency are consistent with that.

If you cannot change concurrency right now, the lower-risk workaround is to avoid the isolated agentTurn path entirely: use sessionTarget: "main" + payload.kind: "systemEvent" with a dedicated sessionKey for that cron job.

If it still stalls after maxConcurrentRuns: 2, please paste openclaw cron get <job-id> plus the latest openclaw cron runs --id <job-id> --limit 20 so it is easier to separate this deadlock from a later transcript-replay failure.

Recommended reading:

• OpenClaw Cron & Heartbeat: Make Your Agent Actually Run 24/7
• Cron: isolated jobs fail with 'Field required' on tools.function

[frankekn]

Closing as fixed on main for the isolated cron embedded-lane deadlock family.

Root cause: isolated cron agentTurn runs could enter the embedded agent path with the cron lane, causing the cron execution to wait on work that was queued back onto the same lane.

Fix on main:

• 825a43570 — fix: avoid cron embedded lane deadlock
• fa0329c34 — test: cover cron nested lane selection

Important version boundary:

• this fix is on main
• it is not included in release v2026.3.8
• users on 2026.3.8 can still reproduce until a newer release includes these commits

If you still reproduce this on a build containing 825a43570 or newer, please open a new issue with the exact build SHA and whether the job is isolated/agentTurn or main/systemEvent.