fix(triggers): prevent evaluate() exceptions from permanently blocking the scheduler#369
Conversation
When runOnce() throws an unexpected exception in ScriptTrigger or CommandsTrigger, the exception propagated out of evaluate(). The Kestra scheduler's onTriggerEvaluated handler updates the next evaluation date but does not unlock the trigger when the evaluation result is null (the state sent back when an exception escapes evaluate()). This leaves the trigger permanently locked, producing the 'schedule is blocked since' warning indefinitely. Fix by wrapping runOnce() in a try-catch inside evaluate() for all affected triggers: Node, Ruby, Go, Shell, and Python. An unhandled exception is now logged as WARN and evaluate() returns Optional.empty() instead of propagating, keeping the trigger healthy. Also replace the two Ruby ScriptTriggerTest integration tests that required the Ruby runtime (not available on all CI machines) with unit tests that validate the condition-matching and edge-mode logic directly via the Output model, matching the approach already used in Go's ScriptTriggerTest.
📦 Artifacts
🧪 Java Unit Tests
🔁 Unreleased Commits1 commits since
|
||||||||||||||||||||||||||||||||||||||||||||||
Tests report quick summary:failed ❌ > tests: 50, success: 46, skipped: 0, failed: 4
Failed tests:plugin-script-go > io.kestra.plugin.scripts.go.CommandsTriggerTest > commandsTrigger_shouldTriggerOnStdoutMatchUsingStructuredOutputs() failed ❌ in 17.240plugin-script-go > io.kestra.plugin.scripts.go.CommandsTriggerTest > commandsTrigger_shouldMatchRegexAgainstStructuredOutputs() failed ❌ in 0.124plugin-script-go > io.kestra.plugin.scripts.go.CommandsTriggerTest > commandsTrigger_shouldTriggerOnImplicitFailureExit1() failed ❌ in 0.050plugin-script-go > io.kestra.plugin.scripts.go.CommandsTriggerTest > commandsTrigger_edgeModeShouldSuppressSecondEmission() failed ❌ in 0.068 |
QA Report — PR #369 — Pradumna repro flows (corrected)Tested on: 2026-05-26
Summary
Result: 5/5 PASS — zero "schedule is blocked since…" warnings, all 5 log tasks executed with correct trigger outputs Flow 1: go_script_trigger ✅ SUCCESSFlow YAMLid: go_script_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "go ScriptTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: go_script_fail
type: io.kestra.plugin.scripts.go.ScriptTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: golang:1.22
script: |
package main
import "os"
func main() { os.Exit(1) }Gantt
Logs synthesis Outputs synthesis Flow 2: node_script_trigger ✅ SUCCESSFlow YAMLid: node_script_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "node ScriptTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: node_script_fail
type: io.kestra.plugin.scripts.node.ScriptTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: node:20-slim
script: |
throw new Error("boom");Gantt
Logs synthesis Flow 3: node_commands_trigger ✅ SUCCESSFlow YAMLid: node_commands_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "node CommandsTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: node_commands_fail
type: io.kestra.plugin.scripts.node.CommandsTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: node:20-slim
commands:
- node -e "throw new Error('boom')"Gantt
Logs synthesis Flow 4: ruby_script_trigger ✅ SUCCESSFlow YAMLid: ruby_script_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "ruby ScriptTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: ruby_script_fail
type: io.kestra.plugin.scripts.ruby.ScriptTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: ruby:3.3-slim
script: |
raise "boom"Gantt
Logs synthesis Flow 5: ruby_commands_trigger ✅ SUCCESSFlow YAMLid: ruby_commands_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "ruby CommandsTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: ruby_commands_fail
type: io.kestra.plugin.scripts.ruby.CommandsTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: ruby:3.3-slim
commands:
- ruby -e "raise 'boom'"Gantt
Logs synthesis Scheduler health (all 5 triggers)Zero |
|
Hi @Pradumnasaraf 👋 This should be fixed, see my QA report and check the JAR from this PR to recheck if you have some time 🙏 Thanks! |
Pradumnasaraf
left a comment
Pradumnasaraf
left a comment
There was a problem hiding this comment.
Tested PR #369 on a clean instance with the built JARs (plugin-script-* 1.7.6-SNAPSHOT). The Go, Ruby and Node triggers still do not create executions. They show No execution found, schedule is blocked since ... and never recover, same as before.
AbstractLogConsumer never stores raw log lines — it only increments line counts and parses ::outputs:: markers. The logs field was being set via getLogConsumer().toString() which always produced a useless object reference (e.g. DefaultLogConsumer@5c3e30c3). Remove logs from: - ExtractedFailure record (no longer needed) - extractFailure() method body - Output class (public API field removed) - buildHaystack() (only vars remain for regex/substring matching) Affects all 9 trigger classes across go, node, python, ruby, shell.
AbstractExecScript defaults taskRunner to Docker, but every trigger's runOnce() was overriding it with Process.instance(), causing: - containerImage property to be silently ignored - scripts to run on the Kestra host (requiring Go/Ruby/etc installed) - trigger to never fire if the language runtime was absent on the host Remove the Process.instance() override and its import so the Script/ Commands task uses its Docker default, actually running inside the configured containerImage (golang, node, ruby, python, ubuntu...).
|
Hi @Pradumnasaraf 👋 tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "go ScriptTrigger fired: exitCode={{ trigger.exitCode ?? '' }} condition={{ trigger.condition ?? '' }}"You need to protect with 
With expected logs: 
|
There was a problem hiding this comment.
Hey @fdelbrayelle,
I tried the ?? '' version on a develop build, no PR JAR, but I get the same result as before.
The trigger still logs No execution found, schedule is blocked since and no execution is created at all. The execution list for the flow is empty, not even a failed one. And the trigger row in the DB shows locked: true with lastTriggeredDate set
Flow used:
id: go_script_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "go ScriptTrigger fired: exitCode={{ trigger.exitCode ?? '' }} condition={{ trigger.condition ?? '' }}"
triggers:
- id: go_script_fail
type: io.kestra.plugin.scripts.go.ScriptTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: golang:1.22
script: |
package main
import "os"
func main() { os.Exit(1) }
|
Hi @Pradumnasaraf 👋 Strange... 🤔 What about the PR JAR? Same result? |
Pradumnasaraf
left a comment
Pradumnasaraf
left a comment
There was a problem hiding this comment.
Hey @fdelbrayelle, still the same issue. I even pulled the PR locally, built the JAR, and tested it, just to make sure the JAR generated by CI is correct.
|
Seems good for me. I checked with the above JAR in the comment. id: go_script_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "go ScriptTrigger fired: exitCode={{ trigger.exitCode ?? '' }} condition={{ trigger.condition ?? '' }}"
triggers:
- id: go_script_fail
disabled: true
type: io.kestra.plugin.scripts.go.ScriptTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: golang:1.22
script: |
package main
import "os"
func main() { os.Exit(1) }
|
Pradumnasaraf
left a comment
Pradumnasaraf
left a comment
There was a problem hiding this comment.
If it is working for you both, let's go ahead and merge this PR. It might be some configuration issue on my side.
QA Report — PR #369 — same scenarios as previous QA commentTested on: 2026-06-03 Summary
Result: 0/5 PASS — all triggers fail with a NullPointerException on every evaluation cycle Root causeEvery trigger evaluation fails with the same NPE: Chain of events:
This is a regression introduced by commit Per-flow detailsAll 5 flows used the same YAML as the previous QA comment. Each produces identical symptoms: Flow 1–5: all triggers (❌ KO — NPE every cycle)Logs synthesis (same for all 5 triggers)
Outputs synthesis Additional observations
|
QA Report — PR #369 — Post-fix validation (TriggerRunContext reflection removed)Tested on: 2026-06-03
Summary
Result: 5/5 PASS — 53 total executions created in ~5 minutes, zero "schedule is blocked since…" warnings Flow 1: go_script_trigger ✅ SUCCESSFlow YAMLid: go_script_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "go ScriptTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: go_script_fail
type: io.kestra.plugin.scripts.go.ScriptTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: golang:1.22
script: |
package main
import "os"
func main() { os.Exit(1) }Gantt
Logs synthesis Outputs synthesis Flow 2: node_script_trigger ✅ SUCCESSFlow YAMLid: node_script_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "node ScriptTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: node_script_fail
type: io.kestra.plugin.scripts.node.ScriptTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: node:20-slim
script: |
throw new Error("boom");Gantt
Logs synthesis Flow 3: node_commands_trigger ✅ SUCCESSFlow YAMLid: node_commands_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "node CommandsTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: node_commands_fail
type: io.kestra.plugin.scripts.node.CommandsTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: node:20-slim
commands:
- node -e "throw new Error('boom')"Gantt
Logs synthesis Flow 4: ruby_script_trigger ✅ SUCCESSFlow YAMLid: ruby_script_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "ruby ScriptTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: ruby_script_fail
type: io.kestra.plugin.scripts.ruby.ScriptTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: ruby:3.3-slim
script: |
raise "boom"Gantt
Logs synthesis Flow 5: ruby_commands_trigger ✅ SUCCESSFlow YAMLid: ruby_commands_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "ruby CommandsTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: ruby_commands_fail
type: io.kestra.plugin.scripts.ruby.CommandsTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: ruby:3.3-slim
commands:
- ruby -e "raise 'boom'"Gantt
Logs synthesis Scheduler health (all 5 triggers)Zero |
3 participants
Summary
runOnce()in a defensive try-catch insideevaluate()for all polling triggers (Node, Ruby, Go, Shell, Python). When an unexpected exception escapesevaluate(), the Kestra scheduler'sonTriggerEvaluatedhandler receives a null result, updates the next evaluation date, but does not unlock the trigger — leaving it permanently locked and producing the 'No execution found, schedule is blocked since...' warning indefinitely.evaluate(), logs it as WARN, and returnsOptional.empty(), keeping the trigger healthy for the next poll cycle.ScriptTriggerTestintegration tests that required the Ruby runtime (not available on all CI machines) with unit tests that validate condition-matching and edge-mode logic directly via theOutputmodel — matching the approach already used in Go'sScriptTriggerTest.closes: #319 #320 #321
Test plan
./gradlew :plugin-script-ruby:test :plugin-script-node:test :plugin-script-go:testpasses (0 failures)plugin-script-python(python not found, uv install failures, Docker tests) andplugin-script-shell(DockerBash tests) are unchanged — confirmed by running the same tests againstmainHEAD./gradlew shadowJarbuilds