perf: enable Gradle build cache and test parallelism to lower CI time#371
Conversation
- org.gradle.caching=true activates the local build cache, which is already persisted across runs by gradle/actions/setup-gradle@v5. On a typical PR touching one module, the other 17 modules get full cache-hits and are skipped entirely. - org.gradle.parallel=true makes this the default for local dev too (CI already passes --parallel on the command line). - maxParallelForks halves test execution time per module on 4-core+ machines (evaluates to 1 on the current 2-core runner, so no regression there).
🧪 Java Unit Tests
📦 Artifacts
🔁 Unreleased Commits8 commits since
|
Tests report quick summary:success ✅ > tests: 222, success: 222, skipped: 0, failed: 0 unfold for details
|
The all_go flow ran go get/go mod tidy for both tasks to pull github.com/go-gota/gota, which took ~50s per task in CI and caused RunnerTest.all_go to time out consistently. Rewrite both tasks using only encoding/csv and os from the Go standard library so the flow completes well within the test timeout.
QA Report — PR #371 — Fix validation: TriggerRunContext without reflectionTested on: 2026-06-03
Summary
Result: 5/5 PASS — 37 total executions created across ~5 minutes, zero "schedule is blocked since…" warnings Flow 1: go_script_trigger ✅ SUCCESSFlow YAMLid: go_script_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "go ScriptTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: go_script_fail
type: io.kestra.plugin.scripts.go.ScriptTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: golang:1.22
script: |
package main
import "os"
func main() { os.Exit(1) }Gantt
Logs synthesis Outputs synthesis Flow 2: node_script_trigger ✅ SUCCESSFlow YAMLid: node_script_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "node ScriptTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: node_script_fail
type: io.kestra.plugin.scripts.node.ScriptTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: node:20-slim
script: |
throw new Error("boom");Gantt
Logs synthesis Outputs synthesis Flow 3: node_commands_trigger ✅ SUCCESSFlow YAMLid: node_commands_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "node CommandsTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: node_commands_fail
type: io.kestra.plugin.scripts.node.CommandsTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: node:20-slim
commands:
- node -e "throw new Error('boom')"Gantt
Logs synthesis Outputs synthesis Flow 4: ruby_script_trigger ✅ SUCCESSFlow YAMLid: ruby_script_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "ruby ScriptTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: ruby_script_fail
type: io.kestra.plugin.scripts.ruby.ScriptTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: ruby:3.3-slim
script: |
raise "boom"Gantt
Logs synthesis Outputs synthesis Flow 5: ruby_commands_trigger ✅ SUCCESSFlow YAMLid: ruby_commands_trigger
namespace: qa.triggers
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message: "ruby CommandsTrigger fired: exitCode={{ trigger.exitCode }} condition={{ trigger.condition }}"
triggers:
- id: ruby_commands_fail
type: io.kestra.plugin.scripts.ruby.CommandsTrigger
interval: PT10S
exitCondition: "exit 1"
edge: true
containerImage: ruby:3.3-slim
commands:
- ruby -e "raise 'boom'"Gantt
Logs synthesis Outputs synthesis Scheduler healthZero |
2 participants
Problem
CI takes ~23 minutes. The dominant costs are:
./gradlew check shadowJar --parallel: ~11 min (Gradle check across 18 submodules)Root causes
gradle/actions/setup-gradle@v5already persists~/.gradle/caches(includingbuild-cache-1/) across CI runs, butorg.gradle.caching=truewas never set, so the cache sits unused.org.gradle.parallelonly set on CI CLI — local dev builds are sequential by default.maxParallelForks— each subproject's tests run in a single JVM even on multi-core machines.Changes
gradle.propertiesorg.gradle.caching=truegradle/actions/setup-gradle. On a PR that touches a single module, the other 17 modules get full cache-hits (compile + test skipped).gradle.propertiesorg.gradle.parallel=true--parallel).build.gradlemaxParallelForks = max(1, processors / 2)Expected impact
plugin-scriptbase (all 17 language plugins invalidated): Minimal improvement on first run; cache populated for the next run.Follow-ups (not in this PR)
plugins.ymlaccepts a parameterizedrunnerinput (defaultubuntu-latest, 2-core). Passing a 4-core runner label would double--parallelthroughput and makemaxParallelForks=2effective in CI. Needs confirmation of the available runner label in the Kestra GitHub org.org.gradle.configuration-cache=truecould save ~1 min of project configuration overhead, butnet.researchgate.releasehas known incompatibilities — needs testing.develocity-injection-enabledinkestra-io/actions/composite/setup-build/action.ymlwould extend caching across machines/branches for cross-PR cache hits.