1.2.68

• A job's resourceLimits caps the CPU, memory, and process count of each of
  its runs, such as cpu: 1.5, memory: 512M, and processes: 64. The caps
  use cgroup v2 on Linux and Job Objects on Windows, and DAG tasks and verify
  checks take them too.
• A run that goes over its memory cap is killed with every process it
  started and fails with memory limit exceeded (512 MiB). A forced cancel of
  a capped run reaches processes that left its process group.
• Where the host cannot enforce a cap, as on macOS, the BSDs, and illumos, the
  run goes ahead without it, the daemon logs a warning, and the run record
  says so. With resourceLimits.required: true, the run fails instead.
• The systemd unit in the .deb and .rpm packages sets
  Delegate=memory pids, so memory and process caps work, and cronstable
  and its jobs share CPU with the other system services by nice value. CPU
  caps need Delegate=yes in a drop-in from systemctl edit cronstable.
  With OOMPolicy=continue, the service keeps running when a run is killed
  at its memory cap.
• With Delegate=yes, while a job has a cpu cap, every run on Linux gets
  a cgroup whose CPU weight follows its priority.
• The dashboards, the HTTP API, Prometheus, statsd, and report templates show
  each job's caps, whether a run's caps applied, and memory-cap kills. The
  dashboards also mark a run whose verify check was killed at its memory cap.
• The terminal dashboard's Resources tab shows the live CPU and memory of a
  running job.

Performance compared with 1.2.67

Regression check: passed. No compared metric exceeded its regression thresholds.

Both versions were benchmarked in alternating rounds on the same machine. Results use the lowest runtime and median memory usage across rounds. Negative changes mean less time or memory; positive changes mean more.

Benchmark comparisons (199 metrics)

Noise (±) estimates run-to-run variation across both versions. A result fails the regression check only if the increase exceeds its percentage limit, its minimum absolute change, and 2 times the noise estimate. n/a means too few rounds to estimate noise; the other limits still apply.

The Regression limit column accounts for the minimum absolute change, expressed as a percentage of the baseline. Not enforced marks informational metrics.

Startup times include Python startup overhead. Their percentage changes exclude that overhead when it can be measured separately and leaves a positive runtime in both versions.

Benchmark	1.2.67	1.2.68	Change	Noise (±)	Regression limit
schedule.due_pass_100k	15.75 ms	14.37 ms	-8.8%	1.1%	63.48% (declared 25%)
webapi.trends_build_5k	150.78 ms	162.78 ms	+8.0%	1.3%	15%
webui.tick_500	78.00 ms	83.60 ms	+7.2%	4.9%	25%
cronexpr.test_match_200k	24.61 ms	26.06 ms	+5.9%	5.0%	40.63% (declared 15%)
webui.dirty_frame_rows_500	853.7 us	804.1 us	-5.8%	38.7%	Not enforced
webui.select_move_500	61.30 ms	58.00 ms	-5.4%	1.5%	25%
schedule.pressure_20k_48h	17.51 ms	16.71 ms	-4.5%	2.6%	57.13% (declared 15%)
schedule.duplicates_20k	18.12 ms	17.31 ms	-4.5%	2.5%	55.19% (declared 15%)
webapi.sse_burst_20k	48.38 ms	46.38 ms	-4.2%	1.5%	15%
mem.gc_pause_100k	159.85 ms	154.19 ms	-3.5%	8.1%	25%
startup.import_cronexpr	25.75 ms	25.48 ms	-2.6%	1.0%	25%
config.jobconfig_3k	68.38 ms	70.11 ms	+2.5%	1.5%	15%
job.launch_reap_2k	156.35 ms	160.13 ms	+2.4%	1.8%	15%
state.gc_sweep_2k_streams	33.18 ms	33.95 ms	+2.3%	1.3%	45.21% (declared 25%)
startup.import_daemon	182.68 ms	186.46 ms	+2.3%	0.4%	25%
loop.stall_launch_herd_8k	78.33 ms	80.11 ms	+2.3%	1.8%	Not enforced
webui.boot_rows_500	415.10 ms	424.20 ms	+2.2%	4.0%	Not enforced
dag.plan_claim_10k	14.02 ms	13.72 ms	-2.2%	2.6%	71.31% (declared 15%)
job.launch_fail_retry_2k	178.71 ms	182.61 ms	+2.2%	0.9%	15%
config.parse_crontab_1k	12.48 ms	12.75 ms	+2.2%	0.5%	80.11% (declared 15%)
prometheus.counter_snapshot_2k	123.39 ms	126.00 ms	+2.1%	0.9%	15%
mem.launch_reap_steady_20x55	0.37 MB	0.38 MB	+2.1%	0.0%	15%
mem.jobconfig_2k	1.63 MB	1.66 MB	+1.9%	0.2%	30.77% (declared 15%)
webapi.index_gzip_bytes	183.06 KB	186.56 KB	+1.9%	0.0%	15%
webapi.index_bytes	639.81 KB	651.90 KB	+1.9%	0.0%	15%
mcp.list_jobs_page_500	152.02 ms	154.84 ms	+1.9%	0.7%	15%
startup.import_config	96.52 ms	97.93 ms	+1.9%	0.7%	25%
tui.wallboard_paint_5k	184.84 ms	181.45 ms	-1.8%	0.4%	15%
job.stream_passthrough_240k	152.16 ms	149.53 ms	-1.7%	1.7%	15%
dag.build_chain_10k	30.32 ms	29.81 ms	-1.7%	1.8%	32.98% (declared 15%)
state.inventory_2k_streams	93.47 ms	92.01 ms	-1.6%	1.5%	Not enforced
loop.stall_mcp_jobs_500	8.61 ms	8.74 ms	+1.4%	0.9%	Not enforced
fingerprint.job_set_id_rich_2k	256.50 ms	252.85 ms	-1.4%	1.9%	15%
webui.tail_term_2k	56.80 ms	57.60 ms	+1.4%	5.1%	25%
state.boot_rehydrate_populated	41.89 ms	42.47 ms	+1.4%	1.7%	23.87% (declared 15%)
dag.build_layered_10k	50.61 ms	49.93 ms	-1.4%	1.5%	19.76% (declared 15%)
loop.stall_completions_500	10.15 ms	10.28 ms	+1.3%	1.7%	49.24% (declared 25%)
schedule.reseed_local_20k	95.54 ms	94.35 ms	-1.2%	1.8%	15%
webapi.resources_poll_20x240	145.66 ms	143.89 ms	-1.2%	1.4%	15%
webui.filter_keys_500	66.50 ms	67.30 ms	+1.2%	0.9%	25%
job.report_noop_100k	53.64 ms	53.02 ms	-1.2%	0.9%	15%
webui.rows_layout_200	60.10 ms	60.80 ms	+1.2%	5.2%	25%
pools.admit_cycle_1k	139.85 ms	141.43 ms	+1.1%	2.9%	25%
webui.dag_tasks_2k	55.40 ms	54.80 ms	-1.1%	1.5%	25%
cronexpr.next_dst_2k	41.17 ms	41.61 ms	+1.1%	1.9%	24.29% (declared 15%)
webui.rows_diff_500	58.70 ms	59.30 ms	+1.0%	2.8%	25%
webapi.pools_poll_4x130	110.66 ms	109.55 ms	-1.0%	0.5%	25%
cronexpr.parse_complex	24.94 ms	24.69 ms	-1.0%	1.4%	40.1% (declared 15%)
resources.monitor_stop_100	292.43 ms	295.33 ms	+1.0%	1.3%	25%
startup.mcp_bridge	104.44 ms	103.52 ms	-1.0%	0.7%	25%
dag.list_runs_warm	12.98 ms	12.85 ms	-1.0%	0.9%	25%
tui.log_restyle_5k	31.41 ms	31.12 ms	-0.9%	1.2%	31.83% (declared 15%)
state.derive_max_cold	21.71 ms	21.51 ms	-0.9%	2.3%	46.06% (declared 15%)
mcp.query_metrics_500	129.01 ms	130.17 ms	+0.9%	2.6%	25%
schedule.next_fires_2k	25.32 ms	25.10 ms	-0.9%	1.1%	39.49% (declared 15%)
fingerprint.job_set_id_10k	296.97 ms	299.60 ms	+0.9%	1.6%	15%
dag.list_dags_warm_x300	62.47 ms	61.94 ms	-0.8%	0.9%	25%
resources.usage_from_dict_500	99.57 ms	100.40 ms	+0.8%	0.7%	15%
tui.drawer_paint_5k	104.84 ms	103.98 ms	-0.8%	1.6%	15%
job.stream_capture_120k	36.76 ms	37.06 ms	+0.8%	1.1%	27.2% (declared 15%)
push.seal_500	91.67 ms	92.40 ms	+0.8%	0.4%	15%
state.fanout_gather_100	10.14 ms	10.07 ms	-0.8%	1.3%	98.59% (declared 25%)
state.append_1k	127.43 ms	126.47 ms	-0.8%	1.9%	39.24% (declared 15%)
mem.rss_daemon_import	38.32 MB	38.61 MB	+0.7%	0.0%	25%
loop.idle_wake_rate	22.29 ms	22.13 ms	-0.7%	1.6%	25%
tui.poll_absorb_5k	82.43 ms	83.02 ms	+0.7%	1.3%	15%
mcp.list_jobs_500	142.77 ms	143.78 ms	+0.7%	1.0%	15%
loop.sla_pass_5k_x10	145.31 ms	146.33 ms	+0.7%	3.2%	15%
mcp.tools_list_bytes	19.95 KB	20.09 KB	+0.7%	0.0%	15%
schedule.cold_build_100k	488.29 ms	491.30 ms	+0.6%	1.2%	15%
dag.mapped_drain_256	19.59 ms	19.47 ms	-0.6%	1.7%	51.05% (declared 25%)
dag.full_sweep_50x200	114.55 ms	115.19 ms	+0.6%	7.1%	25%
dag.recover_plan_from_1k	106.17 ms	105.58 ms	-0.6%	1.2%	15%
state.depends_on_past_gate	3.15 ms	3.13 ms	-0.6%	1.8%	63.45% (declared 25%)
json.loads_wide_int_2k	230.17 ms	228.91 ms	-0.5%	1.3%	15%
redact.clean_20k	20.30 ms	20.19 ms	-0.5%	1.6%	49.27% (declared 15%)
webapi.activity_build_500	181.73 ms	182.71 ms	+0.5%	2.3%	15%
state.derive_max_warm	141.71 ms	142.47 ms	+0.5%	0.6%	15%
cronexpr.next_complex	16.16 ms	16.24 ms	+0.5%	0.7%	61.9% (declared 15%)
state.artifact_get_newest	20.11 ms	20.22 ms	+0.5%	0.9%	25%
cluster.job_owner_2k	58.82 ms	59.12 ms	+0.5%	0.8%	17% (declared 15%)
tui.dag_graph_paint_2k	230.74 ms	229.64 ms	-0.5%	1.3%	15%
webui.render_rows_500	16.98 ms	17.06 ms	+0.5%	0.7%	25%
mem.rss_daemon_10k	143.11 MB	143.78 MB	+0.5%	0.1%	25%
webui.radar_walk_500x150	65.60 ms	65.90 ms	+0.5%	2.5%	25%
tui.week_rows_500	138.17 ms	137.55 ms	-0.5%	0.7%	15%
tui.tail_ingest_30k	174.59 ms	173.81 ms	-0.4%	0.8%	15%
webapi.sse_tail_pump_4x20k	118.68 ms	118.15 ms	-0.4%	0.3%	15%
startup.import_tui	101.90 ms	101.46 ms	-0.4%	0.1%	25%
dag.fanin_barrier_1k	191.57 ms	190.72 ms	-0.4%	1.2%	25%
startup.version	53.80 ms	53.58 ms	-0.4%	0.1%	25%
statsd.emit_2k	13.10 ms	13.05 ms	-0.4%	1.1%	76.31% (declared 25%)
dag.gc_excess_60	84.44 ms	84.09 ms	-0.4%	1.9%	25%
tui.fleet_paint_15x400	150.22 ms	150.82 ms	+0.4%	0.8%	15%
state.list_records_warm	113.34 ms	112.89 ms	-0.4%	0.9%	15%
webui.palette_keys_500	51.20 ms	51.40 ms	+0.4%	0.3%	25%
cluster.fleet_view_15x400	93.06 ms	92.70 ms	-0.4%	1.2%	25%
webui.ledger_analyze_500x600	78.60 ms	78.30 ms	-0.4%	1.3%	25%
dag.adopt_scan_500	100.67 ms	100.30 ms	-0.4%	0.7%	25%
webui.swim_400x49	81.20 ms	80.90 ms	-0.4%	1.1%	25%
webui.timeline_500	54.50 ms	54.70 ms	+0.4%	1.3%	25%
schedule.reseed_100k	204.99 ms	204.25 ms	-0.4%	2.4%	15%
startup.job_set_id_100	187.90 ms	188.44 ms	+0.4%	0.4%	25%
schedule.suggest_slot_5k_x20	77.04 ms	77.31 ms	+0.3%	1.6%	15%
cronexpr.next_simple	39.28 ms	39.15 ms	-0.3%	1.4%	25.46% (declared 15%)
startup.python_baseline	17.35 ms	17.29 ms	-0.3%	0.4%	Not enforced
config.reload_warm_dir_1k	249.05 ms	249.89 ms	+0.3%	1.2%	25%
mem.rss_daemon_web_100	53.17 MB	53.35 MB	+0.3%	0.2%	25%
prometheus.render_500	17.29 ms	17.35 ms	+0.3%	1.9%	57.84% (declared 15%)
json.roundtrip_3k	573.86 ms	575.78 ms	+0.3%	0.7%	15%
mcp.handle_200	160.87 ms	160.34 ms	-0.3%	1.5%	15%
jobapi.sem_denied_64x10	83.66 ms	83.40 ms	-0.3%	1.4%	25%
config.parse_yaml_rich_20	272.02 ms	271.18 ms	-0.3%	0.3%	15%
redact.adversarial_10k	36.41 ms	36.52 ms	+0.3%	2.0%	27.46% (declared 15%)
state.artifact_list_churn_x500	63.81 ms	64.01 ms	+0.3%	0.8%	25%
config.reload_warm_50	43.45 ms	43.59 ms	+0.3%	1.2%	25%
startup.validate_config_100	182.22 ms	182.65 ms	+0.3%	0.5%	25%
push.fanout_8x100	154.12 ms	154.57 ms	+0.3%	0.9%	15%
ical.render_500x7d	50.41 ms	50.56 ms	+0.3%	1.5%	19.84% (declared 15%)
dag.advance_quiescent_chain	70.75 ms	70.55 ms	-0.3%	1.1%	25%
state.list_documents_600	56.01 ms	56.16 ms	+0.3%	1.6%	21.43% (declared 15%)
schedule.lint_2500_zoned	77.21 ms	77.00 ms	-0.3%	0.4%	15%
startup.daemon_first_pass_2k	1.580 s	1.575 s	-0.3%	0.4%	25%
redact.secrets_5k	20.45 ms	20.50 ms	+0.3%	2.2%	48.9% (declared 15%)
push.seal_xwing_500	137.43 ms	137.78 ms	+0.3%	0.3%	15%
loop.stall_metrics_2000	28.01 ms	27.94 ms	-0.3%	2.0%	Not enforced
tui.table_paint_5k	192.25 ms	192.74 ms	+0.3%	0.4%	15%
webapi.dags_poll_1k	135.35 ms	135.01 ms	-0.2%	0.9%	15%
tui.view_sort_5k	113.89 ms	113.61 ms	-0.2%	0.9%	15%
mem.rss_version	14.49 MB	14.52 MB	+0.2%	0.5%	25%
cluster.derive_cold_50	159.32 ms	159.70 ms	+0.2%	1.0%	15%
webapi.auth_scope_20k	87.22 ms	87.42 ms	+0.2%	0.4%	15%
dag.finish_fanin_20x1k	68.46 ms	68.61 ms	+0.2%	0.6%	25%
config.reload_gc_100k	2.697 s	2.702 s	+0.2%	1.8%	25%
cronexpr.parse_simple	44.14 ms	44.05 ms	-0.2%	0.8%	22.66% (declared 15%)
config.parse_yaml_3k	1.819 s	1.822 s	+0.2%	0.4%	15%
config.interp_2k	66.15 ms	66.28 ms	+0.2%	0.9%	15%
mem.jobconfig_envfile_2k	16.38 MB	16.41 MB	+0.2%	0.0%	15%
webui.sort_500	54.90 ms	55.00 ms	+0.2%	3.6%	25%
webui.wallboard_poll_500	55.40 ms	55.30 ms	-0.2%	1.1%	25%
webui.week_walk_500x20	57.60 ms	57.50 ms	-0.2%	0.4%	25%
tui.log_search_20k	15.40 ms	15.42 ms	+0.2%	1.3%	64.95% (declared 15%)
state.lease_renew_200	50.27 ms	50.19 ms	-0.2%	1.6%	99.45% (declared 25%)
cronexpr.occurrences_1k	13.89 ms	13.87 ms	-0.2%	2.1%	71.99% (declared 15%)
dag.run_chain_100	180.56 ms	180.27 ms	-0.2%	0.4%	25%
webui.render_term_5k	103.85 ms	103.70 ms	-0.1%	0.8%	25%
config.parse_dir_cold_200	281.91 ms	281.51 ms	-0.1%	0.8%	25%
cluster.peer_serve_400	157.72 ms	157.93 ms	+0.1%	0.8%	15%
webui.render_fleet_15x400	23.58 ms	23.61 ms	+0.1%	1.1%	25%
webapi.summary_status_500	169.94 ms	170.15 ms	+0.1%	0.4%	15%
loop.stall_jobs_500	20.99 ms	21.02 ms	+0.1%	2.3%	Not enforced
cluster.parse_summaries_6k	84.81 ms	84.91 ms	+0.1%	0.9%	15%
dag.advance_waiting_mapped_1k	122.25 ms	122.11 ms	-0.1%	0.3%	25%
webapi.dag_run_poll_1k	88.86 ms	88.77 ms	-0.1%	0.7%	25%
dag.advance_quiescent_1k	71.85 ms	71.93 ms	+0.1%	0.7%	25%
pools.enqueue_burst_200	141.72 ms	141.57 ms	-0.1%	0.3%	25%
config.parse_yaml_300	184.27 ms	184.08 ms	-0.1%	0.6%	15%
webui.logo_recovery	102.50 ms	102.60 ms	+0.1%	0.3%	25%
webapi.gzip_body_500	63.88 ms	63.83 ms	-0.1%	1.4%	15%
tui.mark_idle_300	280.07 ms	280.29 ms	+0.1%	1.3%	15%
state.list_records_2k	24.69 ms	24.71 ms	+0.1%	0.5%	40.51% (declared 15%)
state.kv_roundtrip_200	55.82 ms	55.86 ms	+0.1%	0.8%	17.91% (declared 15%)
prometheus.first_scrape_after_reload_500	188.72 ms	188.83 ms	+0.1%	1.2%	15%
startup.jobcli_state_get	106.51 ms	106.51 ms	+0.1%	0.7%	25%
config.reload_warm_include_50	94.07 ms	94.02 ms	-0.0%	0.5%	25%
state.artifact_get_oldest_2k	77.10 ms	77.07 ms	-0.0%	0.7%	25%
jobapi.read_path_10k	124.42 ms	124.48 ms	+0.0%	0.5%	15%
pair.qr_encode_14	257.11 ms	257.22 ms	+0.0%	0.5%	15%
cluster.poll_round_15x400	161.66 ms	161.60 ms	-0.0%	0.5%	15%
tui.palette_type_5k	173.65 ms	173.70 ms	+0.0%	1.7%	15%
mem.parse_peak_rich_20	4.21 MB	4.21 MB	+0.0%	0.0%	15%
state.artifact_put_distinct_2k	55.95 ms	55.96 ms	+0.0%	1.7%	25%
mem.run_history_series_5	9.57 MB	9.58 MB	+0.0%	0.0%	15%
json.roundtrip_orjson_3k	205.14 ms	205.18 ms	+0.0%	1.4%	15%
webapi.jobs_payload_500	52.17 ms	52.16 ms	-0.0%	0.5%	19.17% (declared 15%)
mcp.fleet_15x400	165.12 ms	165.10 ms	-0.0%	1.1%	15%
state.mutate_document_1k	145.85 ms	145.84 ms	-0.0%	0.9%	34.28% (declared 25%)
dag.run_wide_500	108.04 ms	108.05 ms	+0.0%	0.6%	25%
mem.gc_manifest_scan_128x2k	14.32 MB	14.32 MB	-0.0%	0.0%	15%
state.append_pruned_1k	72.74 ms	72.74 ms	+0.0%	1.6%	25%
dag.run_doc_bytes_1k	228.53 KB	228.53 KB	+0.0%	0.0%	5%
dag.mapped_doc_bytes_1k	725.81 KB	725.81 KB	+0.0%	0.0%	5%
tui.frame_bytes_5k	60.22 KB	60.22 KB	+0.0%	0.0%	15%
webui.log_count_5k	90.0 us	90.0 us	+0.0%	7.1%	2222% (declared 25%)
webui.append_line_5k	14.0 us	14.0 us	+0.0%	3.7%	25%
webapi.jobs_bytes_500	181.42 KB	181.42 KB	+0.0%	0.0%	15%
webapi.jobs_gzip_500	5.19 KB	5.19 KB	+0.0%	0.0%	15%
webapi.activity_bytes_500	2569.83 KB	2569.83 KB	+0.0%	0.0%	15%
webapi.activity_capped_bytes_500	411.18 KB	411.18 KB	+0.0%	0.0%	15%
webapi.fleet_bytes_15x400	963.62 KB	963.62 KB	+0.0%	0.0%	15%
cluster.peer_bytes_512	83.55 KB	83.55 KB	+0.0%	0.0%	15%
prometheus.counter_snapshot_bytes_2k	620.59 KB	620.59 KB	+0.0%	0.0%	15%
prometheus.exposition_bytes_500	821.17 KB	821.17 KB	+0.0%	0.0%	15%
mcp.list_jobs_bytes_500	155.54 KB	155.54 KB	+0.0%	0.0%	15%
mcp.fleet_bytes_15x400	2052.76 KB	2052.76 KB	+0.0%	0.0%	15%
mem.crontab_10k	2.80 MB	2.80 MB	+0.0%	0.0%	17.85% (declared 15%)
mem.retired_output_1k	0.12 MB	0.12 MB	+0.0%	0.0%	15%
mem.cluster_view_50x400	11.64 MB	11.64 MB	+0.0%	0.0%	15%
mem.tui_state_5k	2.24 MB	2.24 MB	+0.0%	0.0%	15%

Full Changelog: 1.2.67...1.2.68