Add resource budgets to manifest template evaluation

[leynos]

Summary

Manifest template evaluation has no explicit application-level execution, output, or expansion budget.

The original Codex finding described an older implementation that rendered the entire YAML document twice. main no longer has that exact shape: Netsuke now parses YAML first and renders recognized fields individually. The underlying availability defect remains at ac10b783f33661a0af27bd8f5d19065f095f98db:

• each rendered field calls MiniJinja Environment::render_str() and materializes an owned String without a Netsuke output limit;
• manifest macro invocation uses the same unbounded render path;
• foreach expressions call try_iter() and collect the complete sequence;
• expansion clones a target/action map once per item with no cardinality or aggregate expansion limit;
• no shared per-manifest fuel/iteration budget constrains repeated expressions, loops, macro calls, when evaluation, or rendering across fields.

Any MiniJinja internal defaults are library implementation details; Netsuke currently defines no product-level resource contract or localized limit failure.

Examples

A compact recipe field can request enormous output before any build command runs:

targets:
  - name: generated
    command: "{% for _ in range(1000000000) %}x{% endfor %}"

A compact foreach expression can request an enormous number of expanded entries, each cloning the source mapping and later flowing through rendering and IR generation.

These paths affect build/generate operations before recipe execution. foreach, when, and discovery-field rendering can also affect manifest-query operations intended only to inspect metadata.

Impact

A small attacker-controlled Netsukefile can consume unbounded CPU and memory on a developer machine or CI worker during parsing, expansion, and rendering. External cgroup/job limits may eventually kill the process, but Netsuke should fail deterministically and diagnostically before exhausting its host.

Required change

Introduce a coherent resource budget covering the whole manifest evaluation pipeline, not a collection of unrelated local caps.

The design should include:

• an instruction/fuel or equivalent evaluation-work budget shared across template rendering, expressions, when, and macro invocation;
• a maximum rendered byte count per value and across the complete manifest;
• a maximum foreach cardinality and maximum aggregate expanded target/action count;
• early termination while consuming iterators rather than collecting an unbounded sequence first;
• checked accounting for source size, macro imports, cloned expansion data, and rendered output;
• localized errors that identify the exhausted budget and evaluation stage without echoing secret-bearing template/context values;
• operator-controlled configuration whose limits cannot be raised by an untrusted project configuration layer.

Use MiniJinja's supported fuel/output facilities where they provide the required guarantees; retain Netsuke-owned accounting for expansion and aggregate limits that the template engine cannot observe.

Acceptance criteria

• A compact large-loop template fails with a deterministic resource-limit diagnostic before allocating output proportional to the requested loop size.
• foreach iteration stops at a configured bound without first collecting or cloning every item.
• A per-manifest aggregate budget prevents many individually small fields from bypassing the per-field limit.
• Macro calls, when expressions, normal field rendering, and query-mode evaluation consume the appropriate shared budget.
• Limits have safe defaults and may be narrowed by more-trusted operator configuration; project configuration cannot widen them implicitly.
• Ordinary existing manifests below the limits retain byte-for-byte rendered behaviour.
• Tests exercise exact-boundary success, one-unit-over failure, many-field aggregate exhaustion, large foreach, nested macro/loop work, and manifest-query mode using deliberately small test budgets rather than huge allocations.
• Limit diagnostics and telemetry are bounded and do not include raw template text, context values, or rendered secrets.
• The users guide and design document specify defaults, accounting units, configuration precedence, and failure behaviour.
• make check-fmt, make lint, make test, make markdownlint, and make nixie pass.

References

• Codex finding: https://chatgpt.com/codex/cloud/security/findings/00b995cfb7fc81918c1e86e2d314b3a0?repo=https%3A%2F%2Fgithub.com%2Fleynos%2Fnetsuke&sev=low
• Current manifest loading and environment construction: https://github.com/leynos/netsuke/blob/ac10b783f33661a0af27bd8f5d19065f095f98db/src/manifest/mod.rs
• Current per-field rendering: https://github.com/leynos/netsuke/blob/ac10b783f33661a0af27bd8f5d19065f095f98db/src/manifest/render.rs
• Macro/render adapter: https://github.com/leynos/netsuke/blob/ac10b783f33661a0af27bd8f5d19065f095f98db/src/manifest/jinja_macros/mod.rs
• foreach collection and expansion: https://github.com/leynos/netsuke/blob/ac10b783f33661a0af27bd8f5d19065f095f98db/src/manifest/expand.rs
• Related policy-provenance issue: Prevent project configuration from widening trusted fetch policy #644

[leynos]

@CodeRabbit plan

[coderabbitai]

Coding Plan

Summary

• Build one ManifestBudget per parse and enforce it at the shared choke points: render_template for bytes, source size, and fuel, and expand.rs for foreach cardinality and aggregate expansion with lazy, early-terminating iteration.
• Use MiniJinja's fuel feature and a byte-capped writer for engine-observable work, and retain Netsuke-owned aggregate counters for bytes, cardinality, expansion count, and source size that the engine cannot observe.
• Make limits operator-configurable through CliConfig with a layer-aware clamp so a project configuration can only narrow, never widen, the effective budget; thread the resolved limits through ManifestParse so build and manifest-query modes share them.
• Surface exhaustion as a localized, redacted diagnostic and bounded telemetry, and document defaults, accounting units, precedence, and failure behaviour in a new ADR, the design document, and the users guide.

Design Choices

Design Choice 1: How to enforce "operator may narrow, project configuration cannot widen" for numeric budget limits

Options Considered:

1. Layer-aware clamp: let budget fields participate in the user-scope, environment, and CLI layers normally, then clamp any project-scope contribution so it can only lower, never raise, the value established by the more-trusted layers (defaults, user-scope file, environment, CLI). This reuses Netsuke's existing pre-merge project-scope seam in src/cli/discovery_layers.rs.
2. Exclude project scope entirely: forbid the project .netsuke.toml from setting budget fields at all; only defaults, user-scope files, environment, and CLI may set them.
3. Clamp-to-constant only (mirroring MAX_JOBS): reject values above a fixed compile-time ceiling after merge.

Chosen Option: 1

Rationale: The current merge is last-writer-wins with project scope outranking user scope, which is the opposite trust direction. No layer-aware clamp primitive exists. Option 1 precisely matches "may be narrowed by operator configuration; project configuration cannot widen them", follows the codebase's "restrictive rule wins" idiom, and still lets a project narrow its own budget harmlessly. Option 3 is rejected because it lets a project widen an operator's narrower value up to the constant.

Design Choice 2: How to realise a shared, cumulative fuel/evaluation-work budget given MiniJinja's per-render fuel semantics

Options Considered:

1. Per-evaluation fuel cap plus Netsuke-owned aggregate byte/expansion/source counters that truly span the whole manifest, and, on the render path, read consumed fuel from the final render State (via a render-to-writer path that yields State and State::fuel_levels()) to deduct from an aggregate fuel budget.
2. Per-evaluation fuel cap only, with no cumulative fuel accounting, relying entirely on Netsuke-owned byte/expansion counters for the "shared" guarantee.

Chosen Option: 1

Rationale: MiniJinja set_fuel is per-render and resets each render_str/Template::render; there is no public API to charge fuel from a custom function, and render_str returns only a String. Option 1 gives a genuine shared budget for bytes, expansion, and source size, bounds each individual evaluation with fuel, and adds best-effort cumulative fuel accounting where the render path exposes State. If the specific State-returning render signature differs in MiniJinja 2.24, fall back to Option 2 for fuel while keeping the Netsuke-owned aggregate counters.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Establish the budget core and bounded rendering

Enable MiniJinja fuel, define a single per-manifest budget type, and route all template rendering through a byte-capped, fuel-aware path with a localized, redacted failure. This phase creates the shared foundation that later phases consume.

Task 1: Enable MiniJinja fuel and output-limiting facilities

Turn on the engine features required for instruction and output bounding.

• Update Cargo.toml to add the fuel feature to the minijinja dependency alongside the existing loader feature.
• Confirm Cargo.lock resolves without other feature changes.

Task 2: Define the manifest budget module

Create one cohesive budget type rather than scattered local caps.

• Add a new module (for example src/manifest/budget.rs or src/manifest/budget/mod.rs) defining ManifestBudgetLimits (the configured ceilings) and ManifestBudget (the runtime accounting state).
• Model ManifestBudgetLimits on the two-field idiom of RetentionPolicy and CommandConfig, with fields for: per-evaluation fuel, per-value rendered bytes, aggregate rendered bytes, aggregate source bytes (template plus macro-import prefix), maximum foreach cardinality, and maximum aggregate expanded target/action count. Provide safe default constants and reject zero values on construction (mirror StdlibConfig's ensure! pattern).
• Give ManifestBudget remaining-counter fields for each aggregate limit and methods to charge and check each budget, returning a typed budget-exhaustion result that names the exhausted budget and the evaluation stage. Use interior mutability (for example Cell/RefCell) so a shared &ManifestBudget handle can be threaded without wide &mut signature churn.
• Set the default ceilings high enough that ordinary manifests render byte-for-byte unchanged.

Task 3: Add localized budget-exhaustion messages

Provide a redacted, parameterised diagnostic that names the exhausted budget and stage.

• Add a new key to define_keys! in src/localization/keys.rs, for example MANIFEST_BUDGET_EXCEEDED => "manifest.budget.exceeded", in the MANIFEST_* block.
• Add the matching Fluent message with $stage and $limit variables to locales/en-US/messages.ftl and to all other 33 locale catalogues, following the stdlib.command.output_limit_exceeded / runner.io.dyndep.too_large precedent. Ensure identical interpolation variables in every catalogue to satisfy build_l10n_audit.
• Construct the message at call sites with localization::message(keys::MANIFEST_BUDGET_EXCEEDED).with_arg("stage", ...).with_arg("limit", ...); pass only a fixed stage label and the numeric limit, never template text or context values.

Task 4: Route rendering through a fuel-aware, byte-capped writer

Replace the unbounded render_str choke point with a bounded path.

• Add a capped std::io::Write adapter in the budget module (or beside render_template) that carries the per-value byte cap and the shared aggregate remaining-bytes counter, stops accepting bytes when either is exceeded, and signals a budget failure (surfaced by MiniJinja as ErrorKind::WriteFailure).
• Update render_template in src/manifest/jinja_macros/mod.rs to accept the ManifestBudget, account for the concatenated source size (import block plus template) against the source-bytes budget before rendering, set per-evaluation fuel via env.set_fuel(Some(...)), render through the writer path, and, where the render path yields the final State, read State::fuel_levels() to deduct consumed fuel from an aggregate fuel budget.
• Map ErrorKind::OutOfFuel and the writer's byte-limit failure to the localized MANIFEST_BUDGET_EXCEEDED message, following the Error::new(ErrorKind::…, localization::message(...)) pattern already used in env_reader.rs and jinja_macros/invocation.rs.
• Apply the same per-evaluation fuel cap to the expression and when evaluation paths (env.compile_expression(...).eval(...) and evaluate_when_expression).

Task 5: Extend telemetry for budget exhaustion

Record exhaustion with bounded, redacted labels per ADR-009.

• Add a budget-exhaustion instrument to src/manifest/jinja_macros/telemetry.rs, for example a netsuke_manifest_budget_exhausted_total counter with a fixed stage label vocabulary (render, foreach, when, macro, expansion_aggregate, byte_aggregate, source, fuel) and a budget label.
• Emit only closed-vocabulary fields; never emit template text, macro names, context values, or rendered output. Register once via the existing Once pattern.

🤖 Prompt for AI agents

Implement the shared budget foundation that every later phase relies on. Goal:
enable MiniJinja's fuel feature, define a single per-manifest
`ManifestBudget`/`ManifestBudgetLimits` type, force all rendering through a
bounded path, and wire up localized, redacted diagnostics and telemetry for
exhaustion.

- In `Cargo.toml`, add the `fuel` feature to the `minijinja` dependency next to
the existing `loader` feature. Confirm `Cargo.lock` resolves cleanly with no
other feature drift.

- Create a new budget module (`src/manifest/budget.rs` or
`src/manifest/budget/mod.rs`). Define `ManifestBudgetLimits` for configured
ceilings and `ManifestBudget` for runtime accounting state, following the
two-field idiom used by `RetentionPolicy` and `CommandConfig`. Include limits
for: per-evaluation fuel, per-value rendered bytes, aggregate rendered bytes,
aggregate source bytes (template plus macro-import prefix), maximum `foreach`
cardinality, and maximum aggregate expanded target/action count. Supply safe
default constants sized so ordinary manifests render byte-for-byte unchanged.
Reject zero values on construction, mirroring `StdlibConfig`'s `ensure!`
pattern. Give `ManifestBudget` remaining-counter fields per aggregate limit plus
charge/check methods that return a typed exhaustion result naming the exhausted
budget and evaluation stage. Use interior mutability (`Cell`/`RefCell`) so a
shared `&ManifestBudget` can be threaded without widespread `&mut` signature
changes.

- Add `MANIFEST_BUDGET_EXCEEDED => "manifest.budget.exceeded"` to `define_keys!`
in `src/localization/keys.rs`, in the `MANIFEST_*` block. Add the matching
Fluent message with `$stage` and `$limit` variables to
`locales/en-US/messages.ftl` and mirror it, with identical interpolation
variables, across all 33 remaining locale catalogues so `build_l10n_audit`
passes. At call sites, build the message with
`localization::message(keys::MANIFEST_BUDGET_EXCEEDED).with_arg("stage",
...).with_arg("limit", ...)`, passing only a fixed stage label and a numeric
limit — never template text or context values.

- Add a capped `std::io::Write` adapter (in the budget module or beside
`render_template`) that enforces the per-value byte cap and the shared aggregate
remaining-bytes counter, refusing further bytes once either limit is hit and
signalling failure (surfaced by MiniJinja as `ErrorKind::WriteFailure`). Update
`render_template` in `src/manifest/jinja_macros/mod.rs` to accept the
`ManifestBudget`, check the concatenated source size (import block plus
template) against the source-bytes budget before rendering, set per-evaluation
fuel with `env.set_fuel(Some(...))`, render through the capped writer, and,
where the render path exposes the final `State`, read `State::fuel_levels()` to
deduct from the aggregate fuel budget. Map `ErrorKind::OutOfFuel` and the
writer's byte-limit failure to the localized `MANIFEST_BUDGET_EXCEEDED` message,
following the existing `Error::new(ErrorKind::…, localization::message(...))`
pattern from `env_reader.rs` and `jinja_macros/invocation.rs`. Apply the same
per-evaluation fuel cap to expression and `when` evaluation
(`env.compile_expression(...).eval(...)` and `evaluate_when_expression`). Never
call the unbounded `render_str` path again.

- Extend `src/manifest/jinja_macros/telemetry.rs` with a budget-exhaustion
instrument, for example `netsuke_manifest_budget_exhausted_total`, using a fixed
`stage` label vocabulary (`render`, `foreach`, `when`, `macro`,
`expansion_aggregate`, `byte_aggregate`, `source`, `fuel`) plus a `budget`
label. Register once via the existing `Once` pattern. Emit only
closed-vocabulary fields — never template text, macro names, context values, or
rendered output.

Phase 2: Bound expansion with early termination

Enforce foreach cardinality and aggregate expanded-entry limits during expansion, and stop consuming iterators early instead of collecting whole sequences.

Task 1: Thread the budget through the expansion traversal

Reuse the existing accumulator seam.

• Extend ExpansionContext<'a> in src/manifest/expand.rs to hold a shared &ManifestBudget handle, alongside env and section, so parse_foreach_values, expand_target, when_allows, eval_when, and eval_expression can all consult and charge the same budget.
• Update expand_foreach's signature to accept the budget and pass it into expand_section and expand_target.

Task 2: Enforce cardinality and aggregate limits with early termination

Stop before allocating unbounded work.

• Change parse_foreach_values to consume seq.try_iter() lazily and stop once the per-foreach cardinality budget is exceeded, returning the localized budget error rather than collecting the complete sequence first.
• In expand_target, charge the aggregate expanded target/action budget once per produced entry before cloning the source map, and fail deterministically when the aggregate budget is exhausted so many small foreach blocks cannot bypass the per-block limit.
• Charge an estimate of cloned-map cost against the aggregate accounting where practical, keeping the primary levers cardinality and aggregate count.
• Emit the budget telemetry event (stage foreach or expansion_aggregate) on exhaustion.

🤖 Prompt for AI agents

Enforce the manifest budget during the expansion traversal so oversized
`foreach` blocks or aggregate expansion cannot exhaust CPU or memory before the
render phase. Goal: thread the shared `ManifestBudget` into expansion and stop
iteration early rather than materialising whole sequences.

- Extend `ExpansionContext<'a>` in `src/manifest/expand.rs` to carry a shared
`&ManifestBudget` handle alongside the existing `env` and `section` fields, so
`parse_foreach_values`, `expand_target`, `when_allows`, `eval_when`, and
`eval_expression` all read from and charge the same budget instance. Update
`expand_foreach`'s signature to accept the budget and pass it through into
`expand_section` and `expand_target`.

- Change `parse_foreach_values` to consume `seq.try_iter()` lazily, stopping as
soon as the per-`foreach` cardinality budget is exceeded, and return the
localized budget error instead of collecting the full sequence first.

- In `expand_target`, charge the aggregate expanded target/action budget once
per produced entry before cloning the source map, and fail deterministically
once the aggregate budget is exhausted, so many small `foreach` blocks cannot
together bypass the per-block cardinality limit. Where practical, charge an
estimate of cloned-map cost against the aggregate accounting, but keep
cardinality and aggregate count as the primary enforcement levers.

- On exhaustion, emit the budget telemetry event added in Phase 1 with the
appropriate stage label (`foreach` or `expansion_aggregate`).

Phase 3: Configuration, precedence, and end-to-end threading

Make the budget operator-configurable with a project layer that cannot widen it, and thread the resolved limits into both build and manifest-query paths.

Task 1: Add budget fields and safe defaults to configuration

Expose the limits through the normal configuration surface.

• Add budget fields (one per configurable limit) to CliConfig in src/cli/config.rs, with default = ... attributes providing the safe defaults defined in Task 1.2.
• Add a Cli accessor manifest_budget_limits() (in src/cli_policy.rs, analogous to network_policy()) that builds ManifestBudgetLimits from the merged configuration.

Task 2: Enforce the non-widenable, layer-aware clamp

Prevent the project layer from raising operator-set limits.

• Extend the project-scope handling in src/cli/discovery_layers.rs so that, for the budget fields, any project-scope contribution is clamped to be no larger than the value established by the more-trusted layers (defaults, user-scope file, environment, CLI). A project value may lower the effective limit but never raise it.
• Emit a bounded trace when a project value is clamped; do not echo file contents.
• Validate final limits post-merge (extend the existing PostMergeHook/validate_* pattern) to reject non-positive or out-of-range values, mirroring validate_jobs.

Task 3: Thread resolved limits into manifest loading for both modes

Cover build and manifest-query paths uniformly.

• Add a ManifestBudgetLimits field to ManifestParse in src/manifest/mod.rs; construct the runtime ManifestBudget in from_str_named immediately after register_manifest_macros and before expand_foreach, and pass it into expand_foreach and the render calls so both RenderMode::Full and RenderMode::ManifestQuery share it. Do not place the budget solely in StdlibConfig, because manifest-query mode never constructs a StdlibConfig.
• Add a limits parameter to the public manifest entry points (from_path_with_policy, from_path_with_policy_and_env, from_path_for_manifest_query) and to query::from_path_with_registration, applying the limits when building ManifestParse.
• Thread the limits through the runner layers: runner/mod.rs::load_manifest_with_stage_reporting, runner/generation.rs::load_manifest_for_build and load_manifest, and the call sites in runner/graph_generation.rs, runner/graph.rs, and runner/help_query.rs, computing the limits from cli.manifest_budget_limits() alongside the existing NetworkPolicy.

🤖 Prompt for AI agents

Make the manifest budget operator-configurable and ensure the project
configuration layer can only narrow it, then thread the resolved limits through
both build and manifest-query manifest-loading paths. Goal: full end-to-end
wiring from `CliConfig` to `ManifestParse` with a non-widenable project layer.

- Add one field per configurable limit to `CliConfig` in `src/cli/config.rs`,
each with a `default = ...` attribute matching the safe defaults defined in
Phase 1's `ManifestBudgetLimits`. Add a `Cli` accessor
`manifest_budget_limits()` in `src/cli_policy.rs`, following the pattern of the
existing `network_policy()` accessor, that builds a `ManifestBudgetLimits` from
the merged configuration.

- In `src/cli/discovery_layers.rs`, extend the project-scope handling so that,
for every budget field, a project-scope contribution is clamped to no more than
the value already established by the more-trusted layers (defaults, user-scope
file, environment, CLI). A project value may lower the effective limit but must
never raise it. Emit a bounded trace when a project value gets clamped; do not
echo file contents in the trace. Extend the existing
`PostMergeHook`/`validate_*` pattern (mirroring `validate_jobs`) to validate the
final merged limits and reject non-positive or out-of-range values.

- Add a `ManifestBudgetLimits` field to `ManifestParse` in
`src/manifest/mod.rs`. In `from_str_named`, construct the runtime
`ManifestBudget` immediately after `register_manifest_macros` and before
`expand_foreach`, then pass it into `expand_foreach` and the render calls so
both `RenderMode::Full` and `RenderMode::ManifestQuery` share the same budget
instance. Do not place the budget solely inside `StdlibConfig`, since
manifest-query mode never constructs one.

- Add a limits parameter to the public manifest entry points
`from_path_with_policy`, `from_path_with_policy_and_env`, and
`from_path_for_manifest_query`, and to `query::from_path_with_registration`,
applying the limits when building `ManifestParse`.

- Thread the resolved limits through the runner layers:
`runner/mod.rs::load_manifest_with_stage_reporting`,
`runner/generation.rs::load_manifest_for_build` and `load_manifest`, and the
call sites in `runner/graph_generation.rs`, `runner/graph.rs`, and
`runner/help_query.rs`. Compute the limits from `cli.manifest_budget_limits()`
alongside the existing `NetworkPolicy` at each call site.

Phase 4: Tests

Prove the budget with deliberately small test limits, exact-boundary cases, aggregate exhaustion, and bounded, redacted telemetry, following the established three-layer test convention.

Task 1: Unit and property tests for expansion and rendering budgets

Cover boundaries and invariants.

• Add exact-boundary unit tests (LIMIT, LIMIT-1, LIMIT+1) for foreach cardinality and aggregate expansion, modelled on src/manifest/glob/tests/diagnostics.rs::skipped_symlink_diagnostics_retain_a_bounded_sample; drive fixture sizes off the limit constants, not magic numbers.
• Extend src/manifest/expand_test_cases/property_cases.rs and foreach_property_cases.rs with a budget invariant: processed entries never exceed the budget, termination is deterministic, and the report stays internally consistent for any generated size straddling the budget.
• Add unit tests for per-value and aggregate byte limits and for the per-evaluation fuel cap using a compact large-loop template, asserting failure occurs before allocating output proportional to the loop size.

Task 2: Telemetry redaction and shared-budget tests

Confirm bounded labels and cross-mode coverage.

• Add a telemetry test using a local metrics_util::debugging::DebuggingRecorder and scoped tracing capture, modelled on src/manifest/tests/macros_telemetry.rs and macro_invocation_telemetry.rs, asserting the budget counter increments once with only fixed labels and no template text, context values, or rendered secrets.
• Add a redaction proptest over arbitrary manifest content, modelled on macro_telemetry_stays_bounded_for_arbitrary_macros.
• Add tests proving macro calls, when expressions, normal field rendering, and manifest-query mode all consume the shared budget.

Task 3: Integration and BDD coverage with small test budgets

Exercise the feature end to end.

• Extend the manifest-loading test seam so tests can inject small ManifestBudgetLimits (the limits parameter added in Task 3.3 makes this straightforward). Wire a new Given step into tests/bdd/steps/manifest/ that stores a small budget in TestWorld and is consumed by parse_manifest_inner, following the tests/bdd/steps/stdlib/config.rs precedent.
• Add static fixtures under tests/data/ (for example a compact large-loop recipe and an oversized foreach) and scenarios in tests/features/manifest.feature that assert parsing the manifest fails and the error message contains the budget diagnostic text, using the existing localized-error assertion vocabulary.
• Add a regression scenario confirming an ordinary manifest below the limits renders unchanged.

🤖 Prompt for AI agents

Prove the manifest budget works correctly at unit, property, telemetry,
integration, and BDD levels, following the project's established three-layer
test convention. Goal: exact-boundary coverage, redaction guarantees, and
cross-mode shared-budget verification, all driven by deliberately small test
limits.

- Add exact-boundary unit tests (`LIMIT`, `LIMIT-1`, `LIMIT+1`) for `foreach`
cardinality and aggregate expansion, modelled on
`src/manifest/glob/tests/diagnostics.rs::skipped_symlink_diagnostics_retain_a_bounded_sample`.
Drive fixture sizes off the limit constants directly, never magic numbers.
Extend `src/manifest/expand_test_cases/property_cases.rs` and
`foreach_property_cases.rs` with a budget invariant: processed entries never
exceed the budget, termination is deterministic, and the report stays internally
consistent for any generated size that straddles the budget. Add unit tests for
per-value and aggregate byte limits and for the per-evaluation fuel cap using a
compact large-loop template, asserting that failure happens before output
proportional to the loop size gets allocated.

- Add a telemetry test using a local
`metrics_util::debugging::DebuggingRecorder` plus scoped tracing capture,
modelled on `src/manifest/tests/macros_telemetry.rs` and
`macro_invocation_telemetry.rs`. Assert the budget counter increments exactly
once with only fixed labels present, and confirm no template text, context
values, or rendered secrets leak. Add a redaction proptest over arbitrary
manifest content, modelled on
`macro_telemetry_stays_bounded_for_arbitrary_macros`. Add tests proving macro
calls, `when` expressions, normal field rendering, and manifest-query mode all
draw from the same shared budget.

- Extend the manifest-loading test seam so tests can inject small
`ManifestBudgetLimits`, using the limits parameter added in Phase 3. Add a new
`Given` step under `tests/bdd/steps/manifest/` that stores a small budget in
`TestWorld` for consumption by `parse_manifest_inner`, following the
`tests/bdd/steps/stdlib/config.rs` precedent. Add static fixtures under
`tests/data/` (for example a compact large-loop recipe and an oversized
`foreach`) and scenarios in `tests/features/manifest.feature` asserting `parsing
the manifest fails` and `the error message contains` the budget diagnostic text,
reusing the existing localized-error assertion vocabulary. Add a regression
scenario confirming an ordinary manifest below the limits renders byte-for-byte
unchanged.

Phase 5: Documentation

Document defaults, accounting units, precedence, and failure behaviour.

Task 1: Record the architecture decision

Add a new ADR following the house convention.

• Create docs/adr-018-<kebab-case-title>.md using the ADR-016/ADR-017 section skeleton (Status, Date, Context and problem statement, Decision, Rationale, Consequences, Alternatives considered, Implementation references), citing the budget module, expand.rs, render.rs, jinja_macros, and the configuration precedence machinery.
• Add the ADR entry to docs/contents.md under "## Decision records".

Task 2: Update the design document and users guide

Explain the product contract.

• Extend docs/netsuke-design.md to describe the budget accounting units, defaults, and how the budget participates in the configuration precedence ladder, inserting near the "Discovery scopes and layered merging" section and cross-linking the existing fetch/shell limit content.
• Add a "Resource budgets" subsection under "## Configure Netsuke" in docs/users-guide.md, documenting defaults, the operator-narrow / project-cannot-widen rule, configuration keys/flags/environment variables, and the failure diagnostic.
• Run make markdownlint and make nixie against the new and changed docs.

🤖 Prompt for AI agents

Document the new manifest resource budget feature so operators and future
maintainers understand defaults, accounting units, precedence, and failure
behaviour. Goal: a new ADR plus updates to the design document and users guide.

- Create `docs/adr-018-<kebab-case-title>.md` using the section skeleton from
ADR-016/ADR-017: Status, Date, Context and problem statement, Decision,
Rationale, Consequences, Alternatives considered, Implementation references.
Cite the budget module, `expand.rs`, `render.rs`, `jinja_macros`, and the
configuration precedence machinery as implementation references. Add the ADR
entry to `docs/contents.md` under "## Decision records".

- Extend `docs/netsuke-design.md` to describe the budget accounting units, the
defaults, and how the budget fits into the configuration precedence ladder.
Insert this content near the "Discovery scopes and layered merging" section and
cross-link the existing `fetch`/`shell` limit content.

- Add a "Resource budgets" subsection under "## Configure Netsuke" in
`docs/users-guide.md`. Document the defaults, the operator-narrow /
project-cannot-widen rule, the configuration keys, flags, and environment
variables, and the failure diagnostic.

- Run `make markdownlint` and `make nixie` against all new and changed
documentation files.

Research

Netsuke parses a manifest in src/manifest/mod.rs::from_str_named, which builds one minijinja::Environment per parse, registers helpers, stdlib, vars, and manifest macros, calls expand_foreach (src/manifest/expand.rs) to resolve foreach/when, deserialises into NetsukeManifest, then renders every recognised field through render_manifest_with_mode (src/manifest/render.rs).

Every field render funnels through the single choke point render_template in src/manifest/jinja_macros/mod.rs, which currently calls env.render_str and returns an owned String. expand.rs already threads a mutable ExpansionReport accumulator with bounded-sample logic (FILTERED_ENTRY_RETENTION_LIMIT) through the traversal.

Configuration flows via OrthoConfig (src/cli/config.rs), with only NetworkPolicy currently threaded into manifest loading; StdlibConfig holds unused byte-budget builders but does not cover manifest-query mode.

Telemetry follows ADR-009 bounded/redacted discipline in src/manifest/jinja_macros/telemetry.rs. Localization keys are declared in src/localization/keys.rs and mirrored across 34 locale catalogues, enforced by build_l10n_audit.

There is no existing fuel, budget, or operator/project trust mechanism; the minijinja dependency enables only the loader feature.

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

Implement the shared budget foundation that every later phase relies on. Goal:
enable MiniJinja's fuel feature, define a single per-manifest
`ManifestBudget`/`ManifestBudgetLimits` type, force all rendering through a
bounded path, and wire up localized, redacted diagnostics and telemetry for
exhaustion.

- In `Cargo.toml`, add the `fuel` feature to the `minijinja` dependency next to
the existing `loader` feature. Confirm `Cargo.lock` resolves cleanly with no
other feature drift.

- Create a new budget module (`src/manifest/budget.rs` or
`src/manifest/budget/mod.rs`). Define `ManifestBudgetLimits` for configured
ceilings and `ManifestBudget` for runtime accounting state, following the
two-field idiom used by `RetentionPolicy` and `CommandConfig`. Include limits
for: per-evaluation fuel, per-value rendered bytes, aggregate rendered bytes,
aggregate source bytes (template plus macro-import prefix), maximum `foreach`
cardinality, and maximum aggregate expanded target/action count. Supply safe
default constants sized so ordinary manifests render byte-for-byte unchanged.
Reject zero values on construction, mirroring `StdlibConfig`'s `ensure!`
pattern. Give `ManifestBudget` remaining-counter fields per aggregate limit plus
charge/check methods that return a typed exhaustion result naming the exhausted
budget and evaluation stage. Use interior mutability (`Cell`/`RefCell`) so a
shared `&ManifestBudget` can be threaded without widespread `&mut` signature
changes.

- Add `MANIFEST_BUDGET_EXCEEDED => "manifest.budget.exceeded"` to `define_keys!`
in `src/localization/keys.rs`, in the `MANIFEST_*` block. Add the matching
Fluent message with `$stage` and `$limit` variables to
`locales/en-US/messages.ftl` and mirror it, with identical interpolation
variables, across all 33 remaining locale catalogues so `build_l10n_audit`
passes. At call sites, build the message with
`localization::message(keys::MANIFEST_BUDGET_EXCEEDED).with_arg("stage",
...).with_arg("limit", ...)`, passing only a fixed stage label and a numeric
limit — never template text or context values.

- Add a capped `std::io::Write` adapter (in the budget module or beside
`render_template`) that enforces the per-value byte cap and the shared aggregate
remaining-bytes counter, refusing further bytes once either limit is hit and
signalling failure (surfaced by MiniJinja as `ErrorKind::WriteFailure`). Update
`render_template` in `src/manifest/jinja_macros/mod.rs` to accept the
`ManifestBudget`, check the concatenated source size (import block plus
template) against the source-bytes budget before rendering, set per-evaluation
fuel with `env.set_fuel(Some(...))`, render through the capped writer, and,
where the render path exposes the final `State`, read `State::fuel_levels()` to
deduct from the aggregate fuel budget. Map `ErrorKind::OutOfFuel` and the
writer's byte-limit failure to the localized `MANIFEST_BUDGET_EXCEEDED` message,
following the existing `Error::new(ErrorKind::…, localization::message(...))`
pattern from `env_reader.rs` and `jinja_macros/invocation.rs`. Apply the same
per-evaluation fuel cap to expression and `when` evaluation
(`env.compile_expression(...).eval(...)` and `evaluate_when_expression`). Never
call the unbounded `render_str` path again.

- Extend `src/manifest/jinja_macros/telemetry.rs` with a budget-exhaustion
instrument, for example `netsuke_manifest_budget_exhausted_total`, using a fixed
`stage` label vocabulary (`render`, `foreach`, `when`, `macro`,
`expansion_aggregate`, `byte_aggregate`, `source`, `fuel`) plus a `budget`
label. Register once via the existing `Once` pattern. Emit only
closed-vocabulary fields — never template text, macro names, context values, or
rendered output.
===============================================================================

Task: 2

Enforce the manifest budget during the expansion traversal so oversized
`foreach` blocks or aggregate expansion cannot exhaust CPU or memory before the
render phase. Goal: thread the shared `ManifestBudget` into expansion and stop
iteration early rather than materialising whole sequences.

- Extend `ExpansionContext<'a>` in `src/manifest/expand.rs` to carry a shared
`&ManifestBudget` handle alongside the existing `env` and `section` fields, so
`parse_foreach_values`, `expand_target`, `when_allows`, `eval_when`, and
`eval_expression` all read from and charge the same budget instance. Update
`expand_foreach`'s signature to accept the budget and pass it through into
`expand_section` and `expand_target`.

- Change `parse_foreach_values` to consume `seq.try_iter()` lazily, stopping as
soon as the per-`foreach` cardinality budget is exceeded, and return the
localized budget error instead of collecting the full sequence first.

- In `expand_target`, charge the aggregate expanded target/action budget once
per produced entry before cloning the source map, and fail deterministically
once the aggregate budget is exhausted, so many small `foreach` blocks cannot
together bypass the per-block cardinality limit. Where practical, charge an
estimate of cloned-map cost against the aggregate accounting, but keep
cardinality and aggregate count as the primary enforcement levers.

- On exhaustion, emit the budget telemetry event added in Phase 1 with the
appropriate stage label (`foreach` or `expansion_aggregate`).
===============================================================================

Task: 3

Make the manifest budget operator-configurable and ensure the project
configuration layer can only narrow it, then thread the resolved limits through
both build and manifest-query manifest-loading paths. Goal: full end-to-end
wiring from `CliConfig` to `ManifestParse` with a non-widenable project layer.

- Add one field per configurable limit to `CliConfig` in `src/cli/config.rs`,
each with a `default = ...` attribute matching the safe defaults defined in
Phase 1's `ManifestBudgetLimits`. Add a `Cli` accessor
`manifest_budget_limits()` in `src/cli_policy.rs`, following the pattern of the
existing `network_policy()` accessor, that builds a `ManifestBudgetLimits` from
the merged configuration.

- In `src/cli/discovery_layers.rs`, extend the project-scope handling so that,
for every budget field, a project-scope contribution is clamped to no more than
the value already established by the more-trusted layers (defaults, user-scope
file, environment, CLI). A project value may lower the effective limit but must
never raise it. Emit a bounded trace when a project value gets clamped; do not
echo file contents in the trace. Extend the existing
`PostMergeHook`/`validate_*` pattern (mirroring `validate_jobs`) to validate the
final merged limits and reject non-positive or out-of-range values.

- Add a `ManifestBudgetLimits` field to `ManifestParse` in
`src/manifest/mod.rs`. In `from_str_named`, construct the runtime
`ManifestBudget` immediately after `register_manifest_macros` and before
`expand_foreach`, then pass it into `expand_foreach` and the render calls so
both `RenderMode::Full` and `RenderMode::ManifestQuery` share the same budget
instance. Do not place the budget solely inside `StdlibConfig`, since
manifest-query mode never constructs one.

- Add a limits parameter to the public manifest entry points
`from_path_with_policy`, `from_path_with_policy_and_env`, and
`from_path_for_manifest_query`, and to `query::from_path_with_registration`,
applying the limits when building `ManifestParse`.

- Thread the resolved limits through the runner layers:
`runner/mod.rs::load_manifest_with_stage_reporting`,
`runner/generation.rs::load_manifest_for_build` and `load_manifest`, and the
call sites in `runner/graph_generation.rs`, `runner/graph.rs`, and
`runner/help_query.rs`. Compute the limits from `cli.manifest_budget_limits()`
alongside the existing `NetworkPolicy` at each call site.
===============================================================================

Task: 4

Prove the manifest budget works correctly at unit, property, telemetry,
integration, and BDD levels, following the project's established three-layer
test convention. Goal: exact-boundary coverage, redaction guarantees, and
cross-mode shared-budget verification, all driven by deliberately small test
limits.

- Add exact-boundary unit tests (`LIMIT`, `LIMIT-1`, `LIMIT+1`) for `foreach`
cardinality and aggregate expansion, modelled on
`src/manifest/glob/tests/diagnostics.rs::skipped_symlink_diagnostics_retain_a_bounded_sample`.
Drive fixture sizes off the limit constants directly, never magic numbers.
Extend `src/manifest/expand_test_cases/property_cases.rs` and
`foreach_property_cases.rs` with a budget invariant: processed entries never
exceed the budget, termination is deterministic, and the report stays internally
consistent for any generated size that straddles the budget. Add unit tests for
per-value and aggregate byte limits and for the per-evaluation fuel cap using a
compact large-loop template, asserting that failure happens before output
proportional to the loop size gets allocated.

- Add a telemetry test using a local
`metrics_util::debugging::DebuggingRecorder` plus scoped tracing capture,
modelled on `src/manifest/tests/macros_telemetry.rs` and
`macro_invocation_telemetry.rs`. Assert the budget counter increments exactly
once with only fixed labels present, and confirm no template text, context
values, or rendered secrets leak. Add a redaction proptest over arbitrary
manifest content, modelled on
`macro_telemetry_stays_bounded_for_arbitrary_macros`. Add tests proving macro
calls, `when` expressions, normal field rendering, and manifest-query mode all
draw from the same shared budget.

- Extend the manifest-loading test seam so tests can inject small
`ManifestBudgetLimits`, using the limits parameter added in Phase 3. Add a new
`Given` step under `tests/bdd/steps/manifest/` that stores a small budget in
`TestWorld` for consumption by `parse_manifest_inner`, following the
`tests/bdd/steps/stdlib/config.rs` precedent. Add static fixtures under
`tests/data/` (for example a compact large-loop recipe and an oversized
`foreach`) and scenarios in `tests/features/manifest.feature` asserting `parsing
the manifest fails` and `the error message contains` the budget diagnostic text,
reusing the existing localized-error assertion vocabulary. Add a regression
scenario confirming an ordinary manifest below the limits renders byte-for-byte
unchanged.
===============================================================================

Task: 5

Document the new manifest resource budget feature so operators and future
maintainers understand defaults, accounting units, precedence, and failure
behaviour. Goal: a new ADR plus updates to the design document and users guide.

- Create `docs/adr-018-<kebab-case-title>.md` using the section skeleton from
ADR-016/ADR-017: Status, Date, Context and problem statement, Decision,
Rationale, Consequences, Alternatives considered, Implementation references.
Cite the budget module, `expand.rs`, `render.rs`, `jinja_macros`, and the
configuration precedence machinery as implementation references. Add the ADR
entry to `docs/contents.md` under "## Decision records".

- Extend `docs/netsuke-design.md` to describe the budget accounting units, the
defaults, and how the budget fits into the configuration precedence ladder.
Insert this content near the "Discovery scopes and layered merging" section and
cross-link the existing `fetch`/`shell` limit content.

- Add a "Resource budgets" subsection under "## Configure Netsuke" in
`docs/users-guide.md`. Document the defaults, the operator-narrow /
project-cannot-widen rule, the configuration keys, flags, and environment
variables, and the failure diagnostic.

- Run `make markdownlint` and `make nixie` against all new and changed
documentation files.

💡 Iterate on the plan with: `@coderabbitai` <feedback>

Example Feedback
- `@coderabbitai` You can skip phase 3. Add a simple unit test case for phase 2.
- `@coderabbitai` For design choice 1 go ahead with option 3 and replan.

---