Report the resource reads of an evaluation, for cache invalidation

[christopher-buss]

Motivation

SPICE-0001 opens with: "Many build tools require that all inputs are known to them ahead of time, for cacheing purposes." It then defines an input as an import — an amends/extends clause, an import clause, an import expression, or a globbed import.

Resource reads are inputs too, and they are reported nowhere.

A tool that caches an evaluation keyed on the module graph is therefore wrong whenever the module reads a resource:

profiles = if (read?("env:CI") != null) List("ci") else List()

The module bytes are identical in every environment, so an import-keyed cache returns the first evaluated value forever. The first process to evaluate the file decides the answer for every later process, whatever environment that later process had.

hk caches evaluated pkl config keyed on pkl analyze imports output plus file content, and I've hit this problem as discussed in: jdx/hk#1150

env: is the common case, but file:, prop:, http: and external-reader schemes share the problem. read("file:VERSION") is an input that pkl analyze imports does not report.

There is an existing answer to this: the Gradle plugin defaults environmentVariables to [:] — "note that Gradle default differs from CLI default", so evaluation sees only the inputs the build declares. That is hermeticity, and it is right for Gradle. It transfers poorly to a CLI tool: keying a cache on the whole ambient environment over-invalidates on every unrelated variable, and hand-declaring env inputs fails silently when someone forgets, which is the same failure as above, one step later. The evaluator is the only component that knows what was actually read.

Why this cannot be static analysis

pkl analyze imports is possible because import URIs must be string constants:

–– Pkl Error ––
String constant cannot have interpolated values.

read() takes an arbitrary expression, and which reads happen depends on control flow that itself depends on values already read. There is no static equivalent of pkl analyze imports for resources. The record has to come from an evaluation.

Where the record is and isn't reachable today

Every read already funnels through one interface. ResourceReader is public (Optional<Object> read(URI uri)), and EvaluatorBuilder holds a List<ResourceReader> in which ResourceReaders.environmentVariable() is merely a default — so a pkl-core embedder can register a decorating reader and record every URI.

Message-passing clients have a workaround too, though a costly one. A client can register clientResourceReaders for a builtin scheme, and they win by design:

// add client-side resource readers last to ensure they win over builtin ones

But that inverts responsibility. To watch env: reads, the client must reimplement env: resolution, including isGlobbable listing semantics. The observer has to become the resolver.

The CLI has nothing at all: pkl eval emits only the rendered result, and pkl analyze has only the imports subcommand.

So the evaluator already routes every read through a single point. This asks it to report what passed through.

Sketch

One possible shape, mirroring SPICE-0001's three surfaces. Report, per evaluation, which resources were read.

1. Java API — the record attached to the result of an evaluate call.
2. Message Passing API — an optional resourceReads field on EvaluateResponse, opt in via CreateEvaluatorRequest, so clients that do not ask pay nothing and see no wire change.
3. CLI — something like pkl eval --report-reads=<file>, written as JSON.

Four properties matter to a consuming cache:

• Unresolved reads belong in the record. For read?("env:CI") the unset-to-set transition is precisely the one that must invalidate. A record of successful reads only would miss it.
• Globbed reads need the pattern, not just the matches. read*() globs file, env, prop and modulepath. The pattern is itself an input: a newly created file or newly set variable that matches it changes the result while every recorded URI stays identical. SPICE-0001 expands globbed imports into the
  import list, which leaves the same gap, so this needs to go one step further.
• The record is per-evaluation, not cumulative over an evaluator's lifetime, since one evaluator serves many evaluate requests.
• A failed evaluation still needs its record. Knowing a run failed after reading env:CI is what tells a consumer the failure may not reproduce once CI is set.

Scope: this covers reads that pass through the resource-reader interface. Ambient inputs such as pkl.platform, or whatever an external reader process consults behind its URI, are inputs the evaluator cannot see either.

Prior art

pklr, a Rust pkl evaluator, added this for env: in jdx/pklr#145. The change at the point the env: resource resolves was two lines; the remaining ~50 were plumbing the record out through the
API. That says nothing about the cost in pkl's own implementation, but it suggests the shape is small where the reader already sits.

I found no prior issue or discussion asking for this.

AI Disclosure

This issue was triaged via a conversation with Claude Code, but posted and reviewed by myself.