chore: implement persistent terraform directories (experimental)

[Emyrk]

Screencast.From.2025-10-30.14-28-48.webm

Behavior change

Prior to this, every workspace build ran terraform init in a fresh directory. This would mean the modules are downloaded fresh. If the module is not pinned, subsequent workspace builds would have different modules.

This change locks in a modules "version". If different provisioners warm their cache at different times, each provisioner could have a different truth for the module "version"

Implementation

tfpath.go is the implementation of where all terraform file management occurs. This PR makes the implementation an interface (unfortunate, will not use an interface when released).

type LayoutInterface interface {
	WorkDirectory() string
	StateFilePath() string
	// ...
	Cleanup(ctx context.Context, logger slog.Logger, fs afero.Fs)
	CleanStaleSessions(ctx context.Context, logger slog.Logger, fs afero.Fs, now time.Time) error
}

This moves almost the entire implementation into tfpath/x/tfpath.go. All provisioned workspaces use the new x package when the experiment is enabled.

Implementation details

• All terraform.tfstate files are written to a subdirectory terraform.tfstate.d/<session>.
• All terraform cli execs use the new terraform workspace context

TODO:

• Make sure modules are pinned to allow this
• Add more testing. Since it is almost all terraform interactions, it has only been manually tested so far. Hence the x package

[Emyrk]

• chore: per template opt into cached terraform directories #20609
• chore: implement persistent terraform directories (experimental) #20563 👈 (View in Graphite)
• chore: add experiment toggle for terraform workspace caching #20559
• chore: append template metadata to protobuf config #20558
• chore: refactor terraform paths to a central structure #20566
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.