Next Generation Architecture

Next-generation architecture

Status: target architecture — proposed. This document describes the direction from the current v1 implementation to v5. Features are not considered implemented until their corresponding code, tests, compatibility evidence and security review have landed.

Product contract

The target experience is:

platform-factory launch --publish --yes

From a project directory, this command detects the project, freezes inputs, compiles it, assembles a multi-stage OCI image, generates an SBOM and provenance, signs and optionally publishes the immutable digest, then runs it.

The architecture is language-neutral and autonomous:

• the core never contains language-specific decisions;
• support for a language is supplied through the same plugin contract used by third parties;
• build, OCI, registry, security and runtime engines are owned by platform-factory;
• the implementation does not invoke BuildKit, Skopeo, Cosign, Pack, CNB lifecycle, or a Dockerfile frontend;
• compatibility formats and protocols are implemented directly from their stable specifications;
• compilers remain explicit, digest-pinned toolchain inputs rather than code reimplemented by platform-factory.

Autonomy does not mean the absence of external systems. Publishing still talks to a selected OCI registry, and a Docker/Podman-integrated runtime necessarily talks to the engine that owns the container lifecycle.

System architecture

flowchart LR
    UX["CLI / local API"] --> Discovery["Discovery + configuration"]
    Discovery --> Planner["Language-neutral planner"]
    Planner --> DAG["Pipeline DAG"]
    Plugins["Out-of-process plugins"] --> Discovery
    Plugins --> Planner
    DAG --> Executor["Sandboxed stage executor"]
    Executor <--> CAS["Content-addressed cache"]
    Executor --> Assembler["OCI assembler"]
    Assembler --> Evidence["SBOM + provenance + signing"]
    Evidence --> Registry["Native registry client"]
    Assembler --> Runtime["Runtime manager"]
    Runtime --> Container["Container backend"]
    Runtime --> MicroVM["MicroVM backends"]
    Runtime --> Shim["containerd shim / OCI runtime facade"]

Loading

The planner is pure: it converts resolved inputs into a validated plan and does not execute commands. Executors consume plans. This separation makes the plan inspectable, policy-checkable, cacheable and testable before side effects.

Proposed package organization

cmd/
  platform-factory/
  platform-factory-plugin/
  platform-factory-runtime/
  containerd-shim-platform-factory-v2/

api/
  v1/
    build.go
    pipeline.go
    plugin.go
    runtime.go
    security.go

internal/
  app/                  use-case orchestration
  detect/               project evidence, no language policy
  planner/              project + frontend -> DAG
  pipeline/             graph validation and scheduling
  stage/                immutable stage model
  executor/             execution contract
  sandbox/              namespaces, limits and mount policy
  cache/                CAS, records, leases and GC
  source/               normalized source snapshots
  filesystem/           snapshots, diffs and rootfs assembly
  oci/                  image, index, layer and artifact formats
  registry/             distribution client and credentials
  sbom/                 inventory and package correlation
  provenance/           build record and attestations
  signing/              native signing and verification
  policy/               declarative security decisions
  plugin/               discovery, RPC and process sandbox
  runtime/
    container/
    microvm/
    containerdshim/
    ociadapter/
  microvm/
    vmm/
    kvm/
    hvf/
    windows/
    devices/
    guest/
  observability/         events, traces, metrics and audit log

Only api/v1 is importable by consumers. No type from internal/ crosses an API or plugin boundary.

Public Go contracts

type ProjectDetector interface {
	Name() string
	Detect(context.Context, SourceTree) (Detection, error)
}

type BuildPlanner interface {
	Name() string
	Plan(context.Context, Project, BuildRequest) (*Pipeline, error)
}

type StageExecutor interface {
	Execute(context.Context, Stage, ExecutionContext) (StageResult, error)
}

type ImageAssembler interface {
	Assemble(context.Context, ImagePlan) (ImageResult, error)
}

type Publisher interface {
	Push(context.Context, Image, RegistryTarget) (PublishedImage, error)
}

type SecurityProvider interface {
	GenerateSBOM(context.Context, ArtifactSet) (Attestation, error)
	GenerateProvenance(context.Context, BuildRecord) (Attestation, error)
	Sign(context.Context, Subject, SigningPolicy) (Signature, error)
	Verify(context.Context, Subject, VerificationPolicy) error
}

type Runtime interface {
	Create(context.Context, RuntimeSpec) (Instance, error)
	Start(context.Context, InstanceID) error
	Stop(context.Context, InstanceID, StopOptions) error
	Delete(context.Context, InstanceID) error
	Inspect(context.Context, InstanceID) (RuntimeState, error)
	Logs(context.Context, InstanceID, LogOptions) (LogStream, error)
}

All operations accept cancellation and deadlines. Identifiers are opaque; errors are typed; credentials and secrets never appear in returned errors.

Pipeline and stage engine

The intermediate representation is independent of the source frontend:

type Pipeline struct {
	APIVersion string
	Inputs     []Input
	Stages     []Stage
	Outputs    []Output
}

type Stage struct {
	ID        string
	DependsOn []string
	Base      ImageReference
	Command   Command
	Env       map[string]string
	Mounts    []Mount
	Secrets   []SecretReference
	Caches    []CacheMount
	Inputs    []ArtifactReference
	Outputs   []ArtifactDeclaration
	Network   NetworkPolicy
	Resources ResourceLimits
	Sandbox   SandboxPolicy
}

flowchart TD
    Resolve["resolve and pin inputs"] --> Freeze["freeze dependencies"]
    Resolve --> Toolchain["resolve toolchain"]
    Freeze --> Build["compile"]
    Toolchain --> Build
    Build --> Tests["tests"]
    Build --> Analyze["runtime dependency analysis"]
    Tests --> Rootfs["assemble minimal rootfs"]
    Analyze --> Rootfs
    Rootfs --> SBOM["SBOM"]
    Rootfs --> Provenance["provenance"]
    SBOM --> Sign["sign subject digest"]
    Provenance --> Sign
    Sign --> Publish["publish"]
    Publish --> Run["optional run"]

Loading

The scheduler rejects cycles and missing dependencies, launches ready branches with bounded parallelism, cancels descendants after failure and emits one structured event per state transition.

Multi-stage semantics

Each stage produces an immutable snapshot. Artefacts cross stage boundaries only through declared transfers:

type ArtifactTransfer struct {
	FromStage string
	Source    string
	Target    string
	Mode      uint32
	Owner     Identity
}

type ImagePlan struct {
	Base        ImageReference
	Transfers   []ArtifactTransfer
	Config      OCIConfig
	LayerPolicy LayerPolicy
}

Semantic layer groups separate runtime, dependencies, application and metadata. Updating a small application must not rewrite a dependency layer measured in terabytes.

Frontends and plugins

Frontends translate source formats into the same pipeline:

type Frontend interface {
	CanHandle(Project) bool
	Resolve(context.Context, Project, BuildRequest) (*Pipeline, error)
}

Planned frontends include automatic detection, the declarative platform-factory.yaml pipeline, a deliberately scoped Dockerfile-compatible parser, a Buildpack-compatible format, and third-party plugins. Compatibility does not invoke the original external implementation.

Go's in-process plugin mechanism is not the extension boundary. Plugins are separate executables using length-prefixed, versioned JSON RPC over standard input/output:

Content-Type: application/vnd.platform-factory.rpc.v1+json
Content-Length: 128

{"id":"42","method":"v1.plan","params":{...}}

type PluginManifest struct {
	APIVersion   string
	Name         string
	Version      string
	Capabilities []Capability
	Platforms    []Platform
	Executable   string
	Digest       string
}

A plugin can detect, freeze, plan or scan. It cannot write an image directly. The host passes a minimal filesystem view, denies network and secrets by default, verifies the plugin digest, constrains resources and validates every returned plan. Official Go, Rust, Java, Node.js, Python and .NET adapters use the same boundary.

Autonomous stage execution

The internal executor owns graph resolution, snapshots, caching, mounts, secrets, isolation, artefact export and OCI layer generation.

A stage cache key is:

SHA256(
  engine-version
  + canonical-stage-definition
  + pinned-base-digest
  + input-digests
  + platform
  + declared-environment
  + secret-identities-without-secret-values
)

Secrets use ephemeral read-only mounts, are excluded from snapshots and are redacted from events. Network access is an explicit stage capability and defaults to disabled after dependency resolution.

Large-volume CAS

cache/
  blobs/sha256/ab/cdef...
  chunks/sha256/ab/cdef...
  records/<stage-key>.json
  leases/<build-id>.json
  indexes/

The store uses streaming writes, fixed-memory copy buffers, atomic installation after digest verification, leases, garbage collection and resumable records. Very large files use chunk manifests so that a local modification does not force rehashing and recopying an entire multi-terabyte input. A corrupt index is reconstructable from verified content.

Toolchains

platform-factory does not reimplement compilers. Toolchains are explicit, digest-pinned OCI inputs extracted and run by the internal executor:

toolchain:
  image: registry.example/platform-factory/go
  digest: sha256:...

Hermetic mode rejects host toolchains, implicit PATH lookup, undeclared downloads and mutable base tags. Every toolchain and input digest is recorded in provenance.

The three-way boundary

"No reimplemented toolchains" and "no shelling out to third-party tools" sound like the same rule but govern three different things, and the non-negotiable constraint is that none of them ever blur into another:

1. The Go standard library - platform-factory itself is written in Go and uses crypto/*, archive/tar, crypto/x509, etc. freely (e.g. Signing, Architecture Decision Records). This is not "an external toolchain reimplemented" - it's the language platform-factory is built in. Its version is pinned once, at the source: go.mod's go directive. Every CI job installs that exact version via actions/setup-go and additionally sets GOTOOLCHAIN=local on every step that invokes the go tool, so a go.mod bump that outpaces the pinned actions/setup-go version fails the build loudly instead of silently fetching a newer toolchain from the network mid-run. This is asserted at runtime too (ci-quality.yml's "Record hermetic toolchain identity" step: test "$(go env GOTOOLCHAIN)" = local) and, as of this page's last verification, checked structurally across every workflow file by scripts/ci/verify-workflows.py (see Testing and CI/CD) - any step that runs go build/test/vet/run/install/env without GOTOOLCHAIN=local in scope fails validation.
2. Application toolchains - compilers and build tools for the pipeline being built (e.g. the Go compiler used to build a user's application inside a stage) are never reimplemented and never resolved from the runner's PATH. They are always digest-pinned OCI inputs (the toolchain: block above), extracted and run inside the sandboxed executor. internal/project.Dependency.Category and internal/policy.Evaluate's ToolchainPinned check enforce that a pipeline can't declare one any other way.
3. Project-owned code - platform-factory's own logic (packaging, SBOM, provenance, signing, policy) is native Go and does not shell out to external CLIs to do its own work. This is enforced by an os/exec usage allowlist checked in CI (ci-security.yml), so a new call to exec.Command outside the small set of legitimate, explicitly reviewed exceptions (e.g. invoking a pipeline's own declared commands inside the executor) fails the build.

Put together: platform-factory trusts the Go stdlib it's written in (pinned, never silently upgraded), never reimplements a toolchain it hands to a pipeline (always a pinned, sandboxed input), and never delegates its own work to an external binary (native code, mechanically checked).

Reproducibility

type ReproducibilityPolicy struct {
	SourceDateEpoch int64
	UID             uint32
	GID             uint32
	FileMode        uint32
	DirectoryMode   uint32
	SortEntries     bool
	NormalizeXAttrs bool
	NormalizeLocale bool
	NormalizeTZ     bool
	Network         NetworkPolicy
}

The engine freezes timestamps, ownership, modes, entry order, locale, timezone, environment, toolchains, base images, dependencies and compression settings.

platform-factory build --rebuild=2 --require-identical

The two builds use fresh sandboxes and cannot reuse each other's final output. A mismatch publishes a structured diff as evidence and blocks release.

Existing application capture

flowchart LR
    Input["installed application / executable"] --> ELF["ELF analysis"]
    ELF --> Needed["interpreter + recursive DT_NEEDED"]
    Needed --> Trace["optional sandboxed trace"]
    Trace --> Minimal["minimal rootfs"]
    Minimal --> Test["isolated execution test"]
    Test --> Image["OCI image"]

Loading

Modes are static (file analysis), trace (controlled observation) and snapshot (isolated before/after comparison). Unknown programs are never traced directly on the host. Runtime libraries, configuration and dynamically opened assets remain explicit evidence; automatic capture cannot prove that an unexercised code path has no additional dependency.

OCI and registry engines

The native engine owns images, multi-platform indexes, semantic layers, whiteouts, content descriptors, related artifacts, import/export and independent verification.

type ContentStore interface {
	Put(context.Context, MediaType, io.Reader) (Descriptor, error)
	Get(context.Context, Descriptor) (io.ReadCloser, error)
	Exists(context.Context, Descriptor) (bool, error)
	Verify(context.Context, Descriptor) error
}

type ImageIndex interface {
	AddManifest(context.Context, ImageName, Platform, Descriptor) error
	Resolve(context.Context, ImageName, Platform) (Descriptor, error)
}

The registry client performs authentication, blob existence checks, mounted and resumable uploads, manifest publication by digest, referrer lookup and post-publication verification. Registry credentials never enter the builder.

Supply-chain security

SBOM generation correlates the file inventory, lockfiles, system package metadata, plugin evidence and ELF dependencies. Provenance records source, commit, resolved configuration, toolchains, plugins, complete DAG, parameters, builder identity, inputs and output digest.

Signing uses Go's native cryptographic primitives. Compatibility with existing signature and attestation envelopes is implemented as an encoding, not by executing a signing CLI. Attestations are related to the immutable image digest, never only to a mutable tag.

type PolicyEngine interface {
	Evaluate(context.Context, EvaluationInput) (Decision, error)
}

type Decision struct {
	Allowed    bool
	Violations []Violation
	Evidence   []Descriptor
}

The first policy language is a small declarative schema owned by the project. Rules cover pinned bases, verified plugins, vulnerability thresholds, SBOM, provenance, signatures, non-root execution, secret leakage, reproducibility, network access and runtime hardening.

Without an external vulnerability feed, the engine can provide a complete inventory but cannot know newly disclosed vulnerabilities. Offline databases must therefore be explicit, signed inputs with recorded age.

MicroVM runtime

type HypervisorDriver interface {
	Platform() Platform
	Capabilities(context.Context) HypervisorCapabilities
	Prepare(context.Context, Image, VMConfig) (BootBundle, error)
	Create(context.Context, BootBundle, RuntimeSpec) (VMInstance, error)
	Start(context.Context, VMInstance) error
	Stop(context.Context, VMInstance, time.Duration) error
	Delete(context.Context, VMInstance) error
	State(context.Context, VMInstance) (RuntimeState, error)
	AttachIO(context.Context, VMInstance) (IOStreams, error)
}

The project owns one MicroVM engine and does not integrate Firecracker, Cloud Hypervisor, libkrun, Kata or another VMM implementation. Platform drivers call only the host's native virtualization boundary: KVM on Linux first, followed by the native macOS and Windows hypervisor APIs. Shared project-owned code implements VM state, memory layout, virtual CPUs, virtio devices, boot loading, networking, storage, vsock and the guest protocol.

Image conversion produces a pinned kernel/init bundle, rootfs, network definition, boot manifest, control channel and minimal project-owned guest agent. Unsupported host capabilities fail closed; there is no fallback to an installed external hypervisor command.

Docker and Podman visibility

A VM launched beside Docker or Podman cannot honestly appear as a container owned by that engine. The engine must initiate the lifecycle:

sequenceDiagram
    participant U as User
    participant E as Docker / Podman
    participant R as platform-factory runtime facade
    participant V as MicroVM backend
    participant G as Guest agent

    U->>E: run --runtime platform-factory IMAGE
    E->>R: create
    R->>V: create VM
    V->>G: boot
    R-->>E: created
    E->>R: start
    R->>G: launch entrypoint
    G-->>R: stdout, stderr, exit
    R-->>E: state and exit code

Loading

Two thin facades share the internal MicroVM engine:

1. a containerd shim v2 endpoint for Docker, containerd and Kubernetes;
2. an OCI Runtime command facade for Podman.

The engine remains the source of truth for identity, ps, inspect metadata, stdio FIFOs, stop and delete. A proxy or "shadow container" is rejected because it creates two divergent lifecycle authorities.

Versioned APIs

The public surfaces are the Go api/v1 package, versioned plugin RPC and a local daemon API over an authenticated Unix socket.

• additions use optional fields;
• unknown extension fields are preserved where round trips require it;
• clients and plugins negotiate capabilities;
• no internal type leaks;
• cancellation and deadlines are mandatory;
• audit identifiers are stable, while implementation identifiers remain opaque;
• breaking changes require a new API package and migration tooling.

Automated backward compatibility (current state)

Two schemas exist so far: the pipeline definition (api/v1alpha1, promoted verbatim as api/v1beta1 via Go type aliases - identical wire shape, only api_version differs) and the out-of-process plugin RPC wire protocol (sdk/plugin, ProtocolVersion = "v1"; api/plugin is a deprecated compatibility shim). Both are guarded by a golden-fixture regression test, not just in-memory struct construction:

• internal/pipeline/testdata/compat/*.json - real pipeline definitions, one per still-supported api_version, decoded from JSON (not built as Go structs) and required to still Analyze() into the exact same topological order every run (internal/pipeline/compat_test.go).
• api/plugin/testdata/compat/*.json - one fixture per RPC frame/typed payload shape (hello, detect, freeze, plan, request/response envelopes, including an error response), decoded and compared field-by-field (api/plugin/compat_test.go).

This matters because encoding/json silently drops fields it no longer recognizes: a hand-built Go struct in a normal test can't catch a renamed or dropped JSON tag, but a fixture crossing the real decode boundary can. The rule for both suites is the same: a fixture is never edited to make a test pass again - a failure there is a genuine backward-compatibility break and must be fixed in the schema/protocol code, or shipped as an intentional new API version.

User flow

discover
→ resolve and explain plan
→ freeze
→ build
→ rebuild verification
→ SBOM
→ provenance
→ sign
→ publish
→ optional run

--yes accepts routine defaults, but never implicitly accepts host networking, privileged execution, an unverified plugin, an unbounded secret, a surprising registry or replacement of a protected tag.

Design trade-offs

• Owning the complete executor reduces supply-chain dependencies but creates a large amount of security-critical code to maintain and audit.
• Complete Dockerfile compatibility is a separate product-sized effort; the initial compatible subset must fail closed on unsupported instructions.
• Docker and Podman require distinct runtime facades.
• OCI byte determinism does not prove compiler reproducibility.
• Rootless MicroVM operation remains constrained by native host hypervisor support and permissions.
• A local signature proves key possession, not a public workload identity.
• Offline vulnerability knowledge is inevitably time-bounded.
• Chunked storage improves incremental indexing but adds manifests, garbage collection and corruption-recovery complexity.

Roadmap

v1 — deterministic assembler

• deterministic and streaming OCI layouts;
• multi-platform and multi-image indexes;
• independent verification;
• project configuration and dependency freezing;
• Docker/Podman container execution;
• experimental direct MicroVM execution.

v2 — internal pipeline

• validated DAG and stages;
• content-addressed snapshots and cache;
• bounded parallel scheduling;
• ephemeral secrets;
• digest-pinned OCI toolchains;
• out-of-process plugins;
• api/v1alpha1.

v3 — native supply chain

• native registry client;
• SBOM and provenance;
• native signing and verification;
• related attestations;
• declarative policy engine;
• isolated byte-for-byte rebuild;
• api/v1beta1.

v4 — engine-integrated MicroVM

• minimal guest agent;
• native platform-factory VMM with a Linux KVM driver;
• Podman OCI Runtime facade;
• Docker/containerd shim;
• lifecycle, logs, inspect, stop and delete;
• network, volumes and resource reporting;
• native macOS and Windows hypervisor drivers after the KVM state machine and virtio device model are stable.

v5 — distributed platform

• remote workers and scheduler;
• distributed CAS;
• parallel multi-architecture builds;
• workload-identity signing;
• high availability;
• stable api/v1;
• Kubernetes RuntimeClass integration;
• automated admission evidence and rollback.

Delivery order

The current deterministic OCI builder remains the trust nucleus. Implement the immutable models and plan validation first, then CAS and stage execution, followed by plugins and security evidence. Registry publication follows only after local verification is stable. The engine-integrated MicroVM shim comes last because it depends on a reliable image store, runtime state machine, network model and guest agent.