Phase Aware Workspace

Phase-aware workspace

Prompt and decode need different compute-buffer shapes. Phase-aware allocation lets compatible sequential target and MTP contexts share the largest active requirement instead of retaining both peaks.

Control and lifecycle

Enable --phase-aware-workspace; the default is off. Each scheduler publishes its buffer plan at a phase boundary. Growth happens immediately. Shrink waits for every active member, so one context cannot release storage still in use. A later prompt regrows the reservation.

The flag changes transient workspace only. Combine it with live-context workspace: phase-aware sizing follows token geometry, while live-context sizing follows live KV extent.

Note

Backend scope: The shared/resizable allocator contract is backend-generic. Actual physical memory return depends on the backend allocator. Do not equate a smaller logical reservation with an identical drop in driver-reported VRAM on every backend.

Branch implementations

Branch	Implementation
beellama/main	src/llama-context.cpp, ggml/src/ggml-alloc.c
llama/main	src/llama-context.cpp, ggml/src/ggml-alloc.c
llama/dev	src/llama-context.cpp, ggml/src/ggml-alloc.c
moe-cache	src/llama-context.cpp, ggml/src/ggml-alloc.c

Design history: PR 36, share target and MTP phase workspaces.

Feature index - 16 GB setup