Rename folders → memory in HCL (folders kept as alias)#102
Merged
Conversation
User-facing rename: the HCL spellings `shared_memory`, `memory`, `run_memory`,
`memories =`, and `shared_memories.NAME` are now the preferred names for what
were previously called folders. All of the original spellings — `shared_folder`,
`folder`, `run_folder`, `folders =`, `shared_folders.NAME` — keep working
unchanged as backwards-compatible aliases, so existing configs need no edits.
Setting both spellings for the same slot on a single mission (e.g. `memory {}`
+ `folder {}`, or `memories = ...` + `folders = ...`) is rejected at
config-load with a clear error so it can't silently disagree.
Scope is HCL-only by design: Go types (SharedFolder, MissionFolder, etc.),
the file_* tool names, and the `folder` parameter on those tools are
unchanged. Docs (CLAUDE.md, missions/folders.mdx, missions/overview.mdx,
config/overview.mdx, README) updated to lead with the memory spellings and
note the old names as aliases.
Tests: 5 new specs covering the memory spellings and the cross-name
conflict errors; 2 existing specs updated for the refreshed error strings.
Full `go test ./...` passes.
…lings
Builds on the previous folders→memory rename to land a cleaner DSL the
user pushed for:
Top-level shared memory is just `memory "name" { ... }` (no `shared_`
prefix). Mission-scoped memory collapses to a single `memory { type =
"persistent" | "ephemeral" }` block — at most one of each per mission;
`type` defaults to persistent.
Paths are removed from the DSL entirely. Squadron owns every memory
location under <squadron_home>/memories/:
- shared → memories/shared/<name>/
- persistent → memories/mission/<mission_name>/persistent/
- ephemeral → memories/mission/<mission_name>/run/<instance_id>/
The old DSL surfaces are no longer accepted — `shared_folder`,
`shared_memory`, `folder { ... }`, `run_folder { ... }`, `run_memory
{ ... }`, the `folders = ...` attribute, and a `path` attribute on any
memory block all produce explicit errors pointing at the new syntax.
Changes:
- config/memory.go: new Memory (top-level) and MissionMemory (with Type
field) types; drops Path everywhere. SharedFolder, MissionFolder, and
MissionRunFolder are gone.
- config/mission.go: Folders/Folder/RunFolder fields replaced by
Memories/PersistentMemory/EphemeralMemory; Validate signature takes
[]Memory.
- config/config.go: top-level schema gains labeled `memory` blocks,
drops the old block types from the live parse path (still detected to
surface a clear error). Mission schema accepts one `memories` attr
and zero or more `memory {}` blocks with type discrimination.
- mission/folder_store.go: paths derived from paths.SquadronHome() via
new SharedMemoryPath / PersistentMemoryPath / EphemeralMemoryPath
helpers. SweepExpiredEphemeralMemories walks
memories/mission/*/run/* — no per-mission config lookup required.
- cmd/engage.go runFolderCleanupLoop simplified to call the new sweep
once per tick.
- wsbridge: file browser resolves memory paths through the new helpers
instead of reading user-supplied Path. Wire protocol unchanged.
- Tests rewritten for the new DSL and new path scheme (using
paths.SetHome/ResetHome for isolation).
- Docs: CLAUDE.md memory section, missions/folders.mdx, and the
missions/config overview tables updated to describe the new DSL.
User-visible side:
- The file_* tools (file_list, file_read, file_create, file_delete,
file_search, file_grep) now take a `memory` parameter instead of
`folder`. Tool schemas + descriptions agents see say "memory slot"
everywhere — no more "folder" leaking into the LLM-facing surface.
- Agent system-prompt section is now "Available Memory" (was
"Available Folders") with the same `memory` parameter wording.
- The no-code-multi-agent-workflow guide is updated to the new DSL
(it was still showing `folder { path = "./briefs" }`, which would
now fail to parse).
- All in-app docs that referenced "folder" in user-visible copy now
say "memory".
Go internals renamed for coherence with the DSL:
- aitools.FolderStore → aitools.MemoryStore
- aitools.FolderInfo → aitools.MemoryInfo
- aitools.Folder{List,Read,Create,Delete,Search,Grep}Tool
→ aitools.Memory{List,Read,Create,Delete,Search,Grep}Tool
- aitools.MissionFolderName → aitools.PersistentMemoryName
- aitools.RunFolderName → aitools.EphemeralMemoryName
- mission.buildFolderStore → mission.buildMemoryStore
- mission.missionFolderStore → mission.missionMemoryStore
- agent.AgentOptions.FolderStore → MemoryStore (similarly on
CommanderOptions, AgentManagerConfig, internal fields)
- prompts.FormatFolderContext → prompts.FormatMemoryContext
- cmd.runFolderCleanupLoop → cmd.runMemoryCleanupLoop
Files renamed:
- aitools/folder_tools.go → aitools/memory_tools.go
- mission/folder_store.go → mission/memory_store.go
- mission/folder_store_test.go → mission/memory_store_test.go
- wsbridge/shared_folder.go → wsbridge/memory_browser.go
- config/folder_test.go → config/memory_test.go
Intentionally NOT renamed:
- protocol.SharedFolderInfo / protocol.TypeListSharedFolders and the
matching wsbridge handler — these are wire-protocol names owned by
the external squadron-wire repo. Renaming would require a coordinated
cross-repo change.
- internal/paths.ResolveFolderPath — generic path utility used outside
the memory subsystem.
- Generic English uses of "folder" in CLI help text describing OS
directories (e.g. "Dump documentation to a local folder").
- The docs URL /missions/folders stays for link stability; the page
itself renders as "Memory" in the nav.
All tests pass.
shared_memory and run_memory were intermediate names introduced earlier in
this PR's history (between shared_folder → shared_memory → memory, and
run_folder → run_memory → memory { type = "ephemeral" }) and never shipped
to any user. The "no longer supported" errors and the corresponding
detection-only entries in the HCL block schemas don't need to defend
against names nobody has ever written in the wild.
Removed:
- shared_memory entry from the top-level block schema + parse switch
- run_memory entry from the mission block schema + parse switch
- The two unit tests that asserted those errors
- CLAUDE.md mention of shared_memory in the broken-DSL list
Kept (these were actually shipped names): shared_folder, folder, run_folder,
folders=. Their explicit-error paths stay.
Replaces the type-discriminated `memory { type = "persistent" | "ephemeral" }`
block with two semantically distinct block types:
memory { ... } — persistent storage that survives across runs.
At most one per mission. Slot name agents pass:
"memory".
scratchpad { ... } — ephemeral per-run working space. At most one per
mission. Fresh directory per mission run; sidecar-
driven cleanup sweep deletes ones past their
configured window. Slot name: "scratchpad".
Top-level `memory "name" { ... }` (shared, multiple) is unchanged.
The tool parameter agents pass is renamed from `memory` to `slot`, since the
slot values are now literally "memory" and "scratchpad" (a `memory: "memory"`
JSON payload was awkward). Same six tools (file_list, file_read, ...), same
behavior — only the parameter name changed.
On-disk layout:
<squadron_home>/memories/shared/<name>/ — shared memory
<squadron_home>/memories/mission/<mission_name>/ — mission memory
(was .../persistent/)
<squadron_home>/scratchpads/<mission_name>/<run_id>/ — scratchpad
(was .../run/<id>/)
Scratchpads live under their own top-level subdir so the cleanup sweep can
walk one tree without scanning memory directories that should never be
touched.
Go-side renames:
config.MissionMemory — now just { Description } (no Type field)
config.MissionScratchpad — NEW { Description, Cleanup }
Mission.PersistentMemory → Mission.Memory (*MissionMemory)
Mission.EphemeralMemory → Mission.Scratchpad (*MissionScratchpad)
config.PersistentSlotName → config.MemorySlotName ("memory")
config.EphemeralSlotName → config.ScratchpadSlotName ("scratchpad")
config.MemoryType{Persistent,Ephemeral} — removed
config.DefaultEphemeralCleanupDays → DefaultScratchpadCleanupDays
aitools.PersistentMemoryName → MemorySlotName
aitools.EphemeralMemoryName → ScratchpadSlotName
aitools.{tool}.Memory field → .Slot (JSON tag "memory" → "slot")
mission.PersistentMemoryPath → MissionMemoryPath
mission.EphemeralMemoryPath → MissionScratchpadPath
mission.MissionRunRoot — removed (subsumed by ScratchpadsRoot)
mission.SweepExpiredEphemeralMemories → SweepExpiredScratchpads
cmd.runMemoryCleanupLoop → cmd.runScratchpadCleanupLoop
Tests in config/memory_test.go and mission/memory_store_test.go fully
rewritten for the new block types and path scheme. CLAUDE.md and the
public docs (missions/folders.mdx, missions/overview.mdx, sidebar label,
README features list) updated to describe the new design.
Full `go test ./...` passes.
…on required
DSL changes:
- scratchpad is now a boolean attribute on the mission:
mission "x" { scratchpad = true }
No block, no fields. Auto-deletes 7 days after the run starts;
cleanup window is fixed (config.ScratchpadCleanupDays = 7).
- Both the top-level `memory "name" { ... }` and the mission-scoped
`memory { ... }` blocks now take the same single field: a required
`description`. The `label` and `editable` attributes are gone — all
memories are writable.
- A mission with no `memory { ... }` block has no "memory" slot;
agents only see the shared memories it lists (plus the scratchpad,
if enabled).
Internal changes that follow:
- aitools.MemoryStore.ResolvePath drops the writable bool return:
ResolvePath(slot, relPath) (string, error)
aitools.MemoryInfo loses the Writable field. The "slot is
read-only" error paths in file_create and file_delete are gone,
along with the same check in wsbridge's file-browser writer.
- config.MissionScratchpad type is removed. Mission.Scratchpad is
now `bool` instead of `*MissionScratchpad`.
- config.DefaultScratchpadCleanupDays renamed to
config.ScratchpadCleanupDays (no longer "default" — it's the only
value).
- config.Memory: drops Label and Editable; Description is required
by the HCL parser.
- config.MissionMemory: same shape, just a required Description.
Tests rewritten for the simpler surface. Docs (CLAUDE.md, the
missions/folders.mdx page, the missions overview table, README) updated
to describe the new design.
`go test ./...` is green across all packages.
…ref error
Six fixes from the high-recall review of this PR:
1. Validate shared-memory + mission names against path-component abuse
(config/memory.go validateSlotName, called from Memory.Validate and
Mission.Validate). Rejects names containing `/`, `\`, `..`, or starting
with `.` so `memory "../escape"` and `mission "../pwn"` no longer
slip through filepath.Join + MkdirAll to create directories outside
<squadron_home>. The previous code accepted these because HCL labels
are arbitrary quoted strings.
2. Reject shared-memory labels that collide with a mission name
(config/config.go c.Validate). The wsbridge file-browser keys both
under the same string-name namespace; a collision silently masked the
mission's persistent memory in the UI. Now caught at load time with
a clear "memory X: name conflicts with mission X" error.
3. Sort MemoryInfos() output (mission/memory_store.go). Go map iteration
is randomized, and the result feeds the agent's system prompt via
prompts.FormatMemoryContext. Without a sort, the prompt bytes change
run-to-run and Anthropic prompt caching misses on otherwise-identical
missions.
4. SweepExpiredScratchpads no longer aborts on the first per-mission
ReadDir error. Previously a permission-denied or transient IO failure
on one mission's scratchpads dir halted cleanup for every other
mission in the same pass; now it's a `continue`.
5. Legacy-block error messages (config/config.go) used to tell users to
migrate to `memory { type = "persistent" }` and `type = "ephemeral" }`
— neither exists in the final DSL. Updated to point at the actual
replacements: `memory { description = "..." }` and `scratchpad = true`.
6. Always register the `memories` HCL eval-context variable (even when
empty). Previously a mission writing `memories = [memories.foo]` with
no top-level memory block declared got the cryptic HCL error
"Unknown variable; There is no variable named 'memories'"; now it
gets "This object does not have an attribute named 'foo'" which at
least names the bad reference.
Tests added:
- config/memory_test.go: path-component rejection, name collision
rejection (via LoadAndValidate since c.Validate isn't called from
LoadFile), missing-reference error contains the bad name, run_folder
remediation mentions `scratchpad = true`.
- mission/memory_store_test.go: MemoryInfos returns stable order
across multiple calls, sweep doesn't halt when one mission's subdir
is unreadable.
Findings deliberately not addressed (per user decision):
- Resume across PR boundary fails for in-flight sessions (JSON tag
"folder" → "slot" change). Documented as breaking; users restart.
- Scratchpads remain invisible in the command-center file browser
(preserved behavior from the old run_folder; revisit later).
4 tasks
Two doc gaps surfaced after testing the new DSL end-to-end:
1. docs/content/missions/folders.mdx — the Mission Scratchpad section
explained the *what* but not the *when*. Added a "When to use a
scratchpad" subsection with four concrete use cases (multi-step
pipelines, intermediate compute, concurrency isolation, resume
material) and a Memory-vs-scratchpad decision table.
2. docs/content/missions/internal-tools.mdx — the page listed every
commander tool, every dataset tool, but nothing about the six
file_* tools that get auto-attached when a mission declares any
slot. Added a File Tools subsection mirroring the Dataset Tools
layout, calling out:
- the auto-attach rule (file_* appear only when a slot is
declared)
- the slot parameter and its three sources ("memory",
"scratchpad", or a shared memory's label)
- the rejection of absolute paths and `..` escapes
- a link back to /missions/folders for the storage layout
3 tasks
mlund01
added a commit
that referenced
this pull request
May 30, 2026
Follow-up to #102 — fills the test gaps the post-ship audit flagged at the tool-implementation, agent auto-attach, prompt-formatter, and wsbridge file-browser layers. All new tests use Ginkgo + Gomega to match the project's BDD pattern. aitools/memory_tools_test.go (28 specs, package aitools_test) - schema: every file_* tool advertises `slot`, not legacy `folder` - schema: legacy {"folder": ...} payloads are rejected at unmarshal - file_list: flat, recursive, pagination, empty, unknown slot - file_read: contents verbatim, max_lines, max_bytes, missing path, directory rejected, path traversal rejected - file_create: new, fail-if-exists, overwrite, append (success + fail-if-missing), creates parent dirs - file_delete: file removed, directories rejected - file_search: regex match, no-match summary, bad regex - file_grep: flat match, recursive descent, no-recursion-by-default agent/internal/prompts/prompts_test.go (7 specs, new suite) - FormatMemoryContext nil/empty → empty string - "memory" slot gets the persistent-mission label - "scratchpad" slot gets the ephemeral label - Shared slots get no reserved-name label - Output advertises `slot` param + names all six file_* tools - Iteration order is preserved (sort lives in mission/memory_store.go) wsbridge/memory_browser_test.go (17 specs, package wsbridge, joins the existing Wsbridge HumanInput Suite) - collectMemoryInfos: shared + per-mission memory both emitted with correct paths under SquadronHome - collectMemoryInfos: Editable=true for every entry (no read-only mode) - collectMemoryInfos: missions without memory{} skipped, scratchpads not exposed (preserved behavior, pinned) - resolveMemoryPath: shared, mission, collision priority (shared wins — pinned so a future priority swap is loud), unknown name, mission without memory{}, end-to-end real file read - resolveSafePath: rejects ../, mid-path ../, absolute paths; accepts valid relative + empty (root) agent/memory_tools_attach_test.go (3 specs, new suite, package agent_test) - All six file_* tools attached when MemoryStore is non-nil - All six omitted when MemoryStore is nil (negative case) - Attached tools wired to the exact store passed in What's still uncovered (low ROI): - cmd/engage.go runScratchpadCleanupLoop — tiny shim around mission.SweepExpiredScratchpads - Full end-to-end mission with memory + scratchpad — covered by the manual smoke test in #102; a true integration test would need an LLM call
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
shared_memory,memory,run_memory,memories =, andshared_memories.NAMEas the preferred HCL spellings for what were previously called folders.shared_folder,folder,run_folder,folders =,shared_folders.NAME) working unchanged as backwards-compatible aliases — existing configs need no edits.memory {}+folder {}, ormemories = ...+folders = ...) is rejected at config-load with a clear error so the two can't silently disagree.Scope (intentionally narrow)
HCL block names only. Not changed:
SharedFolder,MissionFolder,config/shared_folder.go,mission/folder_store.go, …)file_list,file_read,file_create,file_delete,file_search,file_grep)folderparameter those tools take (still literallyfolder: "mission"etc.)"mission"and"run"This keeps the diff small and avoids breaking any agent prompts, tool catalogs, or persisted sessions that reference the existing tool/parameter names.
Docs
CLAUDE.md— section renamed to "Memory (formerly 'Folders')" with a back-compat tabledocs/content/missions/folders.mdx— page title and content lead with memory; URL unchanged for stabilitydocs/content/missions/overview.mdx— mission attr/block table updateddocs/content/missions/_meta.js— sidebar label flipped to "Memory"docs/content/config/overview.mdx— top-level-blocks table updatedREADME.md— features list updatedTest plan
go build ./...go test ./...— all packages passconfig/folder_test.go:shared_memory+memories+memory+run_memory) parse cleanly togethermemories = ...+folders = ...on the same mission → errormemory {}+folder {}on the same mission → errorrun_memory {}+run_folder {}on the same mission → error