Filesystem artifact inbox and projection adapter

[devkade]

Summary

Implement filesystem artifact handling for raw logs, diffs, transcripts, test output, and human-readable projections.

Parent: #236

Scope

Create crates/artifacts package ilchul-artifacts.

Implement:

ArtifactStore
artifact inbox
hashing/digest sealing
artifact refs
bounded summaries
human-readable projections

Boundary

Filesystem artifacts are not canonical completion state.

Acceptance criteria

• Artifact refs include path/URI and digest.
• Runtime output can be stored as artifact refs.
• EvidenceRecord can reference artifacts.
• Raw logs are not inlined into events by default.
• Artifact tests pass.

[devkade]

AI-understood issue summary

Issue #242 asks for a new Rust crates/artifacts package named ilchul-artifacts that stores large runtime byproducts (raw logs, diffs, transcripts, test output) on the filesystem and exposes sealed artifact references plus bounded human-readable projections. The key boundary is that filesystem artifacts/projections are inspectable evidence support, not canonical completion state.

Why this issue is needed

• Parent SSOT: Ilchul Agent OS — MCP TaskStream control plane MVP #236 defines the rewrite boundary as Rust TaskStream authority with filesystem artifacts/projections outside canonical completion state; this issue is the storage/projection slice that prevents raw runtime output from bloating canonical events.
• Existing TS Kapi already models the same separation: artifacts are durable checkpoints rather than scratchpads (README.md:19-26, README.md:175-185), validation remains evidence/contract driven (README.md:218-230), and thinness explicitly prefers artifact-on-demand over universal logging (README.md:258-266).
• Current worker output storage is intentionally bounded to 4000 trailing characters (src/application/workflow-service.ts:383-400, tested in test/worker-monitor-output-truncation.test.ts:31-45), which supports the issue’s “bounded summaries” requirement but also shows why full raw logs need a separate artifact inbox.

Why this issue may not be needed

• If MVP runtime output remains small and only requires a short tail, the existing bounded-output pattern would be enough (src/application/workflow-service.ts:383-400), and a new artifact crate could be deferred.
• The repository currently has no tracked Rust workspace/crate files (git ls-files found no Cargo.toml, crates/, or .rs files), so Filesystem artifact inbox and projection adapter #242 should probably land after or alongside Rust workspace scaffold + DDD core skeleton #238’s Rust scaffold rather than independently.
• A full object-store abstraction may be premature for the MVP. Official object_store docs show a broad async API for local/cloud object storage, conditional writes, listing, etc. (“object_store - https://docs.rs/object_store/latest/object_store/”), but issue Filesystem artifact inbox and projection adapter #242 only requires local filesystem artifact refs, digest sealing, and projections.

Expected Outcome

After approval and implementation, the implementation agent will add crates/artifacts / ilchul-artifacts with:

• ArtifactStore filesystem adapter rooted under a run/task artifact inbox directory.
• ArtifactRef values that include path/URI, digest algorithm, digest hex, byte length, and a kind/media label where useful.
• Runtime output ingestion APIs that write raw bytes to the inbox and return refs instead of inlining raw output into canonical events by default.
• Evidence integration so EvidenceRecord can reference artifact refs without making filesystem artifacts canonical completion state.
• Bounded projections/summaries for humans, with tests proving raw logs remain external by default.

Evidence-based implementation plan

Feasibility from repo research

Reusable in-repo patterns exist, although they are TypeScript and should be treated as design precedent rather than code to copy into Rust:

• src/domain/types.ts:49-53 defines the current ArtifactRef as name, path, and required; Filesystem artifact inbox and projection adapter #242 should extend this concept in Rust with URI/path + digest metadata.
• src/domain/types.ts:74-93 defines validation evidence with artifactRefs?: string[], showing the existing model already expects evidence to point to artifacts.
• src/domain/state-machine.ts:170-179 creates workflow artifact roots and tracked artifact refs at workflow start.
• src/adapters/file-store.ts:53-90 ensures artifact files/directories exist and supports append/write/read/metadata operations.
• src/adapters/file-store.ts:131-188 rejects empty names, path separators, and paths escaping the artifact root; reuse this safety invariant in Rust.
• src/adapters/file-store.ts:273-300 checks state/artifact ancestors and files for symlink hazards; the Rust implementation should preserve this class of defense.
• src/application/workflow-service.ts:198-243 shows how newly written artifacts become tracked refs, with a prevalidation phase before writes.
• src/domain/workflow-validation.ts:253-319 validates artifact root/path invariants and missing workflow artifacts.
• src/domain/workflow-validation.ts:186-188 and test/evidence-artifact-ref-validation.test.ts:6-35 prove evidence artifact refs must point at tracked artifacts.
• src/presentation/schemas.ts:13-24 exposes artifactRefs and commandRefs in evidence inputs; src/presentation/messages.ts:161-182 renders artifact refs in evidence lists.
• src/application/workflow-service.ts:383-400 and test/worker-monitor-output.test.ts:31-43 persist only a bounded worker-output snapshot today; test/worker-monitor-output-truncation.test.ts:31-45 asserts the 4000-character cap.
• README.md:145-147 documents current artifact path safety and that Kapi may record subagent results as artifacts/evidence without owning the subagent runtime.
• README.md:280-304 documents the current domain/application/adapter split; the Rust crate should preserve this separation by keeping digest/ref domain types independent from filesystem adapter code.
• docs/learning-runtime-boundaries.md:104-107 explicitly says evidence refs point to inspectable artifacts/command outputs/runtime events and evaluation results are not completion authority.

External library/API research:

• sha2 is a suitable low-level digest dependency for stable SHA-256 sealing (“sha2 - https://docs.rs/sha2”). The docs expose Sha256 plus the Digest trait.
• hex is a suitable tiny dependency for digest encoding (“hex - https://docs.rs/hex/”).
• serde / serde_json are suitable for serializable artifact refs, manifests, and projection metadata (“serde - https://docs.rs/serde/latest/serde/”, “serde_json - https://docs.rs/serde_json”).
• tempfile is suitable for filesystem tests with isolated temp dirs (“tempfile - https://docs.rs/tempfile/latest/tempfile/”).
• camino may be useful if the Rust core wants UTF-8 paths in serialized refs (“camino - https://docs.rs/camino/latest/camino/”), but use std::path::PathBuf if arbitrary platform paths matter more than JSON-native UTF-8.
• object_store is a known broader local/cloud object abstraction (“object_store - https://docs.rs/object_store/latest/object_store/”), but it is likely too broad for the MVP unless SSOT: Ilchul Agent OS — MCP TaskStream control plane MVP #236 intends future remote artifact storage in this slice.
• cacache and hd-cas provide content-addressable caches/stores (“cacache - https://docs.rs/cacache”, “hd_cas - https://docs.rs/hd-cas/latest/hd_cas/”), but they appear heavier than needed because Filesystem artifact inbox and projection adapter #242 needs explicit artifact refs/projections under Ilchul’s domain boundaries, not a generic cache as completion-adjacent authority.

Step-by-step instructions for the implementation agent

1. Confirm scaffold from Rust workspace scaffold + DDD core skeleton #238 first. If Cargo.toml / crates/ still do not exist, create only the minimum workspace-compatible crates/artifacts package needed by Filesystem artifact inbox and projection adapter #242, named ilchul-artifacts; otherwise follow the existing Rust workspace conventions. Keep README aligned if new commands, crate layout, or validation gates are documented, per repo instructions.
2. Define domain-first ref types. Add Rust structs/enums such as ArtifactKind, Digest { algorithm, hex }, ArtifactRef { uri/path, digest, size_bytes, kind, projection? }, and an ArtifactStore trait. Ground this in the existing TS precedent that refs are tracked separately from validation evidence (src/domain/types.ts:49-53, src/domain/types.ts:74-93).
3. Implement local filesystem store with path confinement. Mirror the safety invariants from src/adapters/file-store.ts:131-188 and src/adapters/file-store.ts:273-300: reject empty names/path traversal, keep writes under the configured inbox root, avoid following unsafe symlink paths, and write through temp/atomic patterns where practical. Use tempfile for tests.
4. Seal artifacts by digest during ingestion. For every raw log/diff/transcript/test-output write, stream or read the bytes, compute SHA-256 with sha2, encode with hex, persist content under an inbox path, and return an ArtifactRef containing both URI/path and digest. If content-addressed layout is chosen, use the digest in the filename/path; if semantic filenames are chosen, still include digest in the ref.
5. Add bounded projection support. Implement deterministic summary/projection helpers that store or return bounded text (for example first/last N bytes or lines, total byte count, truncation marker, digest). This should replace the current “bounded worker output in state” pattern with artifact-backed raw output plus compact projection, while preserving the principle proven by src/application/workflow-service.ts:383-400 and test/worker-monitor-output-truncation.test.ts:31-45.
6. Integrate with evidence records without changing authority. Update the Rust EvidenceRecord model from Core domain invariants for TaskStream authority #239/Core services and explicit port contracts #240 so it can hold ArtifactRef or artifact-ref ids. Preserve the rule tested in TS that evidence refs must be tracked (test/evidence-artifact-ref-validation.test.ts:6-35) and the boundary from docs/learning-runtime-boundaries.md:104-107 that refs support evidence but do not complete work by themselves.
7. Prevent raw output in canonical events by default. Event payloads should carry artifact refs + bounded projections, not full logs. If an explicit debug mode exists, gate raw inline output separately and keep default event serialization slim.
8. Add artifact tests. Cover: path/URI + digest in refs; digest mismatch detection on read/verify; runtime stdout/stderr/test output saved as artifact refs; evidence records referencing artifacts; raw logs excluded from default event payloads; bounded projection truncation; path traversal and symlink rejection; missing/corrupt artifact behavior. Run the Rust tests for the new crate and the repo’s existing verification command if feasible (npm run verify currently exists in package.json).

Risks / assumptions from research

• The issue names Rust paths, but the checked-out repo is currently TS-only; the implementation agent may need to coordinate with Rust workspace scaffold + DDD core skeleton #238/Core domain invariants for TaskStream authority #239/Core services and explicit port contracts #240 outputs before finalizing APIs.
• Digest algorithm should default to SHA-256 unless a human explicitly wants BLAKE3/content-addressed performance. sha2 is conservative and broadly recognizable for sealing refs.
• Do not adopt object_store, cacache, or hd-cas unless the approved design expands beyond local filesystem artifact inboxes; otherwise they risk hiding Ilchul-specific ref/evidence semantics behind a generic storage API.
• Projection text should be treated as a human-readable convenience. Completion authority must remain with kernel/evidence validation, consistent with README.md:218-230 and docs/learning-runtime-boundaries.md:112-119.

Assumptions / human decisions to resolve asynchronously before /approve

• Assumption: SHA-256 is acceptable for digest sealing in MVP.
• Assumption: local filesystem storage only; no cloud/object-store support in Filesystem artifact inbox and projection adapter #242.
• Assumption: artifact refs may use file:// URIs or workspace-relative paths as long as the ref includes a digest and the implementation consistently validates roots.
• Optional human decision: choose whether artifact files should be content-addressed by digest or stored under semantic run-output names with digest metadata.

Approval reminder

Implementation starts only after a human comment containing /approve. This planning comment is the handoff for the future implementation agent; no branch or PR is created by this planning session.

Agent Signature

Planning agent for dani non-interactive issue review. Read-only repository analysis plus external docs/package research completed; comment posted for asynchronous approval.

[devkade]

/approve

[devkade]

Completed by the merged feature PR for this issue. Closing manually because the PR body did not include a GitHub closing keyword before Dani was updated to add Closes #<issue>.