Technical Guide

Technical Guide

Alpha: use at your own risk; may change daily. See the release policy.

Agent Y details on this page describe 1.9.0-alpha.2 tooling. The older 1.8.9-alpha.1 release includes Agent X/Z only.

.SYSTEMX is a file-based project operating format with optional Python tools. It supplies records, local validation, bounded context, version selection, and reversible installation lifecycle operations. It is not a model runtime or a service that automatically reads every project or starts agents.

Architecture

The Visual Guide provides Mermaid trees and record-flow diagrams. The subagent sequence separates runtime messages from saved project state; neither diagram adds an execution service to this file format.

Host project
└── .SYSTEMX                       exact stored directory name
    ├── GLOBAL, PLAN, MEMORY       project context, outcomes, durable facts
    ├── WORK                       canonical tasks and current focus
    ├── AGENTS                     Agent 0, optional X/Y/Z, and scoped worker memory/jobs
    ├── EVENTS                     event and due ledgers after activation
    ├── REVIEWS                    policy, requests, scorecards after activation
    ├── Projects/REGISTRY.json      explicit child identities
    ├── Projects/NAME/.SYSTEMXP     isolated child records; no nested installation
    ├── project.json               explicit project command arguments
    ├── INSTALLATION.json          selected defaults, update policy, archive receipts
    └── .systemx                   internal manager storage
        ├── releases/<version>     retained, fingerprinted default snapshots
        ├── history                previous installation metadata
        └── operations             local installation operation receipts

The optional sibling .systemx -> .SYSTEMX link is a second route to the same folder. It is different from the nested manager cache. Follow the exact-case contract.

Components and data flow

Component	Responsibility
manager.py	Install/update, release checks, bootstrap refresh, pins, profiles, first run, audit, uninstall, restore, chat export
systemx_paths.py	Exact stored spelling, project boundaries, case-conflict and alias checks
lifecycle.py	File inventories, raw-byte hashes, atomic JSON receipts, backup boundaries
scripts/systemx.py	Configuration validation, menu, doctor, explicitly configured commands
scripts/project_memory.py	Task ledger, focus, generated views, agent records, bounded context
scripts/project_workspaces.py	Explicit child scope routing, blank creation, scoped commands, local snapshots
scripts/agent_standards.py	Explicit X/Y/Z setup, safe records, common scope dispatch
scripts/agent_x.py	Event occurrence/recording time and due-item tracking
scripts/agent_y.py	Bounded external CLI job records, worktree handoff, and guarded GitHub PR transitions
scripts/agent_z.py	Fixed 100-question policy, request/report validation, score math and deltas
config/distribution.json	Explicit public file inventory with release fingerprints
config/template-records.json	Reviewed blank seed fingerprints

The library and CLI use the same manager functions. A managed run verifies its selected default snapshot and invokes its runner with Python -I -B and --root pointing to the outer project records. An already selected, verified snapshot older than 1.8.7-alpha.1 runs through an isolated compatibility shim; the historical runner keeps its old behavior and record limits. The current manager can inspect such an installation, but cannot install or newly select a release below 1.8.7-alpha.1. Historical reselection needs a separately reviewed backup or legacy path. Old root defaults are preserved. Task state is stored once in WORK/TASKS.json; CURRENT and status pages are derived views. The format contract defines record fields and transitions.

Installation lifecycle

First run previews by default. Applying it installs or adopts the canonical folder, pins a new installation, and creates an empty project.json only when absent. It executes no project checks or app setup commands. Existing managed versions and policies remain selected. Follow first-time setup. During a first install or adoption, a preexisting distribution .py, .sh, or .ps1 file with different content is a conflict. Inspect it rather than silently adopting executable code that the reviewed release did not supply.

Updates add snapshots and missing root files. They do not replace project records or remove upstream-deleted files. Manual updates remain the default; in the 1.8.9 and 1.9.0-alpha.2 managers, startup checks are opt-in notices and never select an update. Major upgrades require review. Distribution hashes normalize CRLF to LF for portable text checkouts. They check consistency with the manifest; they are not independent release signatures. A selected release cache must also have exactly the expected files and parent directories: unlisted modules, packages, links, special entries, and unexpected directories are rejected before the cached runner executes. Partial snapshots can be resumed only when every present entry matches the reviewed distribution.

The direct manager.py, scripts/systemx.py, and scripts/release.py entry points enter Python isolated mode (-I) before importing application or standard-library modules. The manager and runner load named local helpers from their local source bytes under private module names, rather than accepting an unlisted module or a substituted bytecode cache. This blocks the direct-script routes for root module shadowing and poisoned .pyc files; it does not authenticate the named source files, the publisher, or code in the host Python installation. Refreshed managed shell and PowerShell launchers invoke the manager with -I -B; use python3 -I -B /reviewed/new/.SYSTEMX/manager.py for a reviewed external manager invocation. The installed systemx console entry point first resolves its package through the launching interpreter's import paths, then restarts in isolated mode before importing the manager. For isolation from the initial import, invoke a reviewed virtual environment's Python with -I -B -m systemx. Importing systemx as a library inherits the calling application's Python trust environment. Existing managed projects may retain an older root manager. If its eight stock bootstrap files differ from the selected new release, update stops before selection. They are manager.py, lifecycle.py, versions.py, systemx_paths.py, SYSTEMX.sh, SYSTEMX.ps1, INSTALL.sh, and INSTALL.ps1. Run bootstrap-refresh from a separately reviewed new tool or checkout, first as a preview and then with --apply. It accepts only bootstrap bytes found in the new distribution or intact retained releases. A genuinely absent stock file can be added if no intact retained release ever contained it; this includes historical installations that predate versions.py. A file that once existed but is now missing, customized, or unrecognized blocks the refresh. Only replaced files have byte-for-byte backups under .SYSTEMX/.systemx/history/; an add-only refresh has no old file to back up. The operation is logged. See installation and updates.

An exact remote tag ZIP can have a separately trusted raw SHA-256 pin checked before archive parsing. INSTALLATION.json records explicit-pin, observed-only, or no remote digest for each selected release. In the 1.8.9 and 1.9.0-alpha.2 managers, startup discovery reads metadata only and waits at most five seconds before continuing with installed defaults; a later manual download can use an independently obtained digest pin. Neither the manifest nor an observed digest makes a movable tag immutable. See installation.

Uninstall is a separate explicit operation: it inventories the complete folder, moves it to an external backup on the same filesystem, removes a valid local alias, and writes an external receipt. It does not infer which customized files are safe to delete. Restore compares raw-byte hashes, empty directories, and link targets against that receipt and refuses an occupied destination. See uninstall and cleanup for limits and recovery.

Concurrency, integrity, and logs

Install/update and removal use an exclusive local writer lock. Task, Agent X, Agent Y, and Agent Z command writes use a separate per-scope coordination lock. Cooperating coordination writers wait up to one second; the lock records a token, PID, host, and creation time. On a crash it remains for manual inspection and is never automatically removed as "stale." Operation receipts record started, completed, or failed work; an abrupt process termination can leave a started receipt and stale lock. Check the owner before recovery. Installation state and lifecycle receipts are written through temporary files and atomic replacement. Earlier state is retained. No local lock is a distributed Drive lock, and no tool can guarantee a consistent snapshot against an uncooperative writer. Stop active workers before removal, restore, or cloud handoff.

Uninstall inventories never follow content symlinks and do not execute files. Backups preserve private contents and must remain private. Filesystem permissions, the host's instruction hierarchy, and existing service authority still apply. The tools do not provide an authentication or sandbox boundary.

Runtime and extension points

The optional CLI requires Python 3.9+ and its standard library. Configured project commands are explicit argument arrays and run with shell=False from the project root. Integrations should preserve that boundary and canonical records. Consult the Stack Guide and library API.

An MCP adapter could later expose bounded read/write operations over this library. No MCP server, daemon, remote agent mesh, OAuth registration, or cloud sync engine is currently installed. Extensions must define ownership, authorization, logs, and removal behavior rather than implying that the template grants those powers.

Designing a future adapter

Keep any new harness or MCP adapter behind the existing project boundary:

• Resolve one explicit project and selected default version for each operation.
• Separate bounded reads, proposed changes, and authorized writes. Honor the host's permissions; role names and reviewer strings are not credentials.
• Use canonical commands and record formats. Map runtime job IDs in coordination notes instead of creating a competing task database without a migration plan.
• Serialize writes across the adapter's actual deployment. The current local lock is not a distributed lock across machines, worktrees, or Drive copies.
• Make retries safe to reconcile: inspect the previous result before repeating a task creation, file write, or external action. Log enough to recover it.
• Define privacy, retained evidence, version compatibility, and removal of any extra integration files, services, or credentials the adapter installs.

These are extension requirements, not capabilities of a shipped MCP server. The current integration surface is the library and CLI.

Task scale and context scale

The task tool reads the JSON ledger as a whole and writes it back while updating generated views. Bounded context output limits what an assistant initially sees; it does not make storage or validation independent of ledger size. Histories, evidence references, and generated status pages grow with recorded work.

The 10,000+ task-event goal concerns project continuity. It is not a throughput benchmark, task queue, or automatic archival feature. Measure representative large ledgers in the target environment before making capacity claims. Preserve IDs, dependencies, and evidence when designing a future storage migration. See Anti-drift and long-running work.

Verification and efficiency

validate --template checks a pristine public distribution. Ordinary validate checks an adopted folder and records; configured check commands test the host project. Neither establishes production readiness by itself. Tests use temporary projects across Windows, macOS, Linux, and supported Python versions. Token and time efficiency explains the scoped-context design and how to measure its effect without claiming unmeasured savings.

Child project scopes

SYSTEMX PROJECTS routes selected operations to Projects/NAME/.SYSTEMXP while reusing project_memory.py for the same record invariants. Configured child commands run from Projects/NAME/. Each ledger has its own local coordination lock; project creation locks the outer registry. Snapshots contain selected task counts, focus, and source hashes, not a service health verdict. Shared default snapshots and version selection remain at the outer installation. A scope selector is not an OS sandbox or distributed lock.

Standard role implementation

agent_standards.py supplies explicit setup, exact-case contained paths, strict JSON reads, stable fingerprints, and common scope dispatch. agent_x.py owns append-by-command events and due-item bookkeeping. agent_y.py owns bounded external CLI job records and verified Git branch/draft-PR handoffs; it needs the user's installed provider, Git, GitHub access, and explicit task authority to perform external actions. agent_z.py owns the fixed policy, review requests, deterministic scoring arithmetic, immutable scorecards, and comparable deltas. The record tools use the Python standard library; Agent Y invokes external installed tools only for a requested job. Agent judgment and original proof remain external inputs; arithmetic and hashes do not establish their truth.

roles-init preserves the existing registry and creates missing records only. The release's seed registry remains Agent 0 only for older bootstrap-manager compatibility. Activating named X/Y/Z roles is an explicit record-format adoption; older selected tools cannot interpret those new role IDs. Shared updates keep project-owned question policies and past reports unchanged.

Agent Y's portable AGENTS/agent.y/JOBS.json belongs to the selected root or child scope. Each job references one existing task, chosen provider, repository identity and observed base, allowed paths, task branch, result, and optional PR identity. Local absolute repo/prompt/worktree paths and configured executable arguments belong in the ignored, owner-only state/agent-y/Y-<id>.json sidecar. This local state is needed to run or resume on that host; do not publish it or assume that copying the portable job alone transfers an executable channel. It is a coordination/evidence record, not a second task ledger or an authentication token. Preparing a job does not start a CLI. Running it must verify the actual provider and checkout; draft-PR creation is opt-in after a successful run. Ready, cancelled-close, and cleanup after merged/accepted state recheck the canonical task and remote PR state. Cancelled unmerged work is retained. There is no Agent Y merge command or background PR watcher. See Agent Y for the workflow and authority gates.

Toolchain and shared context

Read the toolchain map for the boundary between model/harness, records, app tools, verification, and delivery. The shared workspace guide covers cloud snapshot requirements, multi-chat writer ownership, and bot startup context. These are integration patterns over the current CLI/library, not new cloud services.