Operations and Troubleshooting

Operations and troubleshooting

Before a release

Establish the intended repository, source revision, local changes, environment, account, release owner, and artifact. Run the relevant checks and inspect the actual public output. Exclude .SYSTEMX, private records, environment files, and credentials from application deployment bundles.

bash .SYSTEMX/SYSTEMX.sh deploy --dry-run

The plan lists configured checks, build, and deployment commands. Execute deploy only when the project-specific target and release authority are settled. The runner stops on a failed command, but it does not automatically commit, push, select an account, provision services, or verify a live release.

After deployment, check the published artifact/release identifier and the relevant user journey in the target environment. Keep local test success, Git publication, provider deployment, and production acceptance distinct.

Common messages

Message or symptom	Meaning and next action
Python is unavailable	Install or select Python 3.9+ for the CLI, or continue using the documents directly.
No project.json	Run init, then configure actual project commands.
project.json already exists	Initialization preserved the existing file; edit that file deliberately.
No project checks configured	Add real checks before check, build, or deploy.
Executable unavailable	Verify the selected runtime, PATH, executable permission, and project-relative script path.
Unknown task owner	Register the worker before assigning its ID.
Milestone not declared	Add the ID to the master-plan table before using --milestone.
Depends on unfinished work	Complete the prerequisite or correct an inaccurate dependency before starting the task.
Invalid task transition	Follow the transition table in Task workflow.
Generated work page is stale	Review the canonical ledger, run refresh-work, then validate.
Public-template check fails on an adopted project	Use ordinary validate; --template intentionally requires blank records.
Fresh setup reports an existing executable default differs	Inspect the named .py, .sh, or .ps1 file. Preserve custom work and resolve the conflict; setup will not silently trust it.
Managed update says root bootstrap differs	Use the reviewed new external tool to preview bootstrap-refresh; inspect add, replace, and conflicts. Apply only recognized stock changes; only replacements have backups. Then retry. See installation.
Checkpoint pointer no longer exists	Preserve/reconcile the original file, then replace the focus pointer or use focus --clear.
Context output is truncated	Read the named file or task-show record directly.

Recover a coordination lock

Task, registry, and focus writes use .SYSTEMX/state/coordination.lock to serialize CLI writers in one checkout. Writers wait up to one second for a short competing write, then fail instead of overwriting a concurrent update. The lock's owner.json records a process ID, host, creation time, and ownership token.

If a process exits unexpectedly, the lock directory and owner record can remain. Read that record and establish that its writer is no longer running on the named host. A PID alone is insufficient across machines or after process reuse; inspect the actual process and shared-folder state. Only after confirming the lock is stale, remove that lock directory, inspect canonical records, regenerate work views, and validate. SYSTEMX never removes a suspected stale lock automatically. Do not delete task records, agent notes, or unrelated runtime files as recovery.

Atomic replacement protects individual file writes. The ledger and generated views are not one distributed transaction; if interruption leaves a stale view, validate detects it and refresh-work rebuilds it from valid records.

Reconcile a Git merge

Coordinate shared writes through Agent 0. Resolve the canonical task ledger and agent registry and focus first, preserving IDs, owners, dependencies, evidence, and chronological history. Review agent notes and shared-memory changes by their sources.

Then run:

bash .SYSTEMX/SYSTEMX.sh refresh-work
bash .SYSTEMX/SYSTEMX.sh validate

The local coordination lock does not serialize separate machines or independent Git worktrees. Use normal review and merge discipline for those environments.

Upgrade an adopted project

Compare the upstream template version and changelog with the project's copy. Merge the relevant standards and tooling while preserving project.json, the master plan, task history, agent registry, memory, and private runtime data. Do not replace an active .SYSTEMX wholesale with a blank template. Follow Upgrading for the 1.1.0 migration and adoption of older project-specific layouts. No automatic importer renumbers legacy tasks.

Incidents and sensitive information

Use the project's own incident owner, escalation route, and recovery plan. Preserve sanitized evidence, contain the issue, verify recovery, and record the follow-up work. Rotate exposed secrets; removing a file alone does not revoke a credential. Keep private reports and customer data out of public issues and memory records.

For more detail, read the checked-in operations guide and recovery playbook.

Managed version troubleshooting

Use systemx status --target ... to distinguish the active defaults from the preserved outer VERSION file. An older root launcher may still run its old runner after adoption; use systemx run --target ... -- ... to select managed defaults. The current manager may run an already selected, verified release older than 1.8.7-alpha.1 through an isolated compatibility shim. It cannot newly select that historical release; recovery below the floor needs a separately reviewed backup or legacy path. For pins, offline operation, partial snapshots, and stale update locks, follow Installation and updates. Never delete a local project directory or reset its records to make an update pass.

Duplicate or differently cased folders

Use systemx alias --target PATH or .SYSTEMX/SYSTEMX.sh paths to inspect the layout. The real folder must be .SYSTEMX; independent .systemx or mixed-case paths are refused without automatic merge or deletion. Preserve every existing record and follow exact-case recovery before retrying.

Installation history and clean removal

Use systemx audit --target PATH to inspect the selected project's footprint and local operation receipts. A folder that is still installed has clean: false; that flag describes presence, not an error. Uninstall previews by default, then archives the whole .SYSTEMX folder with a verified inventory when explicitly applied. Keep the external receipt and follow cleanup and recovery for interrupted moves, restore, package removal, and intentional leftovers.

Shared-source and bot recovery

A cloud URL that cannot be read is not connected project memory. A successful local write is not proof that another computer received it. Inspect source identity, revision, permissions, sync state, conflicts, and the actual job handle before retrying. Follow shared workspace recovery. Keep canonical writer ownership explicit across machines and worktrees.