Troubleshooting

Troubleshooting

squarepeg raises a specific error type for every failure condition it detects, each carrying a fixed process exit code — see CLI Reference → Exit codes for the summary. This page covers what triggers each one and what to do about it.

Usage errors (exit code 2)

Situation	Message tells you
Unsupported docker flag (--network, --gpus, -p, ...)	The flag name, why it can't work on a remote cluster, and the config-passthrough equivalent if one exists — see the full table.
Malformed -e/-v/quantity value	The offending token and the accepted forms.
-e KEY (bare, no =) but KEY isn't set in your local shell	Which variable was missing.
-v /host/path:/container/path (host bind mount) without opting in	Points at a config-defined PVC-backed volume, or allow_host_path_mounts: true.
--rm combined with --keep	These are mutually exclusive.
--name isn't a valid RFC1123 label	What the valid format is (lowercase alphanumeric + -, ≤63 chars).
A leading-dash token where the image name should be	Names the token, explains it looks like a flag that landed before the image position.

Config errors (exit code 2)

Situation	Message tells you
--config PATH / a $SQUAREPEG_CONFIG entry doesn't exist	The exact path, and which source named it (--config foo.yaml vs $SQUAREPEG_CONFIG). The default location (~/.config/squarepeg/config.yaml) is the one exception — silently skipped if absent, since it's never explicitly requested.
Malformed YAML	The file path and line number.
Unknown top-level config key (e.g. namesapce:)	The key name and the exact file it came from.
Unknown --profile NAME	Every profile name found across all merged config layers.
${VAR} reference to an unset environment variable	The variable name, the exact key path (e.g. kubernetes.spec.imagePullSecrets[0].name), the source file, and a reminder that ${VAR:-default} exists — see Environment Variable Interpolation.
Malformed ${...} syntax (${}, ${1BAD}, unterminated ${VAR)	The malformed token itself.
A config value squarepeg needs as an int/bool doesn't coerce (e.g. timeout: "soon")	The key name and the value it couldn't parse.
volumes.NAME.mount_path isn't an absolute path	The volume name, the source file, and the value it got instead.
volumes.NAME.read_only doesn't coerce to a bool	Same as the general int/bool coercion case above, scoped to that volume's key.

Runner errors (exit code 125)

These mean squarepeg itself couldn't get the container to a known-good completed state — as distinct from the container running and exiting with its own non-zero code, which is reported verbatim instead.

Situation	What happens
No kubeconfig found, and not running in-cluster either	Both failures are named in the message — the kubeconfig error and the in-cluster attempt's error — since squarepeg tries in-cluster config as a fallback when no --context was given (see Execution Lifecycle → Connect to the cluster).
Unknown --context NAME	Names the context that wasn't found in your kubeconfig. This does not fall back to in-cluster config — an explicit --context failure is always surfaced directly.
Image pull failure (ImagePullBackOff, ErrImagePull, InvalidImageName, ...)	Fails fast — doesn't wait out the full --timeout. Names the specific reason and message from the container's waiting state.
Container never starts within --timeout	Reports the last known phase, plus any relevant Kubernetes events (scheduling failures, unbound volumes, etc.), then cleans up (unless --keep).
Container terminates but its exit code can't be read (evicted, deleted mid-run)	Falls back to 125, matching docker run's own convention for "the runner failed" rather than fabricating a container exit code.

Kubernetes API errors (also exit 125, via ApiError)

squarepeg always tries to surface the apiserver's own Status.message (from the response body) rather than just the bare HTTP reason phrase — so instead of a bare Bad Request, you should see the specific field the apiserver rejected, e.g. Pod "x" is invalid: spec.nodeSelector: Invalid value: .... This is the single most useful thing in the error, since it usually names the exact malformed field.

HTTP status	Hint squarepeg adds
400	"the apiserver rejected the manifest as malformed; run with --dry-run to inspect it" — commonly a field with the wrong shape in kubernetes/job passthrough config (e.g. nodeSelector written as a bare key=value string instead of a mapping, or annotations written as a list instead of a mapping)
401	"authentication failed; try kubectl get pods to check your credentials"
403	"permission denied; check with kubectl auth can-i create pods"
404	"not found; check the namespace exists"
409	"already exists; omit --name, or clean up a --keep'd previous run"
422	"the apiserver rejected the manifest; run with --dry-run to inspect it" — the most common symptom of a typo in kubernetes/job passthrough config, since that section is otherwise unvalidated by squarepeg

Interrupt (exit code 130)

Ctrl+C (or SIGTERM) during a run. See Execution Lifecycle → Interrupt handling for the two-stage behaviour — the first press cleans up and exits 130; a second press abandons the resource running instead.

General debugging tips

Symptom	Try
"Why is my namespace/resource-size/whatever what it is?"	squarepeg config show — prints every config file that was loaded, in merge order, plus the fully resolved (post-interpolation) result.
"Is my config producing the manifest I expect?"	squarepeg run --dry-run ... — renders the exact manifest without touching the cluster at all.
A run seems to hang with no output	Check stderr — squarepeg's own progress/event messages go there, separately from container stdout. If genuinely stuck past --timeout, it will time out and report the last known phase.
A container that reads stdin hangs after -t	This is the documented tty implying stdin hazard — see Docker Compatibility → Interactive mode. Real stdin forwarding isn't implemented; avoid -t for containers that block on input.
"Why is this volume mounted — I never passed -v?"	A config layer declared it as an auto-mount (volumes.NAME.mount_path) — see Configuration → Volumes. squarepeg config show will show you exactly which volumes.NAME entries have a mount_path set, and which config file set it.