Task Execution EN

Execute a public task and connect its evidence

한국어

riido-taskrun is a Go executor that gives one small public coding task to Codex CLI and connects that invocation to its files, usage, process termination, and independent verification. It does not automatically intercept ordinary Codex usage. Execution requires --execute. It is currently a macOS development tool for maintainers checking the measurement procedure.

Why an owned executor is needed

A usage trace does not establish that the code is correct. A tested candidate does not establish which invocation produced it. The existing outcome tools provide usage parsing and file verification separately. The new executor connects a process it starts directly to the resulting evidence.

It copies a fixed public task and base files into a fresh workspace, then makes one invocation with an explicit model and reasoning request. After terminating and waiting for the process group, it retains the candidate and checks the requirements independently of the model's claims and authored test assertions. It records unsuccessful attempts too; a savings claim cannot select only a successful first attempt.

This evidence covers one owned CLI invocation. It does not establish the provider's actual model identity, the number of internal model requests, or the absence of other attempts.

Current development pilot and preserved history

Actual external comparison55 executed the existing humanize/UUID logical requests once per Sol6/Luna low request profile. Three candidates passed and one failed; all four executions bind zero process exit, whole usage, budget receipts and auth cleanup. Actual cumulative scope is 20 records,7 distinct requests,5 attempted families and3 repositories. Versions, repeated executions and tests add no distinct requests, training/protected final eligibility or demonstrated savings.

Precommitted plan 50 stopped before coding-model launch when the packaged executor could not resolve its Go toolchain. Those results are retained; modified code does not overwrite that plan with successful outcomes.

Plan 51 stopped after two attempts on one comment task. Its results preserve the exact-contract rejection, requested-profile support failure, and startup-parser compatibility gap.

The repaired tools were used under a separate plan 52, frozen first at commit d202ff042106ad36209ef4cdee6ca1e1c9e64514. Three existing public development tasks requested gpt-6-sol and gpt-6-luna, both with low. All six planned Codex CLI attempts were recorded, and all six candidates passed independent checks within their declared closures. Five attempts exited zero with complete core usage. The Sol attempt on catalog-min-context reached the 120-second deadline and has unknown whole usage. Candidate acceptance and completed execution must not be combined into six successful runs. Read results 52 for per-attempt evidence.

Plan 52 ran one attempt at a time, with a 120-second main-attempt deadline and a separate 45-second verification deadline. Completed plans 52/53/54 and stopped plans 50/51 are not rerun. A new actual run needs a separate plan with verified current profiles and tools.

The six attempts in 52 or the four in each of 53 and 54 do not count internal LLM calls. One CLI invocation may contain multiple model requests and provider retries; those counts are currently unknown. The executor performs no retries, resume, or fallback. A started failure or timeout counts toward the pilot's limit. Maintainers still manage the complete count, order and unexecuted slots for preserved v1 plans. New external v2 tasks use a shared parent ledger to enforce four ordered reservations across two child plans. This applies to one fixed ledger, not every process on the host, the account or internal provider calls.

Task ID	Requirement checked independently	Declared file scope
comment-preview-authority	Make the exact comment change stating that a repository suggestion does not authorize execution	One file, pkg/reporouter/router.go; exact change and gofmt
comment-budget-period	Replace the budget-period comment with the specified sentence	Three pinned files; exact change, gofmt, and pinned tests
catalog-min-context	Add optional MinContext, reject negative values, apply minimum-context eligibility, and preserve existing behavior	Nine pinned files; boundary/JSON contracts and pinned catalog/planner tests

The shared public base for those three tasks is revision 6b2c1bdfd38bea5a9a2c11433ffcfcb94c4143a3, represented by internal/taskverify/testdata/base. The executor checks hashes of its files, LICENSE and NOTICE, task specifications, and independent contract tests. Added files and runtime caches outside the declared scope are unassessed. accepted establishes compliance within that task scope, not approval of the entire repository or arbitrary code.

The behavioral task repo-keyword-language-guard uses the separately versioned base at 146b02b9c37e6d90a11386050ccfdffc036c113e, stored in internal/taskverify/testdata/repo-keyword-language-guard-v1. Its contract checks every keyword of selected repository candidates before Judge invocation. The three-file source scope is go.mod, pkg/reporouter/router.go and pkg/reporouter/router_test.go, with LICENSE/NOTICE pinned separately. The original three task specifications and hashes remain unchanged.

Experiment 53 ran this one request twice per Luna/Sol low profile: four CLI attempts. All four candidates passed independent closure checks with zero exit and complete whole core usage. Four repetitions are not four distinct requests. All four produced the same mutable-source change; the first completed accepted candidate's keyword guard was integrated into runtime and checked with the pinned independent contract. Candidate-authored tests were not adopted as acceptance evidence. Authored reference/mutant verification remains separate from model outcomes.

The historical 51/52 subtotal is eight records over three requests; through 53, there were 12 records, four requests and two actually attempted families. Preserve 52's six accepted candidates, 53's four and 51's rejection/support error.

Experiment 54, under pre-outcome plan 54, ran one taskoutcome-event-key-bounds request twice per Sol6/Luna low profile. All four attempts passed 75 independent terminal tests, exited zero and had complete whole core usage. The tests and repetitions do not add distinct requests. The current cumulative scope through 54 is historically 16 owned records, five distinct requests, three actually attempted code families and one repository.

The local registry remains eight candidates, three families and one repository. The parser's separate revision ee72334166e2962b0821d5198a50fdd13f92ab29 binds its specification, three public source files, LICENSE/NOTICE and independent checks. The contract covers duplicate event-envelope keys, the inclusive 64-key count and 128 decoded UTF-8 byte boundary. Candidate-written tests were not acceptance evidence. The first accepted zero-exit candidate's fixed-array implementation was integrated into runtime. Frozen specifications and the 51–53 records remain unchanged.

The public Go inventory is the preserved historical snapshot at 53: 120 candidates, eight repositories and zero execution-eligible tasks. At 54, independent contracts for two of those external requests were prepared and checked locally, with zero external model attempts, training labels or final-eligible requests. There are seven compiled verifier tasks across three repositories: five actually attempted local tasks and two external contracts. This is a separate scope from the eight-candidate local registry. Contract preparation and authored controls do not add attempts, answers or 2,400 final requests.

55 adds v2 evaluation versions linked to the same two external logical requests. All conditions are exposed in actual stdin, and actor and independent verifier use matching original module language settings. Registry IDs increase by two; distinct verifier tasks remain seven. The subsequent comparison recorded4 actual attempts on those2 existing external requests; training/final eligibility remains zero. Actual results55 and fair comparison conditions explains original language versus the Go 1.27.1 executable, unsupported source shapes and the scope of license checks.

v2 execution requires --parent-file, --parent-sha256, --budget-dir and --global-ordinal alongside existing arguments. A slot is permanently consumed before model launch, including failures. The next slot opens only after checking start/final files and cleanup; unknown launch or completion states stop the ledger. Keep it outside every attempt directory. Plans and hashes must be frozen first; this preparation document is not a precommitted execution plan.

riido-taskrun --budget-status \
  --parent-file /absolute/public/parent.json \
  --parent-sha256 PARENT_SHA256 \
  --budget-dir /absolute/private/shared-ledger

Inspection only reads an existing ledger; it launches no model and performs no reservation or repair. reservations, durable_start_markers, terminal_receipts and launch_state_unknown are separate counts. After a main start, unavailable ledger finalization produces diagnostic parent_budget_finalization_failed and exit 3. Model-attempt JSON is immutable and written first; a separate terminal receipt binds its bytes. An accepted JSON record alone therefore does not prove completion of the parent budget.

What Laya predicted before coding outcomes

The recorded predictions used the unchanged pinned Laya INT8 checkpoint on CPU with one thread and the default threshold of 0.9. No threshold adjustment or training on these three tasks took place. Plan 52 references these unchanged predictions; it does not count them as new predictions or three new independent tasks.

Task	Author's difficulty hypothesis	Laya suggestion	Largest probability	Applied tier
comment-preview-authority	fast	standard	0.480	strong, abstained
comment-budget-period	fast	fast	0.486	strong, abstained
catalog-min-context	standard	standard	0.591	strong, abstained

All three inputs were untruncated but abstained because their confidence was below 0.9. The useful question is which requested coding profile actually meets the requirements, rather than agreement with an author's expected tier. This pilot runs the fixed luna/sol comparison, not the strong model retained by the router's abstention. gpt-6-astra was not executed. The historical standard mapping requested gpt-6.1-sol; it is not reinterpreted as a prediction for this pilot's gpt-6-sol. These values are not calibrated probabilities of task success.

In the historical observations from 50 reused by 52, maximum process RSS was about 1.48 GB (1.38 GiB) in three cold Laya router processes. Each observation includes loading the model and native session for one request. It is not warm request latency, Go heap usage, or GPU memory. These measurements do not establish that the very small memory target has been met.

Before 53's coding outcomes, a separate prediction was sealed. INT8 base, one CPU thread and a 0.9 threshold suggested standard, but a maximum score of about 0.669 caused abstention. The configured fallback is strong/Astra; no Astra coding attempt occurred. The new standard/Sol6 mapping applies only to this prediction. One cold maximum-RSS observation was about 1.50 GB/1.40 GiB; this is not warm latency, Go heap, GPU memory or a calibrated coding-success probability.

54's pre-coding prediction used the same INT8 base, one CPU thread and threshold 0.9. It suggested standard at 0.610351 and abstained. Strong/Astra fallback is configuration; Astra was not run. One cold whole-process observation took 1,183ms, with maximum RSS 1,508,032,512 bytes (about 1.508 GB/1.40 GiB). Native loading is included; this is not warm latency, Go heap, GPU resources or calibrated success probability. It does not establish the ultra-small-resource target.

Usage

Build the Go binary from the repository root. The executor has no Python runtime requirement.

mkdir -p .cache/bin
go build -trimpath -o .cache/bin/riido-taskrun ./cmd/riido-taskrun
.cache/bin/riido-taskrun --help
.cache/bin/riido-taskrun --task comment-preview-authority --spec
.cache/bin/riido-taskrun --task repo-keyword-language-guard --spec
.cache/bin/riido-taskrun --task taskoutcome-event-key-bounds --spec

--help and --spec launch no model. A default invocation refuses execution. An actual attempt requires the precommitted plan, pinned public base, trusted Codex executable and version, trusted Go installation, and an existing local ChatGPT login. The current supported version is codex-cli 0.158.0; an executable hash that differs from the plan is refused before inference. A changed version or plan requires a new plan first.

A deployment-style -trimpath binary may have no default Go installation path. Pass an explicit trusted Go 1.27.1 installation with --go-root. The executor checks the resolved bin/go, version, and executable-byte SHA-256 against the new plan's pins. Without a usable default or explicit root, it refuses before inference with trusted_toolchain_unavailable. It does not follow an arbitrary host PATH or download another toolchain.

The following example shows the argument form for a new precommitted plan. NEW_*, VERIFIED_*, PLANNED_ORDINAL, and /absolute/... are placeholders, not an executable frozen plan. Match the specification and base to the selected task, verify the actual hashes, and freeze the new plan first. The example's gpt-6-luna request must match an entry in that new plan too. The private output directory must not exist yet. An actual invocation may consume account usage. The output must be beneath an OS temporary directory. A repository or AGENTS.md, .codex, or .agents in its ancestors causes prelaunch refusal.

.cache/bin/riido-taskrun \
  --execute \
  --task comment-preview-authority \
  --model gpt-6-luna \
  --reasoning low \
  --base-dir /absolute/laya-tools/internal/taskverify/testdata/base \
  --private-dir /private/tmp/riido-NEW-ATTEMPT \
  --codex-bin /absolute/trusted/codex \
  --codex-sha256 VERIFIED_CODEX_SHA256 \
  --codex-version 'codex-cli 0.158.0' \
  --go-root /absolute/trusted/go-toolchain \
  --plan-file /absolute/private/NEW-FROZEN-PLAN.json \
  --plan-sha256 NEW_PLAN_SHA256 \
  --attempt-ordinal PLANNED_ORDINAL \
  --task-spec-sha256 VERIFIED_TASK_SPEC_SHA256 \
  --auth-source-dir /absolute/private/codex-login \
  --timeout 120s

--auth-source-dir contains an existing ChatGPT auth.json. Do not paste tokens into arguments or chat. API-key authentication is refused, and only the required authentication file is copied. User config, rules, skills, MCP, history, and unrelated inherited environment variables are not carried into the invocation. Copied authentication stays outside the model workspace and is removed on terminal cleanup. The shared background server, apps, hooks, other agents, memories, automatic skill installation and web search are explicitly disabled for this measurement.

The fresh workspace contains only the declared public files. A macOS permissions profile allows workspace writes and necessary tool reads, while denying network access by model-generated commands and access to sibling directories. A real pre-inference probe must demonstrate the enforcement; otherwise no model attempt starts. These command permissions are separate from the parent Codex CLI's communication with its official service. The ordinary checkout is not modified. Commands use the verified Go installation and a controlled environment with toolchain/module downloads and CGO disabled. The same installation is passed to independent verification.

Raw stdout JSONL, stderr, candidate files, and record.json stay in the local private directory. The executor does not upload them to GitHub or Hugging Face. Public records should contain only reviewed numeric values, enums, and hashes for these public tasks. Never commit raw traces, authentication, or local absolute paths.

Read the result as a person or agent

Field	Meaning
provenance	executor_owned_single_attempt identifies one invocation started and collected by this executor
applied_request	The explicit request passed to the child CLI; it does not attest to the provider's actual model
observed_model	Currently unknown; do not populate it from the requested model
process_status, exit_code	CLI termination evidence, separate from code correctness
go_version, go_binary_sha256	Version and executable hash of the local Go toolchain checked against the plan before inference
verification_status	Independent result such as accepted, rejected, or verifier_unknown
usage_summary	Validly parsed usage, retaining partial observations rather than replacing unknown totals with zero
whole_attempt_usage_complete	Whether successful termination and process-group cleanup, complete stdout/stderr collection, and core usage for every turn are all present

A candidate may meet the requirements while its usage remains unknown. That is not an implementation-failure label. Conversely, exit code zero and complete usage do not establish task success when the independent checks reject the code.

Input, cached input, and output are reported separately. Cached input is a subset of input, and reasoning output is a subset of output; neither is added a second time. Tokens are not converted into money, subscription allowance usage, or remaining allowance. The recorded wall time measures the local CLI process, not the separate verification time or remote model GPU usage.

Stdout is limited to 64 MiB and stderr to 1 MiB. Deadline, capture, parsing, and verification-environment failures remain visible. For example, timeout_or_canceled and capture_limit describe execution states; unsupported_candidate_shape describes the narrow verifier's unsupported scope. These cannot all be labeled as model capability failures.

riido-taskrun exit code	Meaning
0	The CLI exited zero, independent checks accepted the declared closure, and copied authentication was cleaned up; usage may still be incomplete
1	A started unsuccessful attempt was recorded; read the JSON process and verification states to distinguish the cause
2	Prelaunch refusal for arguments, pins, authentication, isolation, or related conditions, or an output error
3	An attempt was recorded but acceptance, candidate collection, record persistence, or related evidence is unavailable, or authentication cleanup failed
0 from --help or --spec	Information output succeeded; no model invocation or candidate acceptance occurred

Agents should not create training labels from exit codes alone. Read process, verification, and usage evidence separately. Retain prelaunch refusals and unexecuted entries for the plan rather than replacing them with successful runs. Validating local plan bytes and ordered attempts does not independently attest that the plan was published before outcomes.

What this establishes and what comes next

The command examples explain execution; they do not themselves create model outcomes. Experiment 52 separately records actual executions under its frozen plan. Both profiles have acceptance and complete usage on the two comment requests, but the Sol behavioral attempt has unknown whole usage, preventing a complete cost comparison for that request. Only three requests had been attempted by 52, so that comparison does not establish profile savings or useful production routing. No new training, model-weight release, production routing activation, or final 2,400-case scoring took place.

The stopped records for experiment 50 and experiment 51 remain preserved. A new actual run needs verified available profiles, a separate frozen plan and fresh output directories.

The small development pilot first checks that success, failure, and unknown evidence can be recorded correctly. Later, distinct public tasks and paired profile outcomes can support comparisons of success and usage across every attempt under the same conditions. Only then can upward routing, downward routing, and abstention be judged. Read task usage and independent verification for interpretation and the golden-set design for required scale and split separation. The acquisition plan describes staged collection and execution budgets.

Historical results 53 cover four repetitions of one behavior request. Input 339,805 includes cached 280,576; output 4,890 includes reasoning 76. Luna had less total input, but uncached input was Luna 31,425 versus Sol 27,804. Do not mix those observations with 54 or infer money saved.

Historical results 54 cover four repetitions of one parser request: input 485,609 includes cached 423,552, and output 5,282 includes reasoning 157. Luna had smaller input/output and observed main time in both repetitions of this request, while actual served identity, subscription consumption, money and provider internal-call counts remain unknown. The five cumulative requests through54 do not validate general capability or production routing. Keyword and parser protections were integrated into runtime, with no new fitting, calibration, weight release, Codex policy activation or protected-final scoring.

Subsequent results55 first executed the two existing external requests, extending current observations to seven requests and20 records. These are not new final or training labels; retain the single observation per profile and the operator-pause limitations.