Skip to content

Devin: one optional setup and readiness path for local CLI and hosted v3 - #940

Merged
jeffhuber merged 15 commits into
mainfrom
devin/910-setup-doctor
Sep 12, 2026
Merged

Devin: one optional setup and readiness path for local CLI and hosted v3#940
jeffhuber merged 15 commits into
mainfrom
devin/910-setup-doctor

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Selecting Devin was discoverable, but readiness, permissions, capability modes, and lifecycle remediation were spread across separate surfaces. devin_readiness.py becomes the one place that answers "is the selected Devin posture ready, and what do I run next", and doctor, the session brief, next steps, and the docs read from it. Claude + Codex stay DEFAULT_PARTICIPANTS; a repository that never selected Devin produces no Devin findings, and the new doctor stage stays out of its plan entirely.
  • The two authentications are reported as distinct and non-substitutable: devin_cli is authenticated only by the ambient Devin Desktop/CLI login, devin_api_v3 only by dedicated service-user credentials plus the exact OWNER/REPO acknowledged locally (hosted has no read-only preflight proving GitHub connection scope, so acknowledgement is the gate). Create/view/manage requirements are reported as an owner action with status skip rather than warned about on every repository, because no read-only permission probe exists.
  • Capability modes name the execution mode a transport uses, not maturity, and now match the lifecycle already shipped on main. Hosted devin_api_v3 declares message, cancel, and structured_results as remote_session, because code-mower session message|cancel|collect and the Devin provider really perform those operations; coordinate stays unavailable and review stays evidence_only, so selection still confers no review or merge authority. devin_cli keeps its real gaps (message/cancel unavailable, structured_results local-runner) and never claims remote lifecycle. Every closed copy moves together — the registry, provider_capabilities.schema.json, templates/providers.yml, src/code_mower/templates/providers.yml, templates/providers/devin.yml — and LEGACY_CAPABILITIES accepts both earlier hosted declarations so an existing config still normalizes in memory without rewriting the file. A cross-contract test ties each hosted remote_session mode to the actual status/message/cancel/collect operation behind it.
  • Findings are metadata only: no credential values, service-user identity, organization identifier, configured repository inventory, local path, credential profile filename, or raw provider output. The local check reports the executable basename; hosted credential readiness renders only path-free, identity-free source/status metadata and locally written remediation that names the required environment variables or "a protected credential profile" without naming its location. A variable is named as unresolved only when it really is one of the required variables, so an ambiguous credential profile now answers with the next action — rerun the same pinned configuration and --profile with --provider-profile NAME — instead of rendering the resolver's status where a variable name belongs.
  • Every user-facing readiness command pins the posture it describes. Generated guidance (next steps, setup_instructions) emits the caller's configuration path and explicit --profile, shell-quoted, including recommended; a guidance surface without those inputs says to rerun with the same configuration and profile instead of an unscoped command that could read another profile's Devin lane. Static doc examples use code-mower doctor --profile recommended --devin --repo OWNER/REPO --json and explain replacing recommended. The Code Mower --profile stays distinct from the credential --provider-profile. That pinning now covers every actionable remediation, not only generated guidance: each transport-switch, init, and rerun command in a finding is built from the configuration path and configuration profile that produced it, quoted with shlex.quote() and always carrying an explicit --profile (including recommended), so an operator inspecting ops/custom mower.yml --profile 'custom profile' is never redirected to the default starter or the recommended profile. Public detail and cloud metadata stay path-free; only the locally rendered remediation repeats the caller's own path.
  • Guidance is derived from the maintained declarations rather than restated: the generic lane capability remediation names the modes the transport actually supports and only the genuinely unavailable gaps, so hosted no longer claims messaging and cancellation are unavailable while its own detail reports remote_session, and local CLI still names its real gaps without implying remote lifecycle. Local readiness detail and remediation likewise describe only the effective lane's own candidates (as basenames) and its actual command_env selector, so an operator is never sent to a variable the lane's runtime ignores; the historical override and default devin answer only when no lane configuration was supplied.
  • Devin identity comes from a lane's declaration, not its identifier. A valid custom-named lane such as team_devin declaring product: devin with a canonical provider/transport/driver activates the stage, selects its transport, supplies its own command configuration to readiness, and gets the appended readiness next step, while canonical aliases and fail-closed mixed-transport ambiguity are unchanged. That identity now reaches the findings themselves: devin_effective_lane() returns the selected lane's ID with its effective configuration, and check_devin_readiness() passes it to devin_readiness(), so team_devin findings report lane=team_devin instead of a canonical lane the repository does not have, two custom lanes on one transport stay distinguishable, and an invocation whose lanes span both transports with no explicit transport fails closed through the same bounded provider.devin.selection finding as an ambiguous transport.
  • A profile may validly select several Devin lanes on one transport (team_devin and night_devin both local), so readiness resolves every effective lane the selected transport matches instead of rejecting the configuration. The answers that belong to the product — selection, capabilities, permissions, lifecycle, and hosted credentials/repository scope — are emitted once with the sharing lanes named in detail rather than attributed to one arbitrary lane, and every executable readiness finding is emitted per lane against that lane's own ID and effective command configuration. A single selected lane keeps every finding bound to it as before, and lanes spanning both transports with no explicit transport still fail closed.
  • No generated transport switch narrows unrelated configuration. code-mower init --set-transport devin=devin_cli|devin_api_v3 replaces only Devin's transport, its own profile lanes, and its own participant alias, keeping unrelated participants (Cursor, Gitar, a repository's own custom lane) and every other profile lane as configured; and remediation still carries the exact quoted configuration path and --profile. Because the saved selection (session_defaults.transports.devin plus the participant alias) is repository-wide, the switch retargets the canonical Devin lane of every profile that selects Devin rather than leaving an untouched profile whose declared lane the saved selection contradicts; profiles that do not select Devin are structurally unchanged, and a custom-named Devin lane in any profile is reported instead of rewritten. The --set-transport help, docs, and remediation state that scope.
  • Transport-switch remediation describes the real init lifecycle: a pinned --dry-run preview, a pinned --apply --output-dir .code-mower.generated that stages a reviewable generated tree without touching the configuration it read, a review-and-install step through the normal setup PR, and only then the pinned code-mower doctor ... --devin rerun. Staging never claims the active posture switched.
  • A profile whose Devin lanes are custom-named gets bounded manual guidance naming those lane IDs and every setting a retarget depends on: the public product/provider/transport/driver fields, the transport-dependent capabilities block (dropped for the maintained defaults or restated for the new transport), provider_config.campaign_transport, the saved session_defaults.transports.devin, and the transport-specific participant alias — then the pinned doctor rerun to revalidate. Editing the four declaration fields alone could leave a lane copied from the maintained configuration invalid or still selecting the old transport. That saved selection is repository-wide, so the guidance states the coherent contract rather than aligning one profile in isolation: inspect every profile that selects Devin and retarget its named lanes the same way before saving a transport, or — when profiles intentionally keep different Devin transports — save no session_defaults.transports.devin and no transport-specific alias at all, name the product as devin, and let each profile's lane declaration select its own transport. Profiles are described, never enumerated, so nothing beyond the lane IDs already carried by the finding is echoed. It never emits --interactive, --with, or any command that could rebuild or narrow the profile: the participant picker selects products and canonical reviewer lanes, so it would drop the lanes it cannot name.
  • Observer postures return the local-CLI skip before any executable lookup, so "availability was not inspected" is literally true.
  • Local CLI readiness resolves the effective selected lane's command discovery (provider_config.command_env, command, alternate_commands, then the provider fallback) through a neutral local_cli_commands module shared with lane runtime, so readiness cannot contradict what runtime would execute, and it reads the injected env mapping rather than ambient os.environ. A configured lane's candidates are the whole candidate set — the historical CODE_MOWER_DEVIN_CLI_COMMAND override and the default devin answer only for a caller with no lane configuration, so a configured command that is not installed cannot pass readiness while runtime reports the lane unavailable. When both Devin lanes are active, the lane handed to readiness is the one whose transport was selected (TRANSPORTS[transport].review_lane), not whichever lane comes first. Observer postures (--orchestrator-only, --hosted-builders) skip the local executable requirement in the same modes check_lane_runtime skips it.
  • Transport selection fails closed: when the selected lane set contains both Devin transports and neither profile scoping nor an explicit session_defaults.transports.devin disambiguates them, it raises a bounded ConfigError naming the way out instead of taking first lane order. devin_readiness() forwards the caller's configuration profile to that selection rather than falling back to recommended. An unresolvable selection is itself a readiness answer, so planning keeps the stage and the report carries a failing provider.devin.selection check with that remediation instead of omitting Devin readiness.
  • Stage planning happens after load_inputs, so the stage activates for a repo that selected Devin (by participant or by an active Devin review lane), and --devin additionally prints the local CLI, hosted API, and unavailable postures with next actions before selection:
run_doctor(...)                       # devin=False, repo selected devin_api_v3
  -> devin-readiness stage active
  -> provider.devin.selection         pass
     provider.devin.capabilities      pass   (coordinate unavailable; review evidence-only)
     provider.devin.hosted_credentials warn  (first unresolved variable named, no values or paths)
     provider.devin.repository_scope  warn   (exact slug only, inventory not printed)
     provider.devin.permissions       skip   (owner action)
     provider.devin.lifecycle         pass   (status, message, cancel, collect; never redispatch)
  • session.py appends the posture's authentication and permission line to host instructions only when Devin is selected, so the default brief is unchanged. next_steps.py appends (not inserts) the readiness step, preserving the existing positional reviewer-step semantics. The new modules are registered in the package manifest.
  • doctor_checks/devin.py imports code_mower.devin_readiness per call: at module scope it closes a cycle through devin_apicampaign_adaptersrelease_qualifydoctor_checks/__init__.

Closes #910. Part of #901.

Validation

  • .venv/bin/python -m unittest discover -s tests — 3063 tests, OK (skipped=12)
  • Focused suites: tests.test_devin_capabilities, tests.test_devin_setup_readiness, test_devin_sessions, test_participant_sessions, test_session_lease, test_remote_session, test_devin_work_orders, test_devin_review, test_devin_provider_contract, test_devin_api, test_devin_builder_lane, and every test_doctor* module — all OK
  • Coverage includes hosted/local capability parity across the registry, schema and all provider templates, in-memory migration of both earlier hosted declarations, hosted remote_session modes tied to real status/message/cancel/collect operations, readiness capability and lifecycle rendering, profile-pinned and safely quoted generated commands (default and non-default profiles, custom config paths), guidance without profile inputs, multi-profile local-vs-hosted selection, ambiguity reported by ordinary doctor and by --devin, lane candidates matching resolved_local_cli_command() for an uninstalled configured command, effective-lane selection under both lane orders, injected-env command discovery with configured/custom-env/alternate commands, basename-only reporting, observer-posture parity plus normal local execution, non-default configuration-profile transport selection, credential path/identity privacy, hosted-versus-local generic capability remediation derived from declared modes, a configured lane whose candidates and custom command_env are the only ones named, the no-lane-configuration historical fallback, custom-named local and hosted Devin lanes through doctor and build_next_steps, ambiguous credential profiles naming no variable/profile/filename/path, observer postures performing zero executable lookups, configuration-pinned remediation for local selection, hosted selection and credentials, an unavailable local CLI, transport switching, the observer rerun and unselected guidance under a spaced configuration path with a non-default profile, an explicit recommended profile, custom local and hosted lane identity on every finding, an unidentifiable effective lane failing closed, one custom local lane and one custom hosted lane, two same-transport custom local lanes with different commands reported per lane, two same-transport hosted lanes with credentials and repository scope emitted once, unresolved mixed transports failing closed, an end-to-end init mutation through the real config-writing path with Claude, Codex, Cursor, Gitar, Devin and an unrelated custom lane asserting only Devin's transport and lane move, the default Claude + Codex pair and a configuration with no saved participant list, custom-named local and hosted lanes (single and multiple) receiving manual declaration guidance with no picker/selection command, the rendered switch commands followed end to end so the preview mutates nothing, the apply writes only the review tree, and readiness still reports the installed local posture until the generated configuration is installed, the manual guidance followed end to end from full maintained local and hosted declarations with explicit session defaults for one and for two same-transport custom lanes proving validate_config() passes and pinned readiness reports the target transport, a two-profile switch checking each profile's effective selected transport with a Devin-free profile untouched, a custom-named Devin lane in a second profile failing closed, a multi-profile custom-lane configuration following both routes the guidance offers — every Devin profile aligned together, or the transports kept apart with no saved transport or transport-specific alias — proving no profile is left reporting a transport it does not declare, safe quoting, and an unchanged fresh no-Devin installation
  • .venv/bin/python scripts/privacy_scan.py, .venv/bin/python -m ruff check ., .venv/bin/python -m compileall -q src scripts, .venv/bin/python scripts/smoke_easy_mode.py --code-mower-bin .venv/bin/code-mower --json, and the package/release-hygiene checks — all pass

Risk / Rollback

  • Mostly additive: one new readiness module, one shared command-discovery module, one optional doctor stage, one new CLI flag, one appended next step, and Devin-only session instructions. The behavioral changes outside that are the corrected hosted capability modes (with legacy declarations still accepted in memory, so no config rewrite is required) and Devin transport selection now failing closed on a genuinely ambiguous lane set instead of silently picking one. No existing check name, status, config schema, or default participant changes. Revert the branch to restore prior behavior.

Data Or Secret Exposure

  • This change does not add source, raw diffs, raw transcripts, raw provider output, tokens, or machine-specific paths to public artifacts.

Link to Devin session: https://app.devin.ai/sessions/d75626971cdc455f8db3b053ac2489fc
Open in Devin Desktop: https://app.devin.ai/desktop/session/d75626971cdc455f8db3b053ac2489fc?variant=devin

Report the selected Devin posture, its authentication, permission
requirements, capability gaps, and lifecycle recovery from a single
readiness module surfaced by doctor, the session brief, next steps, and
docs. Claude + Codex stay the default and a repository without Devin
produces no Devin checks.

Closes #910

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@jeffhuber

Copy link
Copy Markdown
Contributor

Codex audit (merge-authority lane)

Head SHA: 620052f7ca3cdc4ae90e79dcc75e4285a2c4d392
Findings: P0=0, P1=0, P2=2, P3=0 (blocker policy: any P0/P1/P2 → BLOCKED)

Codex Audit: BLOCKED

Summary:

The readiness checks can inspect the wrong executable or report the wrong Devin transport for supported configurations, producing misleading setup guidance.

Findings:

  • [P2] Preserve the configured executable path during discovery -- /private/var/folders/f2/g9zqfdjx7z3ckcsx_f46htwm0000gn/T/codex-audit-b89ambp7/wt/src/code_mower/devin_readiness.py:194
    Finding ID: codex:54fdfa67467a912ad65a
    When CODE_MOWER_DEVIN_CLI_COMMAND points to an executable outside PATH, _cli_command() strips its directory before shutil.which() checks it. Doctor therefore warns that a working CLI is unavailable; conversely, a broken override can pass if another executable with the same basename exists on PATH. Check the full configured command and reduce it to a basename only when formatting output.
  • [P2] Infer the transport from selected lanes without a profile -- /private/var/folders/f2/g9zqfdjx7z3ckcsx_f46htwm0000gn/T/codex-audit-b89ambp7/wt/src/code_mower/devin_readiness.py:118
    Finding ID: codex:32a16dfb4c1be9508aa9
    For a legacy configuration with only the hosted devin lane and no explicit session transport, ordinary doctor --devin passes profile=None. configured_transports() deliberately disables lane inference in that case and returns devin_cli, even though the supplied lanes identify hosted Devin. The new checks consequently report local authentication and omit hosted credential and repository-scope checks. Infer the transport from the supplied lanes when no explicit selection exists.

@jeffhuber

Copy link
Copy Markdown
Contributor

Supplemental exact-head audit finding

Exact reviewed head: 620052f7ca3cdc4ae90e79dcc75e4285a2c4d392.

  • P2 — Isolate readiness tests from ambient credential profiles. test_hosted_posture_requires_credentials_and_exact_repository_scope passes env={} but does not supply an isolated credential file or configuration directory. The shared credential resolver therefore legitimately loads an existing protected default Devin profile. On a configured Devin host, the exact-head focused suite reports 25 passes and this one failure because the test expects warn while readiness correctly observes configured credentials and returns pass. This makes the advertised full-suite result dependent on the operator's private machine state. Give tests an explicitly empty temporary credential store or mock the resolver, and add a regression proving an ambient profile cannot alter the missing-credential fixture. No credential value, identity, profile filename, or machine path was emitted during this reproduction.

Please resolve this together with the two P2 findings in the structured Code Mower Codex verdict before requesting a new exact-head audit.

…ed test credentials

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Re: Codex audit (merge-authority lane) — both P2 findings resolved

New head: 314f1e6901eafe44154d57e2aaaa71f2c534b3d3.

  • codex:54fdfa67467a912ad65a — configured executable path preserved. Discovery now calls shutil.which() with the full CODE_MOWER_DEVIN_CLI_COMMAND value, so an override outside PATH resolves and a broken override no longer passes because a same-basename executable exists on PATH. Reporting still reduces the command to a basename, so no local path leaves the machine. Regression: test_configured_command_override_is_discovered_but_reported_as_a_basename.
  • codex:32a16dfb4c1be9508aa9 — transport inferred from selected lanes. When the repository names no Devin transport (no alias in session_defaults.participants, no session_defaults.transports.devin), an active Devin review lane now selects the matching transport, so a legacy hosted devin lane under doctor --devin reports hosted credentials and exact repository scope instead of local authentication. An explicit selection still wins over lane inference. Regression: test_active_hosted_lane_selects_hosted_without_profile_inference.

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Re: supplemental exact-head audit finding — P2 resolved

New head: 314f1e6901eafe44154d57e2aaaa71f2c534b3d3.

Every readiness fixture now resolves credentials through an isolated empty temporary configuration directory, and the doctor CLI cases pass --provider-config-dir, so a stored default Devin profile on the host machine can no longer turn a missing-credential fixture into a pass. Added test_stored_host_profile_cannot_satisfy_the_missing_credential_fixture: a deliberately supplied discoverable profile makes the check pass, while the isolated fixture still warns.

Local validation on this head: privacy scan, Ruff, compileall, easy-mode smoke, and the full suite (3009 tests, OK, 12 skipped). Ready for a new exact-head audit.

@jeffhuber

Copy link
Copy Markdown
Contributor

Codex audit (merge-authority lane)

Head SHA: 314f1e6901eafe44154d57e2aaaa71f2c534b3d3
Findings: P0=0, P1=0, P2=1, P3=0 (blocker policy: any P0/P1/P2 → BLOCKED)

Codex Audit: BLOCKED

Summary:

The readiness command can inspect a different configuration or profile from the setup plan, producing misleading guidance for custom setups.

Findings:

  • [P2] Preserve the selected config and profile in the readiness command -- /private/var/folders/f2/g9zqfdjx7z3ckcsx_f46htwm0000gn/T/codex-audit-p9sqogfb/wt/src/code_mower/next_steps.py:259
    Finding ID: codex:cd4939bf296798ef0230
    When build_next_steps receives a custom config_path or non-default profile, this command discards both and inspects the default configuration's recommended profile instead. A plan for custom.yml --profile custom-devin preserves those arguments in doctor-easy, but its new Devin check can report Devin as unselected or inspect the wrong transport. Include the quoted configuration path and selected profile in this command.

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Re: Codex audit of 314f1e6 — P2 codex:cd4939bf296798ef0230 resolved

New head: 958ec8bc6fe265094df6e4ea57ad88fef3b5a442 (includes a merge of current origin/main).

The appended devin-readiness step now mirrors the same arguments as doctor-easy: the shlex-quoted configuration path when one was supplied, and an explicit --profile for non-default profiles. A plan generated from custom.yml --profile custom-devin therefore inspects that exact configuration and profile, and the default plan's command and step ordering are unchanged.

Regressions: test_next_steps_devin_check_preserves_the_selected_config_and_profile (custom path plus non-default profile, and the existing doctor-easy arguments) and test_next_steps_devin_check_quotes_unsafe_config_paths (path with spaces stays quoted, no --profile on the default profile).

Validation on this head: Ruff, privacy scan, compileall, easy-mode smoke, package/release-hygiene checks, and the full suite (3016 tests, OK, 12 skipped).

@jeffhuber

Copy link
Copy Markdown
Contributor

Codex audit (merge-authority lane)

Head SHA: 958ec8bc6fe265094df6e4ea57ad88fef3b5a442
Findings: P0=0, P1=0, P2=2, P3=0 (blocker policy: any P0/P1/P2 → BLOCKED)

Codex Audit: BLOCKED

Summary:

Two P2 findings: readiness output exposes credential-profile paths, and the generated command can inspect the wrong Devin transport.

Findings:

  • [P2] Exclude credential profile paths from readiness output -- /private/var/folders/f2/g9zqfdjx7z3ckcsx_f46htwm0000gn/T/codex-audit-_n30k9e2/wt/src/code_mower/devin_readiness.py:276
    Finding ID: codex:926e2d79d4394b4a2219
    When hosted credentials use a profile beneath the user's home directory, resolution.safe_detail() includes its home-relative path and filename; credentials.remediation also repeats that path on failure. This violates the readiness contract that local paths and identities are not reported. Filter profile-path fields and use path-free remediation rather than forwarding the resolver's diagnostics unchanged.
  • [P2] Preserve the recommended profile in the generated command -- /private/var/folders/f2/g9zqfdjx7z3ckcsx_f46htwm0000gn/T/codex-audit-_n30k9e2/wt/src/code_mower/next_steps.py:258
    Finding ID: codex:cf39e0d7363d94a6c4e4
    When the selected profile is recommended, the command omits --profile, but doctor defaults to inspecting all configured lanes. With devin_cli in recommended, a dormant devin lane in another profile, and no explicit session transport, the generated command reports hosted readiness instead of local readiness. Always pass the selected profile, or use --easy for recommended, so the guidance checks the posture selected by build_next_steps.

@jeffhuber

Copy link
Copy Markdown
Contributor

Claude audit (merge-authority lane)

Head SHA: 958ec8bc6fe265094df6e4ea57ad88fef3b5a442
Findings: P0=0, P1=0, P2=0, P3=0 (blocker policy: any P0/P1/P2 -> BLOCKED)

Claude Audit: PASS

Summary:

Devin-authored PR adds an optional, well-isolated Devin setup/readiness doctor stage (devin_readiness.py, doctor_checks/devin.py, registry/runner wiring, session/next-steps guidance) with extensive test coverage. Reviewed control flow, stage gating (default runs unaffected, devin-readiness only enabled when selected or --devin passed), credential/privacy handling (no secret values or repository inventory leaked into findings), and doc consistency. No correctness, security, or data-loss blockers found.

Findings: none.

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Re: Codex audit of 958ec8b — both P2s resolved

New head: 5adb6e7d20468fc89feefb705debf1ef366a90e7.

codex:926e2d79d4394b4a2219 — hosted credential readiness no longer forwards the resolver's diagnostics. The profile path and candidate filenames are dropped from the detail, and remediation is now written locally per outcome (missing, ambiguous, malformed, insecure permissions), naming the required environment variables or "a protected credential profile" without naming its location. A new privacy regression resolves credentials from a store on disk in the resolved, malformed, and missing cases and asserts that no absolute path, temporary-directory path, home-relative prefix, or profile filename appears in any message, detail, or remediation.

codex:cf39e0d7363d94a6c4e4 — the appended readiness command now always carries an explicit, safely quoted --profile for the profile the plan was built from, including recommended, so it inspects exactly the posture build_next_steps selected rather than every configured lane. Regressions updated for the default/recommended profile, a non-default profile, and a config path needing quoting; the step remains appended last and earlier step ids are unchanged.

Validation on this head: Ruff, privacy scan, compileall, easy-mode smoke, package checks, and the full suite (3017 tests, OK, 12 skipped).

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Both exact-head Codex P2 findings are fixed in bfd01e775e64c2ec93ae163b6148834500de8992.

1. Init lifecycle in switch remediation: canonical switch remediation now renders a pinned --dry-run preview, then a pinned --apply --output-dir .code-mower.generated, then "review the generated configuration and support files and install them through the normal setup PR", and only then the pinned code-mower doctor ... --devin rerun — stating explicitly that staging writes only that review tree and the active posture keeps reporting the installed configuration until the generated one replaces it. The source configuration is never overwritten. A new regression follows the rendered commands literally: the preview writes nothing, the apply writes only the review tree (whose generated config selects hosted), and readiness on the untouched source still reports the local posture. The staged directory is pinned to init.DEFAULT_APPLY_OUTPUT_DIR by test.

2. Custom-named lanes: the participant picker is gone. A custom-named Devin lane now gets bounded manual guidance naming only the configured lane IDs and the public product / provider / transport / driver fields to set, the caller-supplied config path and profile (shell-quoted), a review-the-diff step, and the pinned doctor rerun. No --interactive, --with, or --set-transport appears in any custom-lane remediation — asserted for one and two custom local lanes and one and two custom hosted lanes, so no lane or participant can be dropped.

Validation on this head: full suite 3059 OK (skipped=12), focused readiness/capability/init/doctor/next-step/release suites, Ruff, privacy scan, compileall, easy-mode smoke.

@jeffhuber

Copy link
Copy Markdown
Contributor

Codex audit (merge-authority lane)

Head SHA: bfd01e775e64c2ec93ae163b6148834500de8992
Findings: P0=0, P1=0, P2=1, P3=0 (blocker policy: any P0/P1/P2 → BLOCKED)

Codex Audit: BLOCKED

Summary:

Manual transport-switch remediation is incomplete for supported custom-lane configurations and can cause validation errors or retain the original transport.

Findings:

  • [P2] Retarget all transport-dependent settings in manual guidance -- /private/var/folders/f2/g9zqfdjx7z3ckcsx_f46htwm0000gn/T/codex-audit-owjbelfj/wt/src/code_mower/devin_readiness.py:886
    Finding ID: codex:ea5a3c9710e3ec0d1383
    For custom-named Devin lanes, following this guidance can leave the configuration invalid or still selecting the old transport. A lane copied from the maintained configuration retains capabilities that lane_transport() rejects after these four fields change; an existing session_defaults.transports.devin or transport-specific participant alias also continues to select the original transport. Include instructions to update the saved selection and remove or replace transport-dependent capabilities and provider settings, rather than preserving every other field.

@jeffhuber

Copy link
Copy Markdown
Contributor

Claude audit (merge-authority lane)

Head SHA: bfd01e775e64c2ec93ae163b6148834500de8992
Findings: P0=0, P1=0, P2=0, P3=2 (blocker policy: any P0/P1/P2 -> BLOCKED)

Claude Audit: PASS

Summary:

Large, well-scoped PR adding optional Devin (local CLI / hosted v3) readiness reporting to doctor, init --set-transport, session briefs, and next-steps, plus updated capability declarations (message/cancel/structured_results now remote_session for hosted). Traced the core selection logic (selected_devin_transport, devin_effective_lanes, config_with_transport), the doctor stage wiring in runner.py, the schema/template mirrors, and the privacy-redaction paths in devin_readiness.py; all are internally consistent with each other and with the very extensive new test suite (1631 lines) that exercises ambiguous-selection, custom-lane, credential-privacy, and transport-switch scenarios. No correctness, security, or data-loss blockers found.

Findings:

  • [P3] selected_devin_transport / devin_selection_ambiguity computed twice per doctor run -- src/code_mower/doctor_checks/runner.py:195
    Finding ID: claude:c49dce0924ea270d3e39
    run_doctor calls devin_readiness_selected(...) and then devin_selection_ambiguity(...) with the same arguments; both internally call selected_devin_transport(...) and re-parse the same config, doing the ambiguity resolution work twice per invocation. Harmless (pure function, no side effects) but avoidable duplication.
  • [P3] --with and --set-transport interaction is unspecified -- src/code_mower/init.py:2482
    Finding ID: claude:7af69b8a0eb41da73b33
    init.main() allows both --with and --set-transport to be passed together; render_init_plan applies config_with_participants first and then config_with_transport on top with no validation or documented precedence, which could surprise a caller who expects --with alone to fully determine the Devin transport.

Manual guidance for custom-named Devin lanes now names every setting a
retarget depends on: the transport-dependent capabilities block, the
lane's campaign transport, the saved session_defaults transport, and the
transport-specific participant alias. Changing only the four declaration
fields could leave a copied lane invalid or still selecting the old
transport.

The saved transport selection is repository-wide, so a targeted switch
now retargets the canonical Devin lane of every profile that selects
Devin instead of leaving another profile whose declared lane the saved
selection contradicts. Profiles that do not select Devin keep their lanes
unchanged, and a custom-named Devin lane in any profile is still reported
rather than rewritten.

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Round 13 on 5ca51a5e36905d98590b4ae579d7f0cb8aeb9530, answering the exact-head Codex P2 on custom-lane guidance and the preaudit's profile-selection defect.

Custom-lane guidance is now complete. custom_lane_guidance() names every setting a retarget depends on, not only product/provider/transport/driver: drop the lane's capabilities block to use the maintained defaults for the target transport (or restate them for it), retarget provider_config.campaign_transport, set session_defaults.transports.devin to the target transport, and replace the Devin participant alias with that transport's alias — then review the diff and rerun the pinned doctor to revalidate. Unrelated lane fields, participants, and profile lanes are still preserved, and no --interactive, --with, or --set-transport is emitted for a named lane. The regression starts from full maintained local and hosted declarations with explicit session_defaults, renames them to custom lanes (one lane, and two same-transport lanes with distinct commands), performs exactly the named edits in both directions, then asserts validate_config() returns no issues and pinned readiness reports the target transport with each lane keeping its own ID.

--set-transport no longer leaks a profile-scoped switch. The saved selection (session_defaults.transports.devin and the participant alias) is repository-wide, so config_with_transport() now retargets the canonical Devin lane of every profile that selects Devin instead of leaving an untouched profile whose declared lane the saved selection contradicts. Profiles that do not select Devin stay structurally unchanged, and a custom-named Devin lane in any profile is reported instead of rewritten. The --set-transport help, the upgrade docs, and the rendered remediation state that scope. A two-profile regression checks every profile's effective selected transport after a switch, plus a fail-closed case for a custom-named lane in a second profile.

Validation on this head: full suite 3062 tests OK (12 skipped), focused readiness/capability/remote-session/review suites, Ruff, privacy scan, compileall, easy-mode smoke, release hygiene.

Claude's two P3s on the previous head need no change: the duplicated ambiguity resolution in run_doctor is a pure function called twice with the same arguments (no side effects, negligible cost), and --with with --set-transport is a deliberate composition — participants are selected first, then only Devin's transport is replaced on top, which is the same result as running the two steps in sequence.

The saved Devin selection lives in session_defaults, which applies to every
profile, so telling an operator to align it for the inspected profile while
leaving every other profile lane as configured could contradict another
profile's named Devin lane. The guidance now says to inspect every profile that
selects Devin and retarget its named lanes before saving a transport, and to save
no transport-specific selection or alias at all when profiles intentionally keep
different transports, letting each profile's lane declaration select its own.
Profiles are described rather than enumerated, so nothing outside the lane IDs
already carried by the finding is echoed.

A multi-profile regression follows both routes from full maintained local and
hosted declarations and checks every profile's effective selected transport.

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Round 14 on 965d002cc0c42519de8729e1e66ef0496ff3f0ac, answering the remaining preaudit P2 on custom-lane guidance versus the repository-wide saved selection.

The saved Devin selection (session_defaults.transports.devin plus the transport-specific participant alias) applies to every profile, so guidance that aligned it for the inspected profile while saying to leave every other profile lane as configured could leave a second profile's named Devin lane contradicted by it. custom_lane_guidance() now states and enforces the coherent contract: inspect every profile that selects Devin and retarget its named lanes the same way before setting the saved transport and alias, or — when profiles intentionally keep different Devin transports — save no session_defaults.transports.devin and no transport-specific alias at all, name the product as devin, and let each profile's lane declaration select its own transport under the supported generic selection. Profiles are described rather than enumerated, so nothing beyond the selected lane IDs already carried by the finding is echoed; the rerun and revalidation stay pinned to the caller's config and --profile, and --interactive, --with, and generated rewriting remain absent.

The new multi-profile regression builds a valid configuration where two profiles each select their own named Devin lane, then follows both offered routes for local→hosted and hosted→local: aligning both profiles together leaves every profile resolving the target transport, and keeping the transports apart with no saved transport or alias leaves each profile resolving exactly the transport its own lane declares, verified through validate_config(), selected_devin_transport(), and pinned readiness per profile. docs/upgrade-existing-repo.md and docs/sessions.md state the same scope.

Validation on this head: full suite 3063 tests OK (12 skipped), focused capability/readiness/remote-session/review suites, Ruff, privacy scan, compileall, easy-mode smoke, release-hygiene and package checks.

@jeffhuber

Copy link
Copy Markdown
Contributor

Codex audit (merge-authority lane)

Head SHA: 965d002cc0c42519de8729e1e66ef0496ff3f0ac
Findings: P0=0, P1=0, P2=0, P3=0 (blocker policy: any P0/P1/P2 → BLOCKED)

Codex Audit: PASS

Summary:

No actionable regressions were identified in the diff. Import checks passed; the full test suite was not run under the read-only sandbox.

Findings: none.

@jeffhuber

Copy link
Copy Markdown
Contributor

Claude audit (merge-authority lane)

Head SHA: 965d002cc0c42519de8729e1e66ef0496ff3f0ac
Findings: P0=0, P1=0, P2=0, P3=1 (blocker policy: any P0/P1/P2 -> BLOCKED)

Claude Audit: PASS

Summary:

Large, well-tested feature PR adding optional Devin setup/readiness reporting (doctor --devin, devin_readiness.py, targeted --set-transport switch) plus a capability upgrade for hosted Devin (message/cancel/structured_results now remote_session instead of unavailable). Reviewed the new readiness module, doctor stage wiring, transport-switch config mutation logic, and capability/schema changes for correctness, privacy leakage, and contract breaks. No blocking defects found; the extensive new test suite (tests/test_devin_setup_readiness.py) covers the privacy-safety and multi-profile edge cases the code implements.

Findings:

  • [P3] --set-transport silently adds Devin to the targeted profile even if that profile never selected it -- src/code_mower/participants.py:300
    Finding ID: claude:a7924c5dedc51fe0cf52
    In config_with_transport, the loop for name, product_lanes in affected.items(): if name != profile and not product_lanes: continue only skips other profiles that never selected the product; the explicitly named profile is always processed and unconditionally gets the target lane appended at the end even when it had zero Devin lanes before. This is exercised intentionally by the 'Add Devin' guidance path (devin_readiness._unselected_findings routes users to select_transport_command to add Devin) and is covered by test_switching_the_transport_keeps_every_other_selection, so it appears to be a deliberate dual-purpose 'add or switch' design rather than a bug — flagging only because the accompanying docstring/comments ('profiles that do not select Devin are untouched') could be read as applying to the named profile too, which could surprise an operator who expected --set-transport to be a no-op switch on a profile without an existing Devin lane.

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Re: Claude's P3 on config_with_transport (claude:a7924c5dedc51fe0cf52) — correct reading, and the dual purpose is deliberate: --set-transport is the "select Devin on this transport" command as well as the switch, because unselected-posture guidance routes an operator here to add Devin to the profile they inspected. The named profile therefore always ends up selecting the target lane; only other profiles are left alone unless they already selected Devin, which is what the "profiles that do not select Devin are untouched" sentence covers.

I'm leaving the head at 965d002cc0c42519de8729e1e66ef0496ff3f0ac rather than pushing a docstring-only change, since both merge-authority lanes passed this exact SHA and the release qualification round is bound to it. The wording is worth tightening — happy to say "other profiles that do not select Devin" in the docstring and the --set-transport help on the next substantive commit if a maintainer wants it in this PR.

@jeffhuber
jeffhuber merged commit d09e523 into main Sep 12, 2026
18 checks passed
@jeffhuber
jeffhuber deleted the devin/910-setup-doctor branch September 12, 2026 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Devin: optional setup, host guidance, and lifecycle doctor

1 participant