Devin: one optional setup and readiness path for local CLI and hosted v3 - #940
Conversation
Report the selected Devin posture, its authentication, permission requirements, capability gaps, and lifecycle recovery from a single readiness module surfaced by doctor, the session brief, next steps, and docs. Claude + Codex stay the default and a repository without Devin produces no Devin checks. Closes #910 Co-Authored-By: bot_apk <apk@cognition.ai>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Codex audit (merge-authority lane)Head SHA: Codex Audit: BLOCKED Summary: The readiness checks can inspect the wrong executable or report the wrong Devin transport for supported configurations, producing misleading setup guidance. Findings:
|
Supplemental exact-head audit findingExact reviewed head:
Please resolve this together with the two P2 findings in the structured Code Mower Codex verdict before requesting a new exact-head audit. |
…ed test credentials Co-Authored-By: bot_apk <apk@cognition.ai>
Re: Codex audit (merge-authority lane) — both P2 findings resolvedNew head:
|
Re: supplemental exact-head audit finding — P2 resolvedNew head: Every readiness fixture now resolves credentials through an isolated empty temporary configuration directory, and the doctor CLI cases pass Local validation on this head: privacy scan, Ruff, |
Codex audit (merge-authority lane)Head SHA: Codex Audit: BLOCKED Summary: The readiness command can inspect a different configuration or profile from the setup plan, producing misleading guidance for custom setups. Findings:
|
Co-Authored-By: bot_apk <apk@cognition.ai>
Re: Codex audit of
|
Codex audit (merge-authority lane)Head SHA: Codex Audit: BLOCKED Summary: Two P2 findings: readiness output exposes credential-profile paths, and the generated command can inspect the wrong Devin transport. Findings:
|
Claude audit (merge-authority lane)Head SHA: Claude Audit: PASS Summary: Devin-authored PR adds an optional, well-isolated Devin setup/readiness doctor stage (devin_readiness.py, doctor_checks/devin.py, registry/runner wiring, session/next-steps guidance) with extensive test coverage. Reviewed control flow, stage gating (default runs unaffected, devin-readiness only enabled when selected or --devin passed), credential/privacy handling (no secret values or repository inventory leaked into findings), and doc consistency. No correctness, security, or data-loss blockers found. Findings: none. |
Co-Authored-By: bot_apk <apk@cognition.ai>
Re: Codex audit of
|
|
Both exact-head Codex P2 findings are fixed in 1. Init lifecycle in switch remediation: canonical switch remediation now renders a pinned 2. Custom-named lanes: the participant picker is gone. A custom-named Devin lane now gets bounded manual guidance naming only the configured lane IDs and the public Validation on this head: full suite 3059 OK (skipped=12), focused readiness/capability/init/doctor/next-step/release suites, Ruff, privacy scan, compileall, easy-mode smoke. |
Codex audit (merge-authority lane)Head SHA: Codex Audit: BLOCKED Summary: Manual transport-switch remediation is incomplete for supported custom-lane configurations and can cause validation errors or retain the original transport. Findings:
|
Claude audit (merge-authority lane)Head SHA: Claude Audit: PASS Summary: Large, well-scoped PR adding optional Devin (local CLI / hosted v3) readiness reporting to Findings:
|
Manual guidance for custom-named Devin lanes now names every setting a retarget depends on: the transport-dependent capabilities block, the lane's campaign transport, the saved session_defaults transport, and the transport-specific participant alias. Changing only the four declaration fields could leave a copied lane invalid or still selecting the old transport. The saved transport selection is repository-wide, so a targeted switch now retargets the canonical Devin lane of every profile that selects Devin instead of leaving another profile whose declared lane the saved selection contradicts. Profiles that do not select Devin keep their lanes unchanged, and a custom-named Devin lane in any profile is still reported rather than rewritten. Co-Authored-By: bot_apk <apk@cognition.ai>
|
Round 13 on Custom-lane guidance is now complete.
Validation on this head: full suite 3062 tests OK (12 skipped), focused readiness/capability/remote-session/review suites, Ruff, privacy scan, compileall, easy-mode smoke, release hygiene. Claude's two P3s on the previous head need no change: the duplicated ambiguity resolution in |
The saved Devin selection lives in session_defaults, which applies to every profile, so telling an operator to align it for the inspected profile while leaving every other profile lane as configured could contradict another profile's named Devin lane. The guidance now says to inspect every profile that selects Devin and retarget its named lanes before saving a transport, and to save no transport-specific selection or alias at all when profiles intentionally keep different transports, letting each profile's lane declaration select its own. Profiles are described rather than enumerated, so nothing outside the lane IDs already carried by the finding is echoed. A multi-profile regression follows both routes from full maintained local and hosted declarations and checks every profile's effective selected transport. Co-Authored-By: bot_apk <apk@cognition.ai>
|
Round 14 on The saved Devin selection ( The new multi-profile regression builds a valid configuration where two profiles each select their own named Devin lane, then follows both offered routes for local→hosted and hosted→local: aligning both profiles together leaves every profile resolving the target transport, and keeping the transports apart with no saved transport or alias leaves each profile resolving exactly the transport its own lane declares, verified through Validation on this head: full suite 3063 tests OK (12 skipped), focused capability/readiness/remote-session/review suites, Ruff, privacy scan, compileall, easy-mode smoke, release-hygiene and package checks. |
Codex audit (merge-authority lane)Head SHA: Codex Audit: PASS Summary: No actionable regressions were identified in the diff. Import checks passed; the full test suite was not run under the read-only sandbox. Findings: none. |
Claude audit (merge-authority lane)Head SHA: Claude Audit: PASS Summary: Large, well-tested feature PR adding optional Devin setup/readiness reporting (doctor --devin, devin_readiness.py, targeted --set-transport switch) plus a capability upgrade for hosted Devin (message/cancel/structured_results now remote_session instead of unavailable). Reviewed the new readiness module, doctor stage wiring, transport-switch config mutation logic, and capability/schema changes for correctness, privacy leakage, and contract breaks. No blocking defects found; the extensive new test suite (tests/test_devin_setup_readiness.py) covers the privacy-safety and multi-profile edge cases the code implements. Findings:
|
|
Re: Claude's P3 on I'm leaving the head at |
Summary
devin_readiness.pybecomes the one place that answers "is the selected Devin posture ready, and what do I run next", and doctor, the session brief, next steps, and the docs read from it. Claude + Codex stayDEFAULT_PARTICIPANTS; a repository that never selected Devin produces no Devin findings, and the new doctor stage stays out of its plan entirely.devin_cliis authenticated only by the ambient Devin Desktop/CLI login,devin_api_v3only by dedicated service-user credentials plus the exactOWNER/REPOacknowledged locally (hosted has no read-only preflight proving GitHub connection scope, so acknowledgement is the gate). Create/view/manage requirements are reported as an owner action with statusskiprather than warned about on every repository, because no read-only permission probe exists.main. Hosteddevin_api_v3declaresmessage,cancel, andstructured_resultsasremote_session, becausecode-mower session message|cancel|collectand the Devin provider really perform those operations;coordinatestaysunavailableandreviewstaysevidence_only, so selection still confers no review or merge authority.devin_clikeeps its real gaps (message/cancelunavailable,structured_resultslocal-runner) and never claims remote lifecycle. Every closed copy moves together — the registry,provider_capabilities.schema.json,templates/providers.yml,src/code_mower/templates/providers.yml,templates/providers/devin.yml— andLEGACY_CAPABILITIESaccepts both earlier hosted declarations so an existing config still normalizes in memory without rewriting the file. A cross-contract test ties each hostedremote_sessionmode to the actual status/message/cancel/collect operation behind it.--profilewith--provider-profile NAME— instead of rendering the resolver's status where a variable name belongs.setup_instructions) emits the caller's configuration path and explicit--profile, shell-quoted, includingrecommended; a guidance surface without those inputs says to rerun with the same configuration and profile instead of an unscoped command that could read another profile's Devin lane. Static doc examples usecode-mower doctor --profile recommended --devin --repo OWNER/REPO --jsonand explain replacingrecommended. The Code Mower--profilestays distinct from the credential--provider-profile. That pinning now covers every actionable remediation, not only generated guidance: each transport-switch,init, and rerun command in a finding is built from the configuration path and configuration profile that produced it, quoted withshlex.quote()and always carrying an explicit--profile(includingrecommended), so an operator inspectingops/custom mower.yml --profile 'custom profile'is never redirected to the default starter or the recommended profile. Public detail and cloud metadata stay path-free; only the locally rendered remediation repeats the caller's own path.unavailablegaps, so hosted no longer claims messaging and cancellation are unavailable while its own detail reportsremote_session, and local CLI still names its real gaps without implying remote lifecycle. Local readiness detail and remediation likewise describe only the effective lane's own candidates (as basenames) and its actualcommand_envselector, so an operator is never sent to a variable the lane's runtime ignores; the historical override and defaultdevinanswer only when no lane configuration was supplied.team_devindeclaringproduct: devinwith a canonical provider/transport/driver activates the stage, selects its transport, supplies its own command configuration to readiness, and gets the appended readiness next step, while canonical aliases and fail-closed mixed-transport ambiguity are unchanged. That identity now reaches the findings themselves:devin_effective_lane()returns the selected lane's ID with its effective configuration, andcheck_devin_readiness()passes it todevin_readiness(), soteam_devinfindings reportlane=team_devininstead of a canonical lane the repository does not have, two custom lanes on one transport stay distinguishable, and an invocation whose lanes span both transports with no explicit transport fails closed through the same boundedprovider.devin.selectionfinding as an ambiguous transport.team_devinandnight_devinboth local), so readiness resolves every effective lane the selected transport matches instead of rejecting the configuration. The answers that belong to the product — selection, capabilities, permissions, lifecycle, and hosted credentials/repository scope — are emitted once with the sharing lanes named in detail rather than attributed to one arbitrary lane, and every executable readiness finding is emitted per lane against that lane's own ID and effective command configuration. A single selected lane keeps every finding bound to it as before, and lanes spanning both transports with no explicit transport still fail closed.code-mower init --set-transport devin=devin_cli|devin_api_v3replaces only Devin's transport, its own profile lanes, and its own participant alias, keeping unrelated participants (Cursor, Gitar, a repository's own custom lane) and every other profile lane as configured; and remediation still carries the exact quoted configuration path and--profile. Because the saved selection (session_defaults.transports.devinplus the participant alias) is repository-wide, the switch retargets the canonical Devin lane of every profile that selects Devin rather than leaving an untouched profile whose declared lane the saved selection contradicts; profiles that do not select Devin are structurally unchanged, and a custom-named Devin lane in any profile is reported instead of rewritten. The--set-transporthelp, docs, and remediation state that scope.initlifecycle: a pinned--dry-runpreview, a pinned--apply --output-dir .code-mower.generatedthat stages a reviewable generated tree without touching the configuration it read, a review-and-install step through the normal setup PR, and only then the pinnedcode-mower doctor ... --devinrerun. Staging never claims the active posture switched.product/provider/transport/driverfields, the transport-dependentcapabilitiesblock (dropped for the maintained defaults or restated for the new transport),provider_config.campaign_transport, the savedsession_defaults.transports.devin, and the transport-specific participant alias — then the pinned doctor rerun to revalidate. Editing the four declaration fields alone could leave a lane copied from the maintained configuration invalid or still selecting the old transport. That saved selection is repository-wide, so the guidance states the coherent contract rather than aligning one profile in isolation: inspect every profile that selects Devin and retarget its named lanes the same way before saving a transport, or — when profiles intentionally keep different Devin transports — save nosession_defaults.transports.devinand no transport-specific alias at all, name the product asdevin, and let each profile's lane declaration select its own transport. Profiles are described, never enumerated, so nothing beyond the lane IDs already carried by the finding is echoed. It never emits--interactive,--with, or any command that could rebuild or narrow the profile: the participant picker selects products and canonical reviewer lanes, so it would drop the lanes it cannot name.skipbefore any executable lookup, so "availability was not inspected" is literally true.provider_config.command_env,command,alternate_commands, then the provider fallback) through a neutrallocal_cli_commandsmodule shared with lane runtime, so readiness cannot contradict what runtime would execute, and it reads the injectedenvmapping rather than ambientos.environ. A configured lane's candidates are the whole candidate set — the historicalCODE_MOWER_DEVIN_CLI_COMMANDoverride and the defaultdevinanswer only for a caller with no lane configuration, so a configured command that is not installed cannot pass readiness while runtime reports the lane unavailable. When both Devin lanes are active, the lane handed to readiness is the one whose transport was selected (TRANSPORTS[transport].review_lane), not whichever lane comes first. Observer postures (--orchestrator-only,--hosted-builders)skipthe local executable requirement in the same modescheck_lane_runtimeskips it.session_defaults.transports.devindisambiguates them, it raises a boundedConfigErrornaming the way out instead of taking first lane order.devin_readiness()forwards the caller's configuration profile to that selection rather than falling back torecommended. An unresolvable selection is itself a readiness answer, so planning keeps the stage and the report carries a failingprovider.devin.selectioncheck with that remediation instead of omitting Devin readiness.load_inputs, so the stage activates for a repo that selected Devin (by participant or by an active Devin review lane), and--devinadditionally prints the local CLI, hosted API, and unavailable postures with next actions before selection:session.pyappends the posture's authentication and permission line to host instructions only when Devin is selected, so the default brief is unchanged.next_steps.pyappends (not inserts) the readiness step, preserving the existing positional reviewer-step semantics. The new modules are registered in the package manifest.doctor_checks/devin.pyimportscode_mower.devin_readinessper call: at module scope it closes a cycle throughdevin_api→campaign_adapters→release_qualify→doctor_checks/__init__.Closes #910. Part of #901.
Validation
.venv/bin/python -m unittest discover -s tests— 3063 tests, OK (skipped=12)tests.test_devin_capabilities,tests.test_devin_setup_readiness,test_devin_sessions,test_participant_sessions,test_session_lease,test_remote_session,test_devin_work_orders,test_devin_review,test_devin_provider_contract,test_devin_api,test_devin_builder_lane, and everytest_doctor*module — all OKremote_sessionmodes tied to real status/message/cancel/collect operations, readiness capability and lifecycle rendering, profile-pinned and safely quoted generated commands (default and non-default profiles, custom config paths), guidance without profile inputs, multi-profile local-vs-hosted selection, ambiguity reported by ordinary doctor and by--devin, lane candidates matchingresolved_local_cli_command()for an uninstalled configured command, effective-lane selection under both lane orders, injected-env command discovery with configured/custom-env/alternate commands, basename-only reporting, observer-posture parity plus normal local execution, non-default configuration-profile transport selection, credential path/identity privacy, hosted-versus-local generic capability remediation derived from declared modes, a configured lane whose candidates and customcommand_envare the only ones named, the no-lane-configuration historical fallback, custom-named local and hosted Devin lanes through doctor andbuild_next_steps, ambiguous credential profiles naming no variable/profile/filename/path, observer postures performing zero executable lookups, configuration-pinned remediation for local selection, hosted selection and credentials, an unavailable local CLI, transport switching, the observer rerun and unselected guidance under a spaced configuration path with a non-default profile, an explicitrecommendedprofile, custom local and hosted lane identity on every finding, an unidentifiable effective lane failing closed, one custom local lane and one custom hosted lane, two same-transport custom local lanes with different commands reported per lane, two same-transport hosted lanes with credentials and repository scope emitted once, unresolved mixed transports failing closed, an end-to-endinitmutation through the real config-writing path with Claude, Codex, Cursor, Gitar, Devin and an unrelated custom lane asserting only Devin's transport and lane move, the default Claude + Codex pair and a configuration with no saved participant list, custom-named local and hosted lanes (single and multiple) receiving manual declaration guidance with no picker/selection command, the rendered switch commands followed end to end so the preview mutates nothing, the apply writes only the review tree, and readiness still reports the installed local posture until the generated configuration is installed, the manual guidance followed end to end from full maintained local and hosted declarations with explicit session defaults for one and for two same-transport custom lanes provingvalidate_config()passes and pinned readiness reports the target transport, a two-profile switch checking each profile's effective selected transport with a Devin-free profile untouched, a custom-named Devin lane in a second profile failing closed, a multi-profile custom-lane configuration following both routes the guidance offers — every Devin profile aligned together, or the transports kept apart with no saved transport or transport-specific alias — proving no profile is left reporting a transport it does not declare, safe quoting, and an unchanged fresh no-Devin installation.venv/bin/python scripts/privacy_scan.py,.venv/bin/python -m ruff check .,.venv/bin/python -m compileall -q src scripts,.venv/bin/python scripts/smoke_easy_mode.py --code-mower-bin .venv/bin/code-mower --json, and the package/release-hygiene checks — all passRisk / Rollback
Data Or Secret Exposure
Link to Devin session: https://app.devin.ai/sessions/d75626971cdc455f8db3b053ac2489fc
Open in Devin Desktop: https://app.devin.ai/desktop/session/d75626971cdc455f8db3b053ac2489fc?variant=devin