Add remote plugin install, index discovery, and plugin dependencies

[ashtom]

https://entire.io/gh/entireio/cli/trails/973

Summary

Expands the kubectl-style external-command layer so users can discover and install plugins without cloning anything, from any git host. 18 files, ~5,540 insertions.

• Remote install — entire plugin install <name|url|path>. Newest stable semver tag via git ls-remote (forge-agnostic, inherits git auth and proxies; no forge REST client anywhere). Optional entire-plugin.yml metadata read via blobless shallow clone. Release assets download over HTTPS through a small per-host URL convention table (GitHub/Gitea-style, GitLab-style, download_url template escape hatch), selected and verified against the release's checksums.txt. Binaries land in pkg/<name>/ with a provenance manifest.yml; bin/ stays the only dispatch surface and the resolver in plugin.go is untouched.
• Upgrade — entire plugin upgrade [name|--all], --pin to hold a tag, next-highest-tag fallback for pushed-but-unpublished releases, Windows locked-binary rename-aside.
• Discovery — krew-style git-synced index (entireio/plugin-index): shallow-cloned into the user cache keyed by URL hash, flock-guarded, 24h TTL, stale-copy-on-offline. Six new subcommands (upgrade, search, info, browse, doctor, index update) join the pre-existing install/list/remove; bare-name installs resolve through the index. Index URL: --index > ENTIRE_PLUGIN_INDEX_URL > built-in default.
• Dependencies — entire-plugin.yml requires (name + optional min_version, no ranges), resolved by name through the index. Install-time transitive planning, apt-style single confirmation, remove guard for depended-on plugins, and entire plugin doctor. Dispatch stays zero-cost — no runtime dependency checks.

Security model

This code ends in downloading a binary and making it executable, so the trust boundaries are the design:

• Transport. HTTPS required for anything off-machine, re-validated on every redirect hop (Go follows redirects across schemes; validating only the entry point is bypassable). Loopback exempt so httptest fixtures work.
• Authentication. A release that publishes no checksums.txt for the platform is refused by default; --allow-unverified opts in, is recorded in the manifest, reported by doctor, and inherited by upgrade.
• Integrity. The installed binary's digest is recorded as binary_sha256 and re-checked by doctor. (sha256 covers the downloaded asset, which is discarded with the staging dir — provenance only.)
• URLs reaching git. Every repo/index URL is scheme-validated by one definition and every git invocation passes -- before positionals. Repo URLs are attacker-influenced (index entries, --index, env), and git reads an option-shaped positional as an option — --upload-pack's value is shell-interpreted.
• The trust root is not repo-choosable. The index URL cannot be set from committed settings; see below.
• Names. The installed name comes from the remote, so callers that already committed to a name pass it as a required argument and a mismatch is fatal.
• Credentials. Userinfo and query strings are stripped from every URL before printing, logging, or persisting (.entire/logs/ is inside the working tree and doctor bundle collects it wholesale; manifest.yml is mode 0644).

Not claimed: authenticity. checksums.txt shares an origin with the asset, so it proves integrity in transit, not publisher identity. Real authenticity needs signatures.

Design notes

• Built on the recorded founding decisions of this layer: PATH dispatch unchanged, built-ins win, agent- prefix reserved, env filtering untouched, no new telemetry.
• The only forge-specific code is the download-URL convention table; version listing, metadata, and the index all ride on the git protocol. No go-github-class dependency (uses existing gopkg.in/yaml.v3 + golang.org/x/mod).
• The plugin layer reads no settings at all. The settings package is byte-identical to main.

Changed since this description was first written

The branch was reviewed (Bugbot ×5 rounds, Copilot, four trail agent reviews, and a three-agent adversarial pass), then merged with main (2053 commits). Reviewers should know these are new:

Behavior changes

1. Checksums are now required by default (--allow-unverified opts out). Previously a release without them installed silently.
2. http:// and git:// are no longer accepted for repo or index URLs — unauthenticated transports, and the catalog decides what installs without a prompt.
3. Prereleases are excluded from tag resolution. v2.0.0-rc1 outranks stable v1.9.0 in semver, so upgrade --all was migrating users onto release candidates.

Removals
4. plugins.index_url and plugins.index_ttl_hours are gone. .entire/settings.json is committed and resolved from the working directory, while an index-listed plugin installs with no prompt — so a cloned repo could redirect the catalog and get an attacker-chosen binary onto PATH silently. The setting's value was that contributors got a different catalog without knowing, which is the vulnerability stated as a feature. Internal catalogs now use ENTIRE_PLUGIN_INDEX_URL, which applies across repos and cannot be chosen by checked-out content. Go takes the same position with GOPROXY; npm's per-project registry is the cautionary counter-example.
5. requires[].repo_url is gone. Dependencies resolve by name through the index, so a dependency's URL comes from the curated catalog rather than the requiring plugin's author — planning contacted that URL before the confirmation prompt. A user can still install an out-of-catalog dependency themselves, with the usual prompt; the authority moved to the user.

Fixes worth naming
6. Argument injection into the git CLI allowing arbitrary command execution from a malicious index entry or dependency URL — verified live on the pre-fix tree, on a path that never prompts.
7. entire-plugin.yml now decodes leniently. Strict decoding meant the first author to adopt any future field would break installs on every older CLI, permanently. This is also what made removals 4 and 5 migration-free for already-published files.
8. Silent truncation of archive entries over 512 MiB, index-cache races (now flock-guarded), a diamond min_version skip, darwin_all universal binaries never matching, empty entire-plugin.yml being fatal, index_ttl_hours overflowing to "always stale", and credential leaks through download errors.

Testing

• 77 unit tests against file:// git repos and httptest asset servers (zero real network in CI), covering tag resolution, asset selection/verification, extraction guards, index sync/TTL/offline/concurrency, dependency planning, doctor.
• 4 integration tests drive the spawned binary end to end (install/dispatch, unlisted-URL gating, dependencies + remove guard, search/info): bare-name install → real dispatch → dependency install → remove guard → doctor exit code. They run the verified path — no test passes --allow-unverified, so their passing is the proof that verification succeeds.
• Security regressions assert the stronger property where it matters: that an injected payload never executes and that credentials never appear, not merely that a call errors. Each was verified RED against the pre-fix tree.
• mise run check green (fmt, lint, unit + integration + Vogon canary).

Docs

• docs/architecture/external-commands.md — remote install, plugin index, dependencies, and the trust-boundary sections.
• README — user-facing Plugins section.

Open, deliberately

• browse and index.json's platforms are candidates to cut — browse adds no capability over search + install and installs unprompted; platforms warns but never enforces.
• min_version is not enforced on the install outcome (the planner computes a constraint nothing downstream checks), and dependencies have no auto_installed marker, so orphans leak.
• Two notions of "official": the officialPlugins telemetry allowlist and index.json's official flag.
• entire upgrade (the self-update plugin) vs entire plugin upgrade (plugins).
• Follow-up outside this branch: a credential-redaction pass at main.go's error print. %w-wrapped url.Error leaks the full URL regardless of call-site discipline, because net/url.Error.Error() does no redaction.
• Outside this repo: graph should be added to entireio/plugin-index, or entire plugin install brain reports its dependency as unindexed.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Expands the existing kubectl-style external-command layer to support a full plugin lifecycle: remote installs/upgrades (via git tag resolution + release asset download), discovery via a git-synced plugin index, and install-time dependency planning with a plugin doctor health check.

Changes:

• Add remote plugin install/upgrade plumbing (git ls-remote tag resolution, optional entire-plugin.yml metadata fetch, release asset download + extraction, provenance manifests).
• Add plugin index sync/search/info/browse/index-update flows with settings/env/flag precedence and TTL-based refresh.
• Add dependency planning/execution + remove guard + plugin doctor, plus docs/README updates and new unit/integration tests.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
README.md	Adds user-facing “Plugins” section and command examples.
docs/architecture/external-commands.md	Documents remote install, index discovery, and dependency semantics.
cmd/entire/cli/settings/settings.go	Adds repo-level plugin discovery settings (index URL + TTL) and validation/merge support.
cmd/entire/cli/settings/settings_plugins_test.go	Unit tests for plugin settings validation/TTL and merge semantics.
cmd/entire/cli/plugin_manifest.go	Introduces managed-install manifest + author metadata parsing (entire-plugin.yml).
cmd/entire/cli/plugin_manifest_test.go	Tests manifest I/O and strict metadata parsing behavior.
cmd/entire/cli/plugin_install_remote.go	Implements remote install orchestration, manifest writing, and upgrade path.
cmd/entire/cli/plugin_install_remote_test.go	Unit tests for remote install, pinning, fallback, upgrade, and removal cleanup.
cmd/entire/cli/plugin_index.go	Implements git-synced plugin index cache with TTL refresh and filtering.
cmd/entire/cli/plugin_index_test.go	Tests index sync/refresh/offline behavior and install-arg classification.
cmd/entire/cli/plugin_group.go	Adds Cobra commands/flags for install/upgrade/search/info/browse/doctor/index and dependency prompts.
cmd/entire/cli/plugin_gitremote.go	Adds git-shellout helpers for semver tag listing and metadata fetch-at-tag.
cmd/entire/cli/plugin_gitremote_test.go	Unit tests for tag sorting, metadata fetch, and repo-url name derivation.
cmd/entire/cli/plugin_fetch.go	Adds release asset URL conventions, checksum handling, downloading, and archive extraction.
cmd/entire/cli/plugin_fetch_test.go	Unit tests for URL conventions, checksum selection, extraction guards, and download verification.
cmd/entire/cli/plugin_deps.go	Adds dependency planning/execution, remove guard helpers, and plugin doctor checks.
cmd/entire/cli/plugin_deps_test.go	Unit tests for dependency planning, cycles, dependents, and doctor reporting.
cmd/entire/cli/login.go	Minor constant usage change for GOOS comparison.
cmd/entire/cli/integration_test/plugin_remote_install_test.go	End-to-end integration coverage for index install, URL confirmation gating, deps/remove-guard, and doctor exit behavior.
cmd/entire/cli/explain.go	Introduces darwinGOOS constant for reuse.

[ashtom]

Addressed all five review findings in cd5984b:

• Huh forms skip RunWithContext (Bugbot): confirmPluginAction and plugin browse now use form.RunWithContext(ctx) and route errors through handleFormCancellation, matching the doctor.go convention.
• Confirm abort treated as skip deps (Bugbot): a prompt error is no longer reported as a user-chosen skip. Ctrl+C/Esc prints "Dependency install cancelled." (clean exit, main install stands); real prompt failures propagate as errors. Non-interactive runs without --yes get a dedicated sentinel — fatal for untrusted-URL installs, informed skip for post-install dependency confirmation.
• Install path ignores context cancel (Bugbot): new silencePluginCancel maps cancellation to NewSilentError per the clean.go/activity_cmd.go convention, applied across install/upgrade/search/info/browse/index-update. It also checks ctx.Err() directly because a killed git child surfaces as "signal: killed" rather than context.Canceled.
• Asset path traversal + partial files (Copilot): fetchAndVerify now rejects asset names containing either separator kind or dot segments before joining into the staging dir (today's callers only pass internally-generated candidates, but the boundary is now enforced where it belongs), and removes the partial download on checksum mismatch, oversize, or write failure.
• rm -rf in test (Copilot): replaced with os.RemoveAll.

New unit tests cover the unsafe-name rejection and mismatch cleanup. mise run check (fmt + lint + unit + integration + canary) is green.

🤖 Generated with Claude Code

[ashtom]

bugbot run

[ashtom]

Addressed both round-two findings in 6ee0a3b:

• Skipped transitive dependency planning: planDeps now walks a satisfied managed dependency's recorded manifest requirements, so installing a parent repairs gaps deeper in the chain (e.g. a grandchild removed with --force after install). Planning stays offline — it reads the local manifest, no extra network. PATH/local-dev-satisfied deps have no manifest to walk; those remain plugin doctor's domain. Regression test: satisfied parent with a missing grandchild produces exactly the grandchild action.
• Failed index clone blocks retries: the un-cloned branch of SyncPluginIndex sweeps the cache directory before cloning, so a partial dir left by an interrupted first clone (no .git) can't wedge discovery behind git's non-empty-target refusal. Regression test: sync recovers from a junk-filled cache dir.

mise run check green.

🤖 Generated with Claude Code

[ashtom]

bugbot run

[ashtom]

Addressed both round-three findings in 117b23b:

• Install abort yields success exit: declining the untrusted-install prompt now prints "Install cancelled." and exits 0, identical to Esc/Ctrl+C — harmonized toward the handleFormCancellation convention every other confirm in the codebase uses. The automation concern can't arise in practice: non-interactive runs fail before the prompt with the --yes hint, so only interactive consistency was at stake.
• Local file shadows index name: classifyInstallArg no longer stats bare arguments. Names always resolve through the index; local paths must be explicit (./entire-foo or any separator-containing path) — git-style disambiguation, and the two spaces are disjoint since plugin names can't contain separators. When an index lookup misses but a same-named local file exists, the error hints at entire plugin install ./<name>. Command help documents the rule.

mise run check green.

🤖 Generated with Claude Code

[ashtom]

bugbot run

[ashtom]

Addressed the round-four finding in 813670a:

• Upgrade compares raw tag strings: both comparisons in UpgradeInstalledPlugin now use semver.Compare on canonicalized tags. Equivalent spellings (v0.2.0 vs 0.2.0) no longer reinstall — the check short-circuits before any download — and when an asset-less newest tag makes the install fall back to the already-installed version, the outcome reports up-to-date instead of an X → X upgrade line. Regression tests cover both (the spelling test runs against a repo with no asset server at all, so any download attempt would fail the test).

Known residual: a genuinely newer tag with an unpublished release still causes a re-download of the installed version on each upgrade run — avoiding that would require persisting known-asset-less-tag state, which doesn't seem worth it. Output is correct either way.

mise run check green.

🤖 Generated with Claude Code

[ashtom]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 813670a. Configure here.}

[ashtom]

Round five: two fixed, one rebutted, in 6da333c.

• Checksum miss aborts other manifests (fixed): a manifest listing no asset for the platform now falls through to the remaining checksumCandidates names and the direct-probe fallback instead of aborting, so a stale or hand-written root checksums.txt can't mask an installable release. No verification weakening — an attacker who controls the manifest could list a malicious digest directly. Regression test: stale root manifest + published conventional asset installs via the probe.
• Planning skips failed transitive fetch (fixed): when a scheduled dependency's tags or metadata can't be inspected, the plan now carries an explicit warning naming the plugin and pointing at entire plugin doctor, instead of silently omitting nested requirements. Regression test: untagged dependency repo plans the action plus the warning.
• Quarantine check ignores symlink target (not a bug): macOS xattr follows symlinks by default — -s is the flag to act on the link itself. Verified empirically: xattr -p com.apple.quarantine <bin-symlink> returns the attribute set on the pkg/ target, exit 0. Doctor's existing probe on the bin entry therefore already covers the real binary, and the suggested xattr -d fix also operates on the target. Added a code comment documenting this so it doesn't resurface.

mise run check green. Findings across rounds: 3 → 2 → 2 → 1 → 2+1 false positive — at this point the remaining surface looks like speculative edge cases, so leaving further review to humans unless something specific comes up.

🤖 Generated with Claude Code

[Soph]

Un-drafted after a review pass and a merge with main

This sat for ~7.5 weeks, so I reviewed the implementation, merged current main into it, and fixed what the review turned up. The Summary and Testing sections above describe the pre-merge state and are now stale — this comment is the current picture.

main moved 2053 commits in the meantime. Both merge conflicts were "both sides added a line in the same place": Plugins next to main's new Checkpoints field in EntireSettings, and the entire plugin README row next to main's new org/project/repo/grant rows.

⚠️ Please read ac0d13e29 closely — argument injection into the git CLI

Repo URLs were passed to git ls-remote and git clone as bare positionals with no validation. They are attacker-influenced: they arrive from index.json entries, from another plugin's requires[].repo_url, and from --index / ENTIRE_PLUGIN_INDEX_URL.

git parses an option-shaped positional as an option, and --upload-pack's value is shell-interpreted. With no positional repository left, git falls back to the ambient repo's origin and runs the payload. So an index entry of --upload-pack=curl … | sh; git-upload-pack executed arbitrary commands during entire plugin install <name> — a path that is treated as trusted and never prompts. Dependency planning was tighter still: it inspects requires[].repo_url before the install confirmation.

I confirmed this was live on the pre-fix tree before fixing it — the regression test asserts the payload never executes, not merely that the call errors, and it reports the marker file being created when run against the old code.

Two independent layers now, either sufficient alone: a scheme allowlist (validatePluginRepoURL) applied at the sinks and at the parse boundaries, plus -- before every git positional. This mirrors validatePluginName, which already refused a leading - on names for exactly this reason.

Behavior change worth a decision: checksums now required by default (26b10db99)

A release publishing no checksums.txt previously fell through to probing candidate names and installing whatever arrived. That now needs --allow-unverified, recorded in the manifest, reported by doctor, and inherited by upgrade. Plaintext HTTP is also refused for non-loopback hosts, because the asset and the checksums.txt authenticating it share an origin — an attacker who can rewrite one can rewrite both.

This is the change most likely to want discussion. It's easy to soften to warn-only, or to require checksums only for index-listed plugins. Flagging it rather than assuming.

Also in that commit: manifest.SHA256 covers the downloaded asset, which is usually an archive discarded with the staging dir — so it recorded provenance but could never detect tampering of the binary actually executed, which is why doctor had no integrity check. Installs now also record BinarySHA256 for the binary under pkg/<name>/, and doctor re-hashes it.

Other commits

• 0636c621b — the index version != 1 gate is now advisory. It guarded a migration that can never happen (the index is one shared resource read by every shipped CLI, so a bump breaks discovery fleet-wide), while punishing the case that does: an index omitting the field decoded to 0 and told the author to upgrade the CLI. Hand-written internal catalogs are exactly what the repo-level plugins.index_url setting is for.
• 56a063e4b — a fixed download_url carrying a query string yielded entire-run.tar.gz?token=abc from path.Base, missing the archive-extension sniff and getting written out verbatim as the binary.
• 2ac8323f9 — two lint errors from main's bf0c56b1c, which brought build-tagged files under golangci-lint after this branch was written.

Testing

mise run check green locally; all GitHub checks green here, including test-windows and both test-canary variants, which did not exist when this branch was last touched. Both release test servers now publish checksums.txt, so the end-to-end tests exercise the verified path production uses — no test passes --allow-unverified, so their passing is the proof that verification succeeds.

Still open (deliberately not addressed)

• Dependency upgrades install the newest tag without confirming min_version was actually reached; only doctor notices.
• Two sources of truth for "official": main's officialPlugins telemetry allowlist (now holding "ci") vs the index's official flag.
• entire upgrade (CLI self-update, special-cased in plugin.go since main's 5e12b0cac) vs entire plugin upgrade (plugins) — collision-free mechanically, confusable in docs.
• The index should probably now list ci and upgrade, both of which shipped while this was open.

🤖 Generated with Claude Code