Skip to content

feat: civitai CLI scaffold — App Blocks authoring (Phase 1) - #1

Merged
ZacxDev merged 2 commits into
mainfrom
scaffold-phase-1
Jun 18, 2026
Merged

feat: civitai CLI scaffold — App Blocks authoring (Phase 1)#1
ZacxDev merged 2 commits into
mainfrom
scaffold-phase-1

Conversation

@ZacxDev

@ZacxDev ZacxDev commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Greenfield Go 1.25 CLI for Civitai — a single static civitai binary in the gh/kubectl/stripe mold. Phase 1 ships the App Blocks authoring feature group under civitai app, replacing the confusing hand-format-a-ZIP flow: the CLI generates the correct project shape, validates the manifest against the platform contract, and packages/submits it.

Audit fixes applied (do not merge — Phase 1 review). The first audit found the code clean (no exec / zip-slip / traversal; honest submit) but flagged that validate oversold its fidelity: the vendored JSON Schema only covered syntactic rules, so it green-lit manifests the server's approve-time BlockManifestValidator rejects. This revision ports the missing semantic checks into the Go validate layer and makes the docs honest. See Validate fidelity below.

Command surface

civitai
├── app
│   ├── init [name] [--template static|page-vite] [--from <slug>]
│   ├── validate [dir]
│   └── submit [dir] [--package-only] [--out file.zip] [--skip-validate]
├── login [--token <token>]
└── whoami
  • app init — scaffolds a ready-to-build project from go:embed templates (static no-build page block; page-vite Vite+React with config-as-code buildCommand/outputDir). Manifests omit the server-owned iframe.src/trustTier. --from <slug> is stubbed with a clear "not yet wired" message — not faked.
  • app validate — a best-effort LOCAL pre-check mirroring the server's approve-time validator (the server remains source of truth). Validates against the vendored JSON Schema (syntactic) plus the ported semantic rules + structural checks (see below).
  • app submit — validates, packages the canonical source tree (manifest + src + build config, NOT a prebuilt dist; excludes .git/node_modules/dist; enforces the server caps 50 MiB / 2000 files / 10 MiB-per-file), then either uploads via a token endpoint (when configured) or writes the .zip + prints exact next steps.
  • login / whoami — token storage/verification via Viper (~/.config/civitai/config.yaml, 0600; CIVITAI_* env overrides).

Audit fixes in this revision

🔴 Sandbox trust-tier allowlist (ported from validateSandbox, validator ~L175-206)

A submitted block is always the unverified tier (trustTier is server-owned and forced at submit), whose sandbox allowlist is {allow-scripts, allow-forms} only. validate now:

  • rejects any token outside that allowlist (allow-popups, allow-top-navigation, allow-same-origin, …) with a field-pathed message that notes the tier is server-forced to unverified;
  • explicitly rejects the allow-same-origin + allow-scripts sandbox-escape combo (defense in depth, matching validator ~L201).

🟡 Other ported semantic checks (same false-VALID class)

  • page ⇒ iframe required (~L504) and renderMode=iframe ⇒ iframe required (~L377).
  • iframe required sub-fields when an iframe block is present: minHeight ∈ [40,4000] + resizable boolean (~L387-415) — the schema validated ranges when present but not required-ness.
  • renderMode tier gate: inline/hybridINLINE_REQUIRES_VERIFIED_TIER for unverified (~L289), rejected.
  • targets[].slotId registry membership + page-slot rejection (~L426-460), against a vendored copy of the 4 slot ids (3 model + app.page). Cheap to vendor; documented sync TODO. We're launching page-only (page apps have no targets[]), so this is rarely exercised today.

🟡 False-INVALID fixes (CLI rejected what the server accepts)

  • Dropped the CLI's 128-char name cap — the server only requires non-empty.
  • outputDir leading-/ / ..-traversal rejection kept (the server companion validates outputDir as a safe relative path), now a clear Go-side message instead of the cryptic JSON-Schema regex error.

🟡 Config 0600 race

internal/config.save() previously used viper WriteConfigAs (creates the file at umask 0644, then chmods 0600) — a brief world-readable window for the token. Now writes to a 0600 temp file and atomically renames it into place (also crash-safe). The perms test still asserts 0600.

🟡 Honesty

  • Added the two real example manifests under examples/ (buzz-generator, notepad — copied from the shipping civitai-block-* apps) + a test asserting validate passes them, so the "validates clean" claim is true. Both validate clean against the now-stricter checks; no divergence found.
  • Reframed README + validate help: validate is a best-effort local pre-check mirroring the server's approve-time validator; the server remains source of truth. Durable direction documented: a future server civitai app validate endpoint that runs the real BlockManifestValidator (the faithful contract), with the published schema as the syntactic half.

Validate fidelity

validate is a local mirror, not the contract. Syntactic rules live in the vendored JSON Schema; the semantic rules (sandbox tier allowlist, page⇒iframe, required iframe sub-fields, renderMode tier gate, slot-registry membership) are ported into Go from block-manifest-validator.service.ts. Necessarily approximate locally:

  • targets[].slotId uses a vendored slot list — a brand-new server slot would false-INVALID locally until the list is updated (it still validates server-side). We launch page-only, so the blast radius is minimal.
  • Origin-binding (iframe.src/assetBundleUrlOauthClient.allowedOrigins) and scope ⊆ client allowedScopes depend on per-app server state the CLI can't see — not reproduced.

Durable fix: a server civitai app validate endpoint that calls the real BlockManifestValidator. The published manifest schema is a step toward that.

Vendored manifest schema

schema/app-block.manifest.schema.json is derived from the server validator: required fields, scopes enum, page config (incl. positive-integer buzzBudgetPerGen), sandbox tokens, contentRating enum, buildCommand/outputDir. Embedded via go:embed. It covers the syntactic half; the semantic half is the ported Go checks above.

Submit path + the cross-repo dependency

The one cross-repo dependency (unchanged from the prior revision). The live route POST /api/blocks/submit-version is session-cookie + moderator (ModEndpoint) — no bearer token — so fully-programmatic submit is blocked on a server change. Implemented today: build + validate the canonical ZIP; if a token and CIVITAI_SUBMIT_PATH are configured, upload with Authorization: Bearer and the existing { "bundleBase64": ... } body; otherwise write the .zip + print exact manual next steps. Network/auth sits behind api.Submitter/api.Verifier interfaces (fully unit-tested without a live server).

Server-side follow-up for a clean submit: a token-authenticated sibling of POST /api/blocks/submit-version that resolves the API key→user, applies the same App-Blocks + (currently) moderator gates, reuses submitVersion unchanged, and returns { publishRequestId, slug, version, status }.

Build / test / quality

  • go build ./..., go test ./..., go vet ./..., gofmt -s -l .all clean.
  • 51 tests pass, 0 fail. New tests cover every ported check (sandbox escape + each disallowed token, page⇒iframe, required iframe sub-fields, renderMode tier gate, unknown/page-slot targets, long-name accept, outputDir leading-slash + traversal) and the example-manifests-validate-clean assertion.
  • Mutation-checked the 🔴 sandbox rule: stubbing sandboxChecks to return nil fails 4 reject tests (the accept test correctly still passes), then restored.
  • goreleaser config + CI in .github/workflows/ci.yml.

Remaining known fidelity gap

targets[].slotId against a vendored slot list (low priority — page-only launch) and the per-app origin/scope binding the CLI can't see. Durable fix = a server civitai app validate endpoint.

Do not merge — Phase 1 review.

🤖 Generated with Claude Code

ZacxDev and others added 2 commits June 18, 2026 09:41
Greenfield Go 1.25 CLI (Cobra + Viper) — single static `civitai` binary in the
gh/kubectl/stripe mold. First feature group: App Blocks authoring.

Commands:
- `civitai app init [name] [--template static|page-vite] [--from <slug>]` —
  scaffolds a ready-to-build block project from go:embed templates. `--from`
  stubbed with a clear "not yet wired" message (needs a server source endpoint).
- `civitai app validate [dir]` — validates block.manifest.json against the
  vendored JSON Schema (schema/app-block.manifest.schema.json, derived from the
  server validator) + structural checks; rejects dev-set iframe.src/trustTier.
- `civitai app submit [dir]` — validates + packages the canonical SOURCE tree
  (excludes .git/node_modules/dist), then uploads via a token-accepting endpoint
  when configured, else writes the .zip + prints exact next steps.
- `civitai login` / `civitai whoami` — token storage/verification via Viper
  (~/.config/civitai/config.yaml, chmod 600; CIVITAI_* env overrides).

Tooling: Makefile, goreleaser config (brew tap), GitHub Actions CI
(vet + gofmt + test + build). go build/test/vet all clean.

Submit/auth: the live upload route POST /api/blocks/submit-version is
session-cookie + moderator only (no token path), so a clean programmatic submit
needs a companion token-authenticated server endpoint — documented in the README
and the api package. Network/auth behind interfaces for testability.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…delity gap

The vendored JSON Schema only covered syntactic rules, so `validate`
green-lit manifests the server's approve-time BlockManifestValidator
rejects. Port the missing semantic checks into internal/validate, mirroring
how the CLI already rejects server-owned fields.

- Sandbox trust-tier allowlist (validator ~L175-206): validate against the
  unverified tier (server-forced at submit) — only allow-scripts/allow-forms;
  explicitly reject allow-same-origin+allow-scripts (sandbox escape) and any
  out-of-allowlist token (allow-popups, allow-top-navigation, ...).
- page ⇒ iframe required (~L504); renderMode=iframe ⇒ iframe required (~L377).
- iframe.minHeight + iframe.resizable required when an iframe block is present
  (~L387-415).
- renderMode inline/hybrid rejected for unverified (INLINE_REQUIRES_VERIFIED_TIER,
  ~L289).
- targets[].slotId registry membership + page-slot rejection (~L426-460), using
  a vendored copy of the 4 slot ids (documented sync TODO; durable fix = server
  validate endpoint).

False-INVALID fixes: drop the CLI's 128-char name cap (server only requires
non-empty); add a clear Go-side outputDir safe-relative-path check (leading-/
+ .. traversal), aligning the message with the server companion.

Config 0600 race (internal/config): replace viper WriteConfigAs+chmod (creates
0644 then chmods — brief world-readable window for the token) with an atomic
0600 temp-file + rename. Existing perms test still asserts 0600.

Honesty: add the two real example manifests (buzz-generator, notepad) under
examples/ + a test asserting they validate clean; reframe README + validate
help as a best-effort LOCAL pre-check mirroring the server validator, with the
server remaining source of truth and a server validate endpoint as the durable
direction.

go build/test/vet/gofmt -s all clean. 51 tests pass, 0 fail. Mutation-checked
the sandbox rule (removing sandboxChecks fails 4 reject tests).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ZacxDev
ZacxDev merged commit 0bd3174 into main Jun 18, 2026
1 check passed
ZacxDev added a commit that referenced this pull request Jun 25, 2026
Audit 🟡#1 (https/host enforcement). Asset download URLs come straight
out of the GitHub release JSON and were fetched with http.DefaultClient,
which follows redirects including https->http downgrades with no
scheme/host check. An http:// checksums.txt + http:// tarball pair would
make the SHA-256 gate self-referential (both halves attacker-controlled).

- validateAssetURL: require scheme==https AND host in an allowlist
  {github.com, objects.githubusercontent.com,
  release-assets.githubusercontent.com}; reject (abort, no download)
  otherwise. Applied to BOTH the tarball and checksums.txt URLs before
  any bytes are read.
- assetDownloadClient: dedicated *http.Client with CheckRedirect that
  re-validates every hop, so an https->http downgrade redirect is
  rejected rather than followed. Context timeouts + body-size caps kept.
- Checksum-verify-before-replace gate unchanged (defense-in-depth on
  transport).

Tests: validateAssetURL allow/reject table; an http:// asset URL and an
off-host https asset URL each abort before download with the binary
untouched; the asset client rejects an https->http redirect. httptest
fixtures now serve TLS and inject the loopback host via a test seam so
the production allowlist is never weakened to pass tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ZacxDev added a commit that referenced this pull request Jun 25, 2026
…-update (#39)

* feat(cli): cached non-blocking update notice + `civitai upgrade` self-update

Adds two related features in one PR:

PART 1 — daily, cached, non-blocking "new version available" notice
- A root PersistentPostRun hook prints at most ONE dim stderr line after any
  successful command. It never does a synchronous network call: it reads a
  cache (~/.config/civitai/update-check.json) and, when stale, spawns a
  DETACHED `civitai __update-check` (hidden subcommand) that fetches the latest
  release and rewrites the cache. The current run uses the cached value; the
  refresh lands for next time. First run (no cache) only kicks off the refresh.
- Refresh at most once / 24h (last_check); notice shown at most once / 24h
  (last_notified) even while behind. Corrupt/missing cache => empty, fail-silent.
- Suppressed when: stderr is not a TTY, CI env is set, --no-update-check /
  CIVITAI_NO_UPDATE_CHECK, or the command is version/upgrade/completion/help/
  __update-check/__complete*. Only fires when current parses and latest > current.
- stderr only — never pollutes stdout, so pipes/scripts are unaffected.
- Reuses the existing fetchLatestRelease / semver helpers from update_check.go;
  `version` keeps its own explicit synchronous check (no double-notify).

PART 2 — `civitai upgrade` self-update
- Resolves the latest release (unauthenticated GitHub, no token ever sent).
  Already >= latest and not --force => "already up to date" no-op.
- Homebrew detection: if the resolved executable lives under a brew path
  (/Cellar/, /Caskroom/, /opt/homebrew, /usr/local/Homebrew|Cellar,
  /home/linuxbrew/.linuxbrew), prints the brew upgrade command instead of
  self-replacing (--force overrides).
- Otherwise downloads the platform tarball + checksums.txt, VERIFIES the
  tarball SHA-256 against checksums.txt and ABORTS on mismatch (binary left
  untouched), extracts the binary, and atomically replaces the running
  executable via github.com/minio/selfupdate. Permission-denied => clear
  sudo/brew/go-install guidance, non-zero exit, no half-written binary.

Detaching uses a small build-tagged platform split (unix Setpgid /
windows CREATE_NEW_PROCESS_GROUP); the parent never Waits. Spawn + apply +
executable-path + TTY are behind injectable seams so tests assert behavior
without forking or replacing the test binary.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(upgrade): enforce https + GitHub host on asset downloads

Audit 🟡#1 (https/host enforcement). Asset download URLs come straight
out of the GitHub release JSON and were fetched with http.DefaultClient,
which follows redirects including https->http downgrades with no
scheme/host check. An http:// checksums.txt + http:// tarball pair would
make the SHA-256 gate self-referential (both halves attacker-controlled).

- validateAssetURL: require scheme==https AND host in an allowlist
  {github.com, objects.githubusercontent.com,
  release-assets.githubusercontent.com}; reject (abort, no download)
  otherwise. Applied to BOTH the tarball and checksums.txt URLs before
  any bytes are read.
- assetDownloadClient: dedicated *http.Client with CheckRedirect that
  re-validates every hop, so an https->http downgrade redirect is
  rejected rather than followed. Context timeouts + body-size caps kept.
- Checksum-verify-before-replace gate unchanged (defense-in-depth on
  transport).

Tests: validateAssetURL allow/reject table; an http:// asset URL and an
off-host https asset URL each abort before download with the binary
untouched; the asset client rejects an https->http redirect. httptest
fixtures now serve TLS and inject the loopback host via a test seam so
the production allowlist is never weakened to pass tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ZacxDev added a commit that referenced this pull request Aug 3, 2026
…s a to-do list

The app-analytics handoff has been sitting on this branch, unmerged, describing three open
PRs and a re-gate in flight. All of that shipped — 8 PRs across both repos, plus follow-ups
#1, #2 and #4 — so as written the doc's most prominent content is a set of live-sounding
directives for work that is done. A stranded doc that is also stale is worse than no doc.

Changes:

- A RESOLVED banner up top with the full merged-PR table, replacing "nothing merged".

- Neutralised the "Do not merge until that re-gate reports" imperative, and recorded that
  the rebase it anticipated WAS needed, for a different reason: #3566 landed later and
  edited the same `detail: {}` object, turning #3561 CONFLICTING after its gate had passed.

- Struck follow-ups #1, #2 and #4 with what actually happened, including the two places
  this doc was WRONG:
    * #1's entry missed a second proc with the identical live defect —
      `getMyForgejoCloneInfo`, which `civitai app pull` drives. It was found by an audit,
      not by the list, which is worth knowing about ranked follow-up lists in general: the
      list is not a survey.
    * #2's suggested fix (reuse `humaniseScopeEndpoint`) would have shipped a bug. Measured
      against the real function it returns '(no workflow id)' for `workflow:submit` and ''
      for `user-settings:write`, because it is the per-ROW labeller and an aggregate bucket
      has no `detail`.

- Recorded #1's scope decision with the prod evidence that later confirmed it: 331 live
  tokens unblocked, 30 of which lack bit 26 — so copying the nearest precedent
  (AppBlocksDevTunnel) would have left those 30 still 403ing. Plus the measurement trap:
  `(mask & Full) = Full` is also true when `mask == Full` and reports 145 false hits; the
  strict-superset form needs `AND mask <> Full`.

- New "Still open — start here" section: the CI `component`-tier gap (three PRs shipped
  browser tests that have never run on a canonical browser), the stale-node_modules trap
  that silently removes ~1,126 tests, the unverified `addCollaborator` downgrade lead, and
  a note that `installs: 0` should be assumed broken until a positive control exists.

- "What actually caught the bugs": across 12 adversarial audit rounds every fix round found
  a defect in the previous fix, and the mechanical gate caught none of them — the suite and
  typecheck were green at every tip.

Doc-only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant