ci: split e2e workflow into PR and scheduled runs by binoy14 · Pull Request #995 · sanity-io/cli

binoy14 · 2026-04-21T20:57:03Z

Description

Split the E2E workflow into two separate workflows:

e2e.yml — runs on PRs and pushes to main in pack mode (builds from source). Removed workflow_dispatch trigger, registry mode steps, and Slack failure notifications.
e2e-scheduled.yml — runs hourly via cron and on manual dispatch in registry mode (installs sanity@latest from npm). Includes Slack failure notifications.

Updated release.yml to dispatch e2e-scheduled.yml with latest instead of passing the @sanity/cli version (which doesn't match the sanity package version on npm).

Closes SDK-1294

What to review

e2e-scheduled.yml — new workflow with hourly cron, workflow_dispatch, registry mode, and Slack notifications
e2e.yml — simplified to only handle PR/push in pack mode
release.yml — post-release dispatch updated to target e2e-scheduled.yml with latest

Testing

CI workflow changes — will be validated by GitHub Actions on merge. The scheduled workflow can also be tested manually via gh workflow run e2e-scheduled.yml.

Note

Low Risk
Low risk since changes are limited to GitHub Actions wiring, though misconfiguration could reduce E2E coverage or alter when failures get reported.

Overview
Splits E2E coverage into two workflows. e2e.yml is simplified to only run on PRs/pushes, always building the CLI from source (pack mode), and it removes workflow_dispatch, npm/registry install steps, and Slack failure notifications.

Adds a new scheduled/manual registry-mode run. New e2e-scheduled.yml runs hourly (and via workflow_dispatch) across Node 20/22/24 by installing sanity@<cli_version> from npm, building @sanity/cli-test, running E2Es, and sending a Slack alert on failures.

Adjusts post-release triggering. release.yml now dispatches e2e-scheduled.yml with cli_version=latest when @sanity/cli was published, instead of trying to pass a package version through e2e.yml.

^{Reviewed by Cursor Bugbot for commit f60f54f. Bugbot is set up for automated code reviews on this repo. Configure here.}

Move registry-mode e2e tests and Slack failure notifications into a dedicated hourly scheduled workflow. The PR/push workflow now only runs in pack mode. Post-release dispatch in release.yml updated to target the new scheduled workflow with `latest` instead of a specific version. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

claude · 2026-04-21T20:57:23Z

Claude finished @binoy14's task in 2m 41s —— View job

Review

Read changed workflow files
Check for changeset requirement
Provide review feedback

Changeset: Not required — CI config only, no runtime package changes. ✓

`e2e-scheduled.yml`

cancel-in-progress: true can swallow Slack alerts (e2e-scheduled.yml:15): The notify-failure job uses if: failure(), which does not fire when a run is cancelled. If a run is actively failing but a new run arrives (from cron overlap or a release.yml trigger) before the notify-failure job executes, the cancellation silently drops the alert. With a 20-min timeout and hourly schedule, true overlaps are unlikely — but the release.yml post-release trigger can arrive mid-hour and cancel an in-progress cron run.

Options:

Use cancel-in-progress: false so scheduled runs always complete (and always notify on failure).
Or add if: failure() || cancelled() to notify-failure so cancellations also send a Slack alert.

create-sanity is always @latest regardless of cli_version (e2e-scheduled.yml:42):

npm install --prefix "$INSTALL_DIR" "sanity@${CLI_VERSION}" "create-sanity@latest"

When a specific version is passed via workflow_dispatch (e.g. 5.19.0), create-sanity@latest may not correspond to that release. This is inherited from the original e2e.yml, but it's worth resolving here since workflow_dispatch is the primary entry point for version-specific testing. Fix: use create-sanity@${CLI_VERSION} (assuming versions are published in sync).

`release.yml`

No issues. Switching to latest instead of the @sanity/cli version is the correct fix — sanity and @sanity/cli share a version but only sanity is what gets installed from npm.

`e2e.yml`

No issues. Removal of workflow_dispatch and registry-mode steps is clean and consistent with the split.

github-actions · 2026-04-21T20:57:35Z

📦 Bundle Stats — `@sanity/cli`

Compared against main (511eb5cf)

`@sanity/cli`

Metric	Value	vs main (`511eb5c`)
Internal (raw)	2.1 KB	-
Internal (gzip)	799 B	-
Bundled (raw)	10.94 MB	-
Bundled (gzip)	2.05 MB	-
Import time	849ms	-1ms, -0.1%

`bin:sanity`

Metric	Value	vs main (`511eb5c`)
Internal (raw)	975 B	-
Internal (gzip)	460 B	-
Bundled (raw)	9.84 MB	-
Bundled (gzip)	1.77 MB	-
Import time	2.02s	-28ms, -1.3%

🗺️ View treemap · Artifacts

Details

Import time regressions over 10% are flagged with ⚠️
Sizes shown as raw / gzip 🗜️. Internal bytes = own code only. Total bytes = with all dependencies. Import time = Node.js cold-start median.

📦 Bundle Stats — `@sanity/cli-core`

Compared against main (511eb5cf)

Metric	Value	vs main (`511eb5c`)
Internal (raw)	93.8 KB	-
Internal (gzip)	21.9 KB	-
Bundled (raw)	21.62 MB	-
Bundled (gzip)	3.42 MB	-
Import time	810ms	-4ms, -0.5%

🗺️ View treemap · Artifacts

Details

Import time regressions over 10% are flagged with ⚠️
Sizes shown as raw / gzip 🗜️. Internal bytes = own code only. Total bytes = with all dependencies. Import time = Node.js cold-start median.

📦 Bundle Stats — `create-sanity`

Compared against main (511eb5cf)

Metric	Value	vs main (`511eb5c`)
Internal (raw)	976 B	-
Internal (gzip)	507 B	-
Bundled (raw)	50.7 KB	-
Bundled (gzip)	12.6 KB	-
Import time	❌ ChildProcess denied: node	-

Details

Import time regressions over 10% are flagged with ⚠️
Sizes shown as raw / gzip 🗜️. Internal bytes = own code only. Total bytes = with all dependencies. Import time = Node.js cold-start median.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit f60f54f. Configure here.}

cursor · 2026-04-21T21:20:20Z

+
+concurrency:
+  group: ${{ github.workflow }}
+  cancel-in-progress: true


Shared concurrency group can cancel post-release E2E runs

Low Severity

The concurrency group ${{ github.workflow }} with cancel-in-progress: true is shared across both schedule and workflow_dispatch triggers. When release.yml dispatches this workflow after a release, the in-progress post-release run can be silently cancelled by the next hourly cron trigger (and vice versa). Cancelled runs don't trigger the notify-failure job, so a post-release validation failure could go unnoticed. Including the event name in the group (e.g., ${{ github.workflow }}-${{ github.event_name }}) would isolate the two trigger types.

Additional Locations (1)

.github/workflows/release.yml#L117-L118

^{Reviewed by Cursor Bugbot for commit f60f54f. Configure here.}

* fix(cli): replace preferred-pm with local implementation (#984) * fix(cli): replace preferred-pm with local implementation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): address review feedback for preferredPm Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): detect pnpm-workspace.yaml in parent directory walk-up Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): normalize path separators for workspace glob matching on windows Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): continue workspace walk-up when inner workspace globs don't match When a nested package.json has a workspaces field whose globs don't match the target path, continue traversing parent directories instead of returning null. Also simplify error handling in detectFromNodeModules to swallow all read errors for best-effort detection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): remove continue that caused infinite loop in workspace walk-up The continue statement skipped dir = path.dirname(dir) at the bottom of the while loop, causing an infinite loop when a workspace's globs didn't match the target path. Removing it lets the code fall through to the directory advancement naturally. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): skip chmod-based permission test on Windows chmod doesn't enforce read permissions on Windows, so the file remains readable and the test returns 'pnpm' instead of falling through to 'npm'. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): remove circular import and add malformed package.json test Address PR review feedback: inline DetectablePackageManager type to break circular dependency between preferredPm and packageManagerChoice, and add test covering malformed package.json handling in workspace walk-up. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: split e2e workflow into PR and scheduled runs (#995) Move registry-mode e2e tests and Slack failure notifications into a dedicated hourly scheduled workflow. The PR/push workflow now only runs in pack mode. Post-release dispatch in release.yml updated to target the new scheduled workflow with `latest` instead of a specific version. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(cli): replace default namespace imports with named imports (#993) Use named imports for semver, nock, and typescript to resolve 167 import-x/no-named-as-default-member lint warnings. Also replace one `as any` with a proper type cast. CJS modules (json5, dotenv, tar-fs, tar-stream) are left unchanged since they don't support named ESM imports at runtime. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): default to TypeScript in unattended mode for sanity init (#1004) * fix(cli): default to TypeScript in unattended mode for sanity init When running sanity init in unattended mode (non-interactive terminal or --yes flag), the CLI now defaults to TypeScript when --typescript or --no-typescript is not explicitly provided, matching the interactive prompt default. Uses the existing flagOrDefault/shouldPrompt helpers already used by initNextJs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update auto-generated changeset for PR #1004 --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: squiggler-app[bot] <265501495+squiggler-app[bot]@users.noreply.github.com> * refactor(cli): extract init command into framework-agnostic initAction (#999) * refactor(cli): extract init command body into framework-agnostic initAction Moves ~1100 lines of logic out of the oclif InitCommand into actions/init/initAction.ts, leaving init.ts as a thin wrapper that maps oclif flags → InitOptions and catches InitError. Flow files (initApp, initStudio, initNextJs, scaffoldTemplate) now accept `options: InitOptions` instead of individual flag params, and throw InitError directly instead of receiving an `error` callback. Zero test assertion changes — all 14 init test files pass as-is. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(cli): clean up init telemetry types - Make `visibility` optional in `CreateOrSelectDatasetStep` (can be undefined in unattended mode) - Remove dead `ConfigureAppProjectStep` interface (no trace.log call exists for it) - Remove unnecessary `as 'private' | 'public'` cast now that the type accepts undefined Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review feedback — restore telemetry, fix env path, add tests - Restore configureAppProject trace calls dropped during extraction (thread `trace` param into promptForAppTemplateSetup) - Replace `throw new InitError('', 0)` with plain `return` on --env path (it's a normal success path, not an error) - Remove exitCode===0 special-case from command wrapper - Add ConfigureAppProjectStep to telemetry union type - Port 4 new test files from #845: flagsToInitOptions, initAction, initError, init.command - Update 2 existing tests: --env path no longer throws Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(cli): close remaining oclif decoupling gaps in init flow files - bootstrapTemplate: make overwriteFiles/useTypeScript optional - initHelpers: use InitContext['output'] instead of Output - initNextJs: throw InitError instead of output.error(), pass options directly - initStudio: dynamic import for ImportDatasetCommand, use workDir over process.cwd() - initApp: use workDir over process.cwd() - scaffoldTemplate: remove as boolean casts and unnecessary try/catch Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review feedback — clean up test names, static import, tighten useTypeScript type - Rename describe blocks to simpler names (mcpMode resolution, error handling) - Use static import for InitError instead of dynamic import - Make useTypeScript a required boolean since flagOrDefault always returns boolean Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: update selectTemplate tests for refactored options signature Tests from #1004 used the old flat-argument signature; wrap in options object to match the current selectTemplate interface. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(cli): split initAction.ts into plan/ and project/ modules Extract helper functions from the 1175-line initAction.ts into one-function-per-file modules under plan/ and project/ subdirectories. initAction.ts now contains only the main orchestrator, flag validation, and auth — down to ~300 lines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: forward datasetDefault to app template dataset creation The refactor from class methods to standalone functions dropped the implicit this.flags['dataset-default'] read that getOrCreateDataset relied on. Thread datasetDefault through getProjectDetails → promptForAppTemplateSetup → getOrCreateDataset so --dataset-default applies to app templates again. Adds a regression test exercising the app template + --dataset-default code path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Binoy Patel <me@binoy.io> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: squiggler-app[bot] <265501495+squiggler-app[bot]@users.noreply.github.com> Co-authored-by: Matthew Dent <mttdnt@gmail.com>

binoy14 marked this pull request as ready for review April 21, 2026 21:12

binoy14 requested a review from a team as a code owner April 21, 2026 21:12

binoy14 requested review from mttdnt and rexxars and removed request for a team April 21, 2026 21:12

cursor Bot reviewed Apr 21, 2026

View reviewed changes

mttdnt approved these changes Apr 23, 2026

View reviewed changes

binoy14 merged commit 2d7b3bd into main Apr 23, 2026
25 checks passed

binoy14 deleted the feat/SDK-1294-e2e-scheduled-workflow branch April 23, 2026 14:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: split e2e workflow into PR and scheduled runs#995

ci: split e2e workflow into PR and scheduled runs#995
binoy14 merged 1 commit intomainfrom
feat/SDK-1294-e2e-scheduled-workflow

binoy14 commented Apr 21, 2026 •

edited by cursor Bot

Loading

Uh oh!

claude Bot commented Apr 21, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Apr 21, 2026 •

edited

Loading

Uh oh!

cursor Bot left a comment

Uh oh!

cursor Bot Apr 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

binoy14 commented Apr 21, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

What to review

Testing

Uh oh!

claude Bot commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review

e2e-scheduled.yml

release.yml

e2e.yml

Uh oh!

github-actions Bot commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📦 Bundle Stats — @sanity/cli

@sanity/cli

bin:sanity

📦 Bundle Stats — @sanity/cli-core

📦 Bundle Stats — create-sanity

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Apr 21, 2026

Choose a reason for hiding this comment

Shared concurrency group can cancel post-release E2E runs

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

binoy14 commented Apr 21, 2026 •

edited by cursor Bot

Loading

claude Bot commented Apr 21, 2026 •

edited

Loading

`e2e-scheduled.yml`

`release.yml`

`e2e.yml`

github-actions Bot commented Apr 21, 2026 •

edited

Loading

📦 Bundle Stats — `@sanity/cli`

`@sanity/cli`

`bin:sanity`

📦 Bundle Stats — `@sanity/cli-core`

📦 Bundle Stats — `create-sanity`