Skip to content

Faster unit tests#7029

Open
gonzaloriestra wants to merge 9 commits intofaster-cifrom
faster-unit-tests
Open

Faster unit tests#7029
gonzaloriestra wants to merge 9 commits intofaster-cifrom
faster-unit-tests

Conversation

@gonzaloriestra
Copy link
Contributor

@gonzaloriestra gonzaloriestra commented Mar 17, 2026

WHY are these changes introduced?

Local unit test wall-clock time is ~67s, largely due to vitest overhead (fork pool, eager includeSource scanning, heavy setup imports) and unnecessary sleep/setTimeout delays scattered across tests.

WHAT is this pull request doing?

Reduces local unit test wall-clock time from 67s → 53s (21% faster).

With the improvements from both branches in the stack, CI time is reduced from ~10 min to less than 5 min (+50% faster).

17-41-4sgzv-2r0ga.png

Changes:

  • Switch vitest pool from forks to threads — saves ~12s by avoiding per-file process spawning
  • Remove includeSource config and extract the one in-source test into its own file — cuts ~90s off aggregate collect time
  • Replace eager zod import in vitest setup with lazy constructor.name check — faster test bootstrap
  • Use define in vite config for compile-time env vars (SHOPIFY_UNIT_TEST, FORCE_HYPERLINK, FORCE_COLOR)
  • Replace vitest.workspace.json with test.projects in root vite.config.ts
  • Reduce excessive sleep/setTimeout delays in tests (100ms → 10ms for waitForInputsToBeReady, sendInputAndWait, and various test timers)
  • Remove setImmediate/clearImmediate from fakeTimers (not available in threads pool)

How to test your changes?

pnpm test:unit

Measuring impact

How do we know this change was effective? Please choose one:

  • n/a - this doesn't need measurement, e.g. a linting rule or a bug-fix
  • Existing analytics will cater for this addition
  • PR includes analytics changes to measure impact

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows)
  • I've considered possible documentation changes

…94 tests\n\nResult: {"status":"keep","wall_clock_s":67.45,"collect_s":505.19,"tests_s":64.54,"transform_s":15.89,"setup_s":6.51,"prepare_s":23.01,"passed":3794,"files_passed":393}
…), collect dropped 70s\n\nResult: {"status":"keep","wall_clock_s":55.56,"collect_s":434.08,"tests_s":62.96,"transform_s":17.05,"setup_s":4.67,"prepare_s":23.11,"passed":3794,"files_passed":393}
…eline), collect down to 415s\n\nResult: {"status":"keep","wall_clock_s":54.01,"collect_s":415.38,"tests_s":62.36,"transform_s":17.15,"setup_s":5.31,"prepare_s":23.18,"passed":3794,"files_passed":393}
… 100→10ms, fix tcp.test.ts sleep, use waitForContent for polling test — 54.72s (−19%), tests_s 52.6s (−28%)\n\nResult: {"status":"keep","wall_clock_s":54.72,"collect_s":433.03,"tests_s":52.61,"transform_s":18.18,"setup_s":3.25,"prepare_s":23.74,"passed":3794,"files_passed":393}
… Tasks, AutocompletePrompt, http-reverse-proxy — 53.93s, tests_s 50.66s\n\nResult: {"status":"keep","wall_clock_s":53.93,"collect_s":426.8,"tests_s":50.66,"transform_s":17.95,"setup_s":3.5,"prepare_s":24.11,"passed":3794,"files_passed":393}
…s\n\nResult: {"status":"keep","wall_clock_s":53.52,"collect_s":424.63,"tests_s":48.87,"transform_s":17.23,"setup_s":3.59,"prepare_s":24.34,"passed":3794,"files_passed":393}
Copy link
Contributor Author

gonzaloriestra commented Mar 17, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@gonzaloriestra gonzaloriestra changed the title Baseline: 67.45s wall clock, 505s collect, 64.5s tests, 393 files, 3794 tests\n\nResult: {"status":"keep","wall_clock_s":67.45,"collect_s":505.19,"tests_s":64.54,"transform_s":15.89,"setup_s":6.51,"prepare_s":23.01,"passed":3794,"files_passed":393} Faster unit tests Mar 17, 2026
@gonzaloriestra gonzaloriestra mentioned this pull request Mar 17, 2026
5 tasks
@github-actions
Copy link
Contributor

github-actions bot commented Mar 17, 2026

Coverage report

St.
Category Percentage Covered / Total
🟢 Statements 81.94% 14619/17842
🟡 Branches 74.39% 7253/9750
🟢 Functions 81.06% 3710/4577
🟢 Lines 82.34% 13820/16784

Test suite run success

3806 tests passing in 1462 suites.

Report generated by 🧪jest coverage report action from efa0833

@gonzaloriestra gonzaloriestra force-pushed the faster-unit-tests branch 2 times, most recently from 3854b50 to a8b96c5 Compare March 17, 2026 12:53
@gonzaloriestra gonzaloriestra marked this pull request as ready for review March 17, 2026 15:02
@gonzaloriestra gonzaloriestra requested review from a team as code owners March 17, 2026 15:02
@github-actions
Copy link
Contributor

We detected some changes at packages/*/src and there are no updates in the .changeset.
If the changes are user-facing, run pnpm changeset add to track your changes and include them in the next release CHANGELOG.

Caution

DO NOT create changesets for features which you do not wish to be included in the public changelog of the next CLI release.

@github-actions
Copy link
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/public/node/api/rest-api-throttler.d.ts
@@ -13,6 +13,13 @@ export declare function throttle<T>(request: () => T): Promise<T>;
  * @param response - The response object.
  */
 export declare function updateApiCallLimitFromResponse(response: RestResponse): void;
+/**
+ * Extracts the retry delay in milliseconds from the response's  header.
+ *
+ * @param response - The response object.
+ * @returns The retry delay in milliseconds, or 0 if not present/invalid.
+ */
+export declare function extractRetryDelayMsFromResponse(response: RestResponse): number;
 /**
  * Retries an operation after a delay specified in the response headers.
  *
@@ -20,4 +27,11 @@ export declare function updateApiCallLimitFromResponse(response: RestResponse):
  * @param operation - The operation to retry.
  * @returns - The response of the operation.
  */
-export declare function delayAwareRetry(response: RestResponse, operation: () => Promise<RestResponse>): Promise<RestResponse>;
\ No newline at end of file
+export declare function delayAwareRetry(response: RestResponse, operation: () => Promise<RestResponse>): Promise<RestResponse>;
+/**
+ * Extracts the API call limit (used/total) from the response's  header.
+ *
+ * @param response - The response object.
+ * @returns A tuple of [used, limit], or undefined if the header is missing/invalid.
+ */
+export declare function extractApiCallLimitFromResponse(response: RestResponse): [number, number] | undefined;
\ No newline at end of file
packages/cli-kit/dist/public/node/themes/api.d.ts
@@ -5,7 +5,6 @@ export type ThemeParams = Partial<Pick<Theme, 'name' | 'role' | 'processing' | '
 export type AssetParams = Pick<ThemeAsset, 'key'> & Partial<Pick<ThemeAsset, 'value' | 'attachment'>>;
 export declare function fetchTheme(id: number, session: AdminSession): Promise<Theme | undefined>;
 export declare function fetchThemes(session: AdminSession): Promise<Theme[]>;
-export declare function findDevelopmentThemeByName(name: string, session: AdminSession): Promise<Theme | undefined>;
 export declare function themeCreate(params: ThemeParams, session: AdminSession): Promise<Theme | undefined>;
 export declare function fetchThemeAssets(id: number, filenames: Key[], session: AdminSession): Promise<ThemeAsset[]>;
 export declare function deleteThemeAssets(id: number, filenames: Key[], session: AdminSession): Promise<Result[]>;
packages/cli-kit/dist/public/node/themes/theme-manager.d.ts
@@ -8,8 +8,8 @@ export declare abstract class ThemeManager {
     protected abstract removeTheme(): void;
     protected abstract context: string;
     constructor(adminSession: AdminSession);
-    findOrCreate(name?: string, role?: Role): Promise<Theme>;
-    fetch(name?: string, role?: Role): Promise<Theme | undefined>;
+    findOrCreate(): Promise<Theme>;
+    fetch(): Promise<Theme | undefined>;
     generateThemeName(context: string): string;
     create(themeRole?: Role, themeName?: string): Promise<Theme>;
 }
\ No newline at end of file

Copy link
Contributor

@graygilmore graygilmore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it

Copy link
Contributor

@aswamy aswamy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love these kind of fixes ❤️ Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants