ci(test): enable turbopack test (#49466)

<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:

## For Contributors

### Improving Documentation or adding/fixing Examples

- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md

### Fixing a bug

- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

### Adding a feature

- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md



## For Maintainers

- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change



### Why?

### How?

Closes NEXT-
Fixes #

-->

### What?

closees WEB-1019.

As discussed, this PR enables running a set of next.js integration tests
with turbopack as blocking CI check. The tests listed as `enabled` is
considered as stable, that should not fail in any case except upstream
test flakyness.

Current approach is simple as we have only small set of tests and does
not consider it becomes a bottleneck. Adding a test under
https://github.com/vercel/next.js/pull/49466/files#diff-fdc68d6039dbe36607cda253b318cd3598a09a94ccb8994f3695aa1d9e1e404bR2
will makes run with turbopack in the CI. in the future, I'll revise test
suite itself so runner, or test case can mark & execute test with
existing test group / scheduling.

.github/workflows/build_test_deploy.yml

@@ -822,16 +822,12 @@ jobs:
       TEST_TIMINGS_TOKEN: ${{ secrets.TEST_TIMINGS_TOKEN }}
       # Enabling backtrace will makes snapshot tests fail
       RUST_BACKTRACE: 0
-      # Path to the custom next-swc bindings located in **docker container** image.
+      # Path to the custom next-swc bindings.
       NEXT_BINDINGS_BIN: /work/packages/next-swc/native/next-swc.linux-x64-gnu.node
       # Glob pattern to run specific tests with --turbo.
       NEXT_DEV_TEST_GLOB: '*'
-      # List of test files to run with turbopack as blocking CI check.
-      # [TODO]: as list grows we should consider different way to manage this.
-      TEST_FILES_LIST: |
-        test/development/acceptance-app/dynamic-error.test.ts \
-        test/development/acceptance-app/unsupported-app-features.test.ts \
-        test/development/acceptance-app/ReactRefresh.test.ts
+      NEXT_EXTERNAL_TESTS_FILTERS: /work/packages/next-swc/crates/next-dev-tests/tests-manifest.json
+
     strategy:
       fail-fast: false
     steps:
@@ -855,7 +851,7 @@ jobs:
           name: next-swc-test-binary
           path: packages/next-swc/native
 
-      - run: docker run --rm -v $(pwd):/work mcr.microsoft.com/playwright:v1.28.1-jammy /bin/bash -c "cd /work && NODE_VERSION=${{ env.NODE_LTS_VERSION }} ./scripts/setup-node.sh && node -v && npm i -g pnpm@${PNPM_VERSION} > /dev/null && __INTERNAL_NEXT_DEV_TEST_TURBO_DEV=TRUE __INTERNAL_CUSTOM_TURBOPACK_BINDINGS=${NEXT_BINDINGS_BIN} __INTERNAL_NEXT_DEV_TEST_TURBO_GLOB_MATCH=${NEXT_DEV_TEST_GLOB} NEXT_E2E_TEST_TIMEOUT=240000 NEXT_TEST_JOB=1 NEXT_TEST_MODE=dev TEST_TIMINGS_TOKEN=${{ secrets.TEST_TIMINGS_TOKEN }} xvfb-run node run-tests.js --type development --timings -c 1 $TEST_FILES_LIST >> /proc/1/fd/1"
+      - run: docker run --rm -v $(pwd):/work mcr.microsoft.com/playwright:v1.28.1-jammy /bin/bash -c "cd /work && NODE_VERSION=${{ env.NODE_LTS_VERSION }} ./scripts/setup-node.sh && node -v && npm i -g pnpm@${PNPM_VERSION} > /dev/null && NEXT_EXTERNAL_TESTS_FILTERS=${NEXT_EXTERNAL_TESTS_FILTERS} __INTERNAL_NEXT_DEV_TEST_TURBO_DEV=TRUE __INTERNAL_CUSTOM_TURBOPACK_BINDINGS=${NEXT_BINDINGS_BIN} __INTERNAL_NEXT_DEV_TEST_TURBO_GLOB_MATCH=${NEXT_DEV_TEST_GLOB} NEXT_E2E_TEST_TIMEOUT=240000 NEXT_TEST_JOB=1 NEXT_TEST_MODE=dev TEST_TIMINGS_TOKEN=${{ secrets.TEST_TIMINGS_TOKEN }} xvfb-run node run-tests.js --type development --timings -c 1 >> /proc/1/fd/1"
         name: Run test/development
         if: ${{needs.build.outputs.docsChange == 'nope'}}
 

packages/next-swc/crates/next-dev-tests/tests-manifest.json

@@ -0,0 +1,6 @@
+[
+  "test/development/acceptance-app/dynamic-error.test.ts",
+  "test/development/basic/legacy-decorators.test.ts",
+  "test/integration/plugin-mdx-rs/test/index.test.js",
+  "test/e2e/app-dir/underscore-ignore-app-paths/underscore-ignore-app-paths.test.ts"
+]

run-tests.js

@@ -12,6 +12,11 @@ const { createNextInstall } = require('./test/lib/create-next-install')
 const glob = promisify(_glob)
 const exec = promisify(execOrig)
 
+// Try to read an external array-based json to filter tests to be executed.
+// If process.argv contains a test to be executed, this'll append it to the list.
+const externalTestsFilterLists = process.env.NEXT_EXTERNAL_TESTS_FILTERS
+  ? require(process.env.NEXT_EXTERNAL_TESTS_FILTERS)
+  : []
 const timings = []
 const DEFAULT_NUM_RETRIES = os.platform() === 'win32' ? 2 : 1
 const DEFAULT_CONCURRENCY = 2
@@ -138,7 +143,9 @@ async function main() {
 
   console.log('Running tests with concurrency:', concurrency)
 
-  let tests = process.argv.filter((arg) => arg.match(/\.test\.(js|ts|tsx)/))
+  let tests = process.argv
+    .filter((arg) => arg.match(/\.test\.(js|ts|tsx)/))
+    .concat(externalTestsFilterLists)
   let prevTimings
 
   if (tests.length === 0) {