Test using `@firebase/rules-unit-testing` with Firestore Emulator is failed

[k1350]

Operating System

macOS Sequoia 15.1.1 Apple M2 Pro

Environment (if applicable)

Node.js v20.14.0

Firebase SDK Version

11.0.1

Firebase SDK Product(s)

Firestore

Project Tooling

  "dependencies": {
    "firebase": "^11.0.1",
    "firebase-admin": "^12.7.0"
  },
  "devDependencies": {
    "@biomejs/biome": "^1.9.4",
    "@firebase/rules-unit-testing": "^4.0.1"
    "typescript": "^5.7.2",
    "vitest": "^2.1.5"
  },

% firebase --version
13.27.0

% java --version
java 23.0.1 2024-10-15
Java(TM) SE Runtime Environment (build 23.0.1+11-39)
Java HotSpot(TM) 64-Bit Server VM (build 23.0.1+11-39, mixed mode, sharing)

Detailed Problem Description

When running tests using the emulator, an error occurs with initializeTestEnvironment.

% npm run test:firestore

> test:firestore
> firebase emulators:exec --only firestore 'npm run vitest:firestore'

i  emulators: Starting emulators: firestore
i  firestore: Firestore Emulator logging to firestore-debug.log
✔  firestore: Firestore Emulator UI websocket is running on 9150.
i  Running script: npm run vitest:firestore

> vitest:firestore
> vitest run --project firestore


 RUN  v2.1.5 /Users/k/dev/functions-sample

 ❯ |firestore| tests/firestore/rules.test.ts (1) 392ms
   ❯ firestore.rules (1) 391ms
     ⠼ [ beforeAll ]
     ↓ sample test [skipped]
   ⠙ [ afterAll ]

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Failed Suites 2 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

 FAIL |firestore|  tests/firestore/rules.test.ts [ tests/firestore/rules.test.ts ]
TypeError: Cannot read properties of undefined (reading 'cleanup')
 ❯ tests/firestore/setup.ts:21:17
     19| 
     20| afterAll(async () => {
     21|   await testEnv.cleanup();
       |                 ^
     22| });
     23| 

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/2]⎯

 FAIL |firestore|  tests/firestore/rules.test.ts > firestore.rules
Error: {"error":{"code":503,"message":"Connection closing while performing protocol negotiation for [NettyClientHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0]","status":"UNAVAILABLE"}}
 ❯ loadFirestoreRules node_modules/@firebase/rules-unit-testing/src/impl/rules.ts:63:11
 ❯ Module.initializeTestEnvironment node_modules/@firebase/rules-unit-testing/src/initialize.ts:94:5
 ❯ Module.init tests/firestore/setup.ts:11:13
      9| 
     10| export async function init(projectId: string) {
     11|   testEnv = await initializeTestEnvironment({
       |             ^
     12|     projectId,
     13|     firestore: {
 ❯ tests/firestore/rules.test.ts:15:15

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/2]⎯

 Test Files  1 failed (1)
      Tests  1 skipped (1)
   Start at  18:40:00
   Duration  689ms (transform 11ms, setup 89ms, collect 12ms, tests 392ms, environment 0ms, prepare 38ms)

⚠  Script exited unsuccessfully (code 1)
i  emulators: Shutting down emulators.
i  firestore: Stopping Firestore Emulator
i  logging: Stopping Logging Emulator

Error: Script "npm run vitest:firestore" exited with code 1

firestore-debug.log

Nov 25, 2024 6:39:58 PM com.google.cloud.datastore.emulator.firestore.websocket.WebSocketServer start
INFO: Started WebSocket server on ws://127.0.0.1:9150
API endpoint: http://127.0.0.1:8080
If you are using a library that supports the FIRESTORE_EMULATOR_HOST environment variable, run:

   export FIRESTORE_EMULATOR_HOST=127.0.0.1:8080

If you are running a Firestore in Datastore Mode project, run:

   export DATASTORE_EMULATOR_HOST=127.0.0.1:8080

Note: Support for Datastore Mode is in preview. If you encounter any bugs please file at https://github.com/firebase/firebase-tools/issues.
Dev App Server is now running.

Nov 25, 2024 6:40:01 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
INFO: Detected non-HTTP/2 connection.
Nov 25, 2024 6:40:01 PM io.gapi.emulators.netty.HttpHandler$1 onError
INFO: Exception when handling request: UNAVAILABLE: Connection closing while performing protocol negotiation for [NettyClientHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0]
*** shutting down gRPC server since JVM is shutting down
*** server shut down

Steps and code to reproduce issue

minimal reproduction: https://github.com/k1350/functions-sample
The main branch uses an outdated version of the SDK, while the update-versions branch uses the latest version. Therefore, after cloning the repository, please switch the branch.
After npm install, please execute npm run test:firestore.

Since it works correctly on GitHub Actions (runs-on: ubuntu-latest), it might be an issue specific to macOS Sequoia or only my environment.

[google-oss-bot]

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

[DellaBitta]

Hi @k1350

I wasn't able to reproduce your original error with a slightly older version of the Firebase CLI and using the same JDK that you're using.

However, when I updated to the latest of Firebase CLI version I ran into the same error as this one reported in the Firebase Tool GitHub repo (the Firebase CLI is maintained by the firebase-tools team).

So there could be two things happening here.

1. There might be a problem in your environment, as you hinted to in your original post.
2. There might be an issue with the version of the Firebase CLI you're using and it might be worth posting to that thread, and downgrading your version or installing over npm.

I hope this helps!

[k1350]

@DellaBitta Thank you for looking into this.
After downgrading to Firebase CLI v13.26.0, I was able to execute tests without any issues.
Then I upgraded back to v13.27.0, it somehow started working correctly.
I'm not sure what caused the problem, but I will close this issue since it has been resolved.
Thank you.