cloud-storage-rules-runtime crashes due to missing com/ibm/icu/text/UTF16$StringComparator

[ronba]

My apologies if this is not the right repository to file this in.
It looks like cloud-storage-rules-runtime isn't part of this repository but is being invoked by downloadableEmulators.ts.

[REQUIRED] Environment info

firebase-tools: 9.12.1

Platform: macOS

[REQUIRED] Test case

Create the following files:

Create a test file (this is the easiest way I've found to reproduce this so far), test.ts:

import * as rulesUnitTesting from '@firebase/rules-unit-testing';

const myApp = rulesUnitTesting.initializeTestApp({
  storageBucket: 'my-bucket',
});

describe('some sample tests', () => {
  it('should throw an exception', async () => {
    await myApp.storage().ref('folder/1.txt').putString('abcd');
    await myApp.storage().ref('folder/1.txt').getMetadata();
  });
});

storage.rules:

rules_version = '2';
service firebase.storage {
  match /b/{bucket}/o {
    match /{allPaths=**} {
      allow write: if true;
      allow read: if request.auth.token.keys().hasAny(resource.metadata.keys());
    }
  }
}

[REQUIRED] Steps to reproduce

1. Launch the firebase storage emulator.
2. Run the test file.

[REQUIRED] Expected behavior

The test completes without the storage emulator crashing.

[REQUIRED] Actual behavior

The storage emulator crashes with the following exception is thrown:

  some sample tests
⚠  Unexpected rules runtime output: Exception in thread "main" 
⚠  Unexpected rules runtime output: java.lang.NoClassDefFoundError: com/ibm/icu/text/UTF16$StringComparator
        at com.google.firebase.rules.runtime.common.UnicodeComparator.<clinit>(UnicodeComparator.java:20)
        at com.google.firebase.rules.runtime.impl.types.MapType$MapKeys.doExecute(MapType.java:216)
        at com.google.firebase.rules.runtime.utils.GuardedRuntimeFunction.lambda$execute$0(GuardedRuntimeFunction.java:28)
        at com.google.firebase.rules.runtime.utils.InvocationGuard.execute(InvocationGuard.java:36)
        at com.google.firebase.rules.runtime.utils.GuardedRuntimeFunction.execute(GuardedRuntimeFunction.java:27)
        at com.google.firebase.rules.runtime.impl.StackMachine.executeFunction(StackMachine.java:674)
        at com.google.firebase.rules.runtime.impl.StackMachine.callFunction(StackMachine.java:651)
        at com.google.firebase.rules.runtime.impl.StackMachine.evaluateCall(StackMachine.java:452)
        at com.google.firebase.rules.runtime.impl.StackMachine.evaluateExpression(StackMachine.java:240)
        at com.google.firebase.rules.runtime.impl.StackMachine.evaluateNextExpression(StackMachine.java:209)

⚠  Unexpected rules runtime output:     at com.google.firebase.rules.runtime.impl.StackMachine.evaluate(StackMachine.java:166)
        at com.google.firebase.rules.runtime.impl.IterativeInterpreter.evaluate(IterativeInterpreter.java:33)
        at com.google.firebase.rules.runtime.impl.Interpreter.interpret(Interpreter.java:28)
        at com.google.firebase.rules.runtime.impl.DefaultEvaluator.evaluate(DefaultEvaluator.java:130)
        at com.google.firebase.rules.tools.local.server.EmulatorRuleClient$EmulatorRuleEvaluator.evaluate(EmulatorRuleClient.java:76)
        at com.google.firebase.rules.tools.local.server.EmulatorRuleClient$EmulatorRuleEvaluator.evaluate(EmulatorRuleClient.java:64)
        at com.google.firebase.rules.tools.local.server.ServerActionVerify.perform(ServerActionVerify.java:107)
        at com.google.firebase.rules.tools.local.server.Server.performActionFromLine(Server.java:79)
        at com.google.firebase.rules.tools.local.server.Server.handleRequest(Server.java:46)
        at com.google.firebase.rules.tools.local.server.Server.start(Server.java:36)
        at com.google.firebase.rules.tools.local.FirebaseRulesTooling.main(FirebaseRulesTooling.java:83)

⚠  Unexpected rules runtime output: Caused by: java.lang.ClassNotFoundException: com.ibm.icu.text.UTF16$StringComparator
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)

⚠  Unexpected rules runtime output:     ... 21 more

Workaround:

Not really a workaround but for now I manually added com/ibm/icu/text/UTF16$StringComparator to the java class path using these steps:

1. Download the jar containing java.lang.NoClassDefFoundError: com/ibm/icu/text/UTF16$StringComparator, for example: https://mvnrepository.com/artifact/com.ibm.icu/icu4j/69.1

2. Place it in a library that's easily accessible, for example $HOME/jars/

3. Open downloadableEmulators.js from firebase-tools, it should be here: node_modules/firebase-tools/lib/emulator/downloadableEmulators.js

4. Find the entry for storage

5. Edit it so it looks like:
   storage: {
   binary: "java",
   args: [
   "-classpath",
   `$DIRECTORY_FROM_STEP_1/icu4j-69.1.jar:${getExecPath(types_1.Emulators.STORAGE)}`,
   "-Duser.language=en",
   "com.google.firebase.rules.tools.local.FirebaseRulesTooling",
   "serve",
   ],
   ...

6. Run the emulator

[samtstern]

@ronba thanks for the detailed report and the workaround! This class comes from the icu4j library which should be included in our .jar. @abeisgoat can investigate.

[abeisgoat]

Hey there, to add some context here, we are really aggressive about stripping unused code from our java jars, so we were just overly aggressive and stripped out something we needed. I'll take a look and see about white listing this. Thanks for the report.

[Andrew-Bekhiet]

The issue is still there (aggressively stripping libraries) when using this rule:

allow get: if request.auth != null
  && (('someClaim' in request.auth.token.keys()
  && request.auth.token.someClaim in resource.metadata.values()
  )
  ||
  ('otherClaim' in request.auth.token.keys()
    && request.auth.token.otherClaim.size() > 0
    && request.auth.token.otherClaim[0] in resource.metadata.values()
   ));

Throws similar exception:

!  Unexpected rules runtime error: Exception in thread "main"
!  Unexpected rules runtime error: java.lang.NoClassDefFoundError: com/ibm/icu/impl/Utility
        at com.ibm.icu.text.UTF16$StringComparator.compare(UTF16.java:2483)
        at com.ibm.icu.text.UTF16$StringComparator.compare(UTF16.java:2336)
        at java.util.TimSort.countRunAndMakeAscending(Unknown Source)
        at java.util.TimSort.sort(Unknown Source)
        at java.util.Arrays.sort(Unknown Source)
        at com.google.common.collect.ImmutableList.sortedCopyOf(ImmutableList.java:372)
        at com.google.firebase.rules.runtime.impl.types.MapType$MapKeys.doExecute(MapType.java:217)
        at com.google.firebase.rules.runtime.utils.GuardedRuntimeFunction.lambda$execute$0(GuardedRuntimeFunction.java:28)
        at com.google.firebase.rules.runtime.utils.GuardedRuntimeFunction.execute(GuardedRuntimeFunction.java:27)
        at com.google.firebase.rules.runtime.impl.StackMachine.executeFunction(StackMachine.java:674)
        at com.google.firebase.rules.runtime.impl.StackMachine.callFunction(StackMachine.java:651)
        at com.google.firebase.rules.runtime.impl.StackMachine.evaluateCall(StackMachine.java:452)
        at com.google.firebase.rules.runtime.impl.StackMachine.evaluateExpression(StackMachine.java:240)
        at com.google.firebase.rules.runtime.impl.StackMachine.evaluateNextExpression(StackMachine.java:209)
        at com.google.firebase.rules.runtime.impl.StackMachine.evaluate(StackMachine.java:166)
        at com.google.firebase.rules.runtime.impl.IterativeInterpreter.evaluate(IterativeInterpreter.java:33)
        at com.google.firebase.rules.runtime.impl.Interpreter.interpret(Interpreter.java:28)
        at com.google.firebase.rules.runtime.impl.DefaultEvaluator.evaluate(DefaultEvaluator.java:130)
        at com.google.firebase.rules.tools.local.server.EmulatorRuleClient$EmulatorRuleEvaluator.evaluate(EmulatorRuleClient.java:76)
        at com.google.firebase.rules.tools.local.server.EmulatorRuleClient$EmulatorRuleEvaluator.evaluate(EmulatorRuleClient.java:64)
        at com.google.firebase.rules.tools.local.server.ServerActionVerify.perform(ServerActionVerify.java:129)
        at com.google.firebase.rules.tools.local.server.Server.performActionFromLine(Server.java:79)
        at com.google.firebase.rules.tools.local.server.Server.handleRequest(Server.java:46)
        at com.google.firebase.rules.tools.local.server.Server.start(Server.java:36)
        at com.google.firebase.rules.tools.local.FirebaseRulesTooling.main(FirebaseRulesTooling.java:83)

!  Unexpected rules runtime error: Caused by: java.lang.ClassNotFoundException: com.ibm.icu.impl.Utility
        at java.net.URLClassLoader.findClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        ... 26 more


Error: Storage Emulator Rules runtime exited unexpectedly.

CLI Version 9.21.0 and 9.22.0

[sikado]

Hi,

Issue is still there (on CLI 10.0.1 - MacOS) with this set of rules:

rules_version = '2';
service firebase.storage {
  match /b/{bucket}/o {
    match /In/{country} {
      match /finImport/{ImportFileName} {
        allow write: if request.resource.metadata.keys().hasAll(['country'])
      }
    }
  }
}

Unexpected rules runtime error: Exception in thread "main" 
⚠  Unexpected rules runtime error: java.lang.NoClassDefFoundError: com/ibm/icu/impl/Utility
        at com.ibm.icu.text.UTF16$StringComparator.compare(UTF16.java:2483)
        at com.ibm.icu.text.UTF16$StringComparator.compare(UTF16.java:2336)
        at java.base/java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
        at java.base/java.util.TimSort.sort(TimSort.java:220)
        at java.base/java.util.Arrays.sort(Arrays.java:1232)
        at com.google.common.collect.ImmutableList.sortedCopyOf(ImmutableList.java:372)
        at com.google.firebase.rules.runtime.impl.types.MapType$MapKeys.doExecute(MapType.java:217)
        at com.google.firebase.rules.runtime.utils.GuardedRuntimeFunction.lambda$execute$0(GuardedRuntimeFunction.java:28)
        at com.google.firebase.rules.runtime.utils.InvocationGuard.execute(InvocationGuard.java:36)
        at com.google.firebase.rules.runtime.utils.GuardedRuntimeFunction.execute(GuardedRuntimeFunction.java:27)
        at com.google.firebase.rules.runtime.impl.StackMachine.executeFunction(StackMachine.java:674)
        at com.google.firebase.rules.runtime.impl.StackMachine.callFunction(StackMachine.java:651)
        at com.google.firebase.rules.runtime.impl.StackMachine.evaluateCall(StackMachine.java:452)
        at com.google.firebase.rules.runtime.impl.StackMachine.evaluateExpression(StackMachine.java:240)
        at com.google.firebase.rules.runtime.impl.StackMachine.evaluateNextExpression(StackMachine.java:209)
        at com.google.firebase.rules.runtime.impl.StackMachine.evaluate(StackMachine.java:166)
        at com.google.firebase.rules.runtime.impl.IterativeInterpreter.evaluate(IterativeInterpreter.java:33)
        at com.google.firebase.rules.runtime.impl.Interpreter.interpret(Interpreter.java:28)
        at com.google.firebase.rules.runtime.impl.DefaultEvaluator.evaluate(DefaultEvaluator.java:130)
        at com.google.firebase.rules.tools.local.server.EmulatorRuleClient$EmulatorRuleEvaluator.evaluate(EmulatorRuleClient.java:76)
        at com.google.firebase.rules.tools.local.server.EmulatorRuleClient$EmulatorRuleEvaluator.evaluate(EmulatorRuleClient.java:64)
        at com.google.firebase.rules.tools.local.server.ServerActionVerify.perform(ServerActionVerify.java:129)
        at com.google.firebase.rules.tools.local.server.Server.performActionFromLine(Server.java:79)
        at com.google.firebase.rules.tools.local.server.Server.handleRequest(Server.java:46)
        at com.google.firebase.rules.tools.local.server.Server.start(Server.java:36)
        at com.google.firebase.rules.tools.local.FirebaseRulesTooling.main(FirebaseRulesTooling.java:83)

⚠  Unexpected rules runtime error: Caused by: java.lang.ClassNotFoundException: com.ibm.icu.impl.Utility
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
        ... 26 more

[ronba]

@sikado not sure if helpful but while waiting for the fix I've been working around this by:

1. Installed firebase-tools to the root of my project.
2. Downloaded the missing jar file and placed it in a patches directory at the root of my project.
3. Created patches/firebase-tools+10.1.2.patch with the content (a) below.

Then I used patch-package to get the patch installed.

If you install firebase-tools to the root of the project you'll need to either use npx to call firebase emulators (so npx firebase emulators ...) or call it from node_modules/.bin/firebase.

(a): patches/firebase-tools+10.1.2.patch

diff --git a/node_modules/firebase-tools/lib/emulator/downloadableEmulators.js b/node_modules/firebase-tools/lib/emulator/downloadableEmulators.js
index 9ecb15c..299fb56 100644
--- a/node_modules/firebase-tools/lib/emulator/downloadableEmulators.js
+++ b/node_modules/firebase-tools/lib/emulator/downloadableEmulators.js
@@ -131,9 +131,10 @@ const Commands = {
     storage: {
         binary: "java",
         args: [
-            "-jar",
+            "-classpath",
+            [path.join(process.cwd(),'patches','icu4j-70.1.jar'), getExecPath(types_1.Emulators.STORAGE)].join(path.delimiter),
             "-Duser.language=en",
-            getExecPath(types_1.Emulators.STORAGE),
+            "com.google.firebase.rules.tools.local.FirebaseRulesTooling",
             "serve",
         ],
         optionalArgs: [],

• edit: changed .join(':') to .join(path.delimiter) for windows support.