Firestore and Google speech to text- Duplicate class with :mergeDexRelease failure

[savanakbari]

Issue

We are using "@react-native-firebase/firestore": "^6.7.1", and google-stt (speech to text) in React native app and while building .apk, it is complaining about protolite-well-known-types
which is used in both package.

/Users/WebstormProjects/App/android/google-stt/build/.transforms/854148fb8211844869a1b1d33f9420b2/classes/classes.dex: D8: Type com.google.api.AnnotationsProto is defined multiple times: /Users/WebstormProjects/App/android/google-stt/build/.transforms/854148fb8211844869a1b1d33f9420b2/classes/classes.dex, /Users/WebstormProjects/App/android/app/build/intermediates/external_libs_dex/release/out/classes2.dex
com.android.builder.dexing.DexArchiveMergerException: Error while merging dex archives:
Learn how to resolve the issue at https://developer.android.com/studio/build/dependencies#duplicate_classes.
Type com.google.api.AnnotationsProto is defined multiple times: /Users/WebstormProjects/App/android/google-stt/build/.transforms/854148fb8211844869a1b1d33f9420b2/classes/classes.dex, /Users/WebstormProjects/App/android/app/build/intermediates/external_libs_dex/release/out/classes2.dex
at com.android.builder.dexing.D8DexArchiveMerger.getExceptionToRethrow(D8DexArchiveMerger.java:132)
at com.android.builder.dexing.D8DexArchiveMerger.mergeDexArchives(D8DexArchiveMerger.java:119)
at com.android.build.gradle.internal.transforms.DexMergerTransformCallable.call(DexMergerTransformCallable.java:102)
at com.android.build.gradle.internal.tasks.DexMergingTaskRunnable.run(DexMergingTask.kt:445)
at com.android.build.gradle.internal.tasks.Workers$ActionFacade.run(Workers.kt:348)
at org.gradle.workers.internal.AdapterWorkAction.execute(AdapterWorkAction.java:50)
at org.gradle.workers.internal.DefaultWorkerServer.execute(DefaultWorkerServer.java:47)
at org.gradle.workers.internal.NoIsolationWorkerFactory$1$1$1.create(NoIsolationWorkerFactory.java:65)
at org.gradle.workers.internal.NoIsolationWorkerFactory$1$1$1.create(NoIsolationWorkerFactory.java:61)
at org.gradle.internal.classloader.ClassLoaderUtils.executeInClassloader(ClassLoaderUtils.java:98)
at org.gradle.workers.internal.NoIsolationWorkerFactory$1$1.execute(NoIsolationWorkerFactory.java:61)
at org.gradle.workers.internal.AbstractWorker$1.call(AbstractWorker.java:44)
at org.gradle.workers.internal.AbstractWorker$1.call(AbstractWorker.java:41)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
at org.gradle.workers.internal.AbstractWorker.executeWrappedInBuildOperation(AbstractWorker.java:41)
at org.gradle.workers.internal.NoIsolationWorkerFactory$1.execute(NoIsolationWorkerFactory.java:56)
at org.gradle.workers.internal.DefaultWorkerExecutor$3.call(DefaultWorkerExecutor.java:215)
at org.gradle.workers.internal.DefaultWorkerExecutor$3.call(DefaultWorkerExecutor.java:210)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.runExecution(DefaultConditionalExecutionQueue.java:215)
at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.runBatch(DefaultConditionalExecutionQueue.java:164)
at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.run(DefaultConditionalExecutionQueue.java:131)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.android.tools.r8.CompilationFailedException: Compilation failed to complete
at com.android.tools.r8.utils.O.a(:65)
at com.android.tools.r8.D8.run(:11)
at com.android.builder.dexing.D8DexArchiveMerger.mergeDexArchives(D8DexArchiveMerger.java:117)
... 34 more
Caused by: com.android.tools.r8.utils.b: Error: /Users/userid/WebstormProjects/App/android/google-stt/build/.transforms/854148fb8211844869a1b1d33f9420b2/classes/classes.dex, Type com.google.api.AnnotationsProto is defined multiple times: /Users/userid/WebstormProjects/App/android/google-stt/build/.transforms/854148fb8211844869a1b1d33f9420b2/classes/classes.dex, /Users/userid/WebstormProjects/App/android/app/build/intermediates/external_libs_dex/release/out/classes2.dex
at com.android.tools.r8.utils.y0.a(:21)
at com.android.tools.r8.utils.x0.a(:30)
at com.android.tools.r8.utils.x0.a(:10)
at java.util.concurrent.ConcurrentHashMap.merge(ConcurrentHashMap.java:1990)
at com.android.tools.r8.utils.x0.a(:6)
at com.android.tools.r8.graph.A0$c.a(:6)
at com.android.tools.r8.dex.a.a(:85)
at com.android.tools.r8.dex.a.a(:12)
at com.android.tools.r8.D8.d(:6)
at com.android.tools.r8.D8.b(:1)
at com.android.tools.r8.utils.O.a(:30)
... 36 more


FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:mergeDexRelease'.
> A failure occurred while executing com.android.build.gradle.internal.tasks.Workers$ActionFacade
> com.android.builder.dexing.DexArchiveMergerException: Error while merging dex archives:
Learn how to resolve the issue at https://developer.android.com/studio/build/dependencies#duplicate_classes.
Type com.google.api.AnnotationsProto is defined multiple times: /Users/userid/WebstormProjects/App/android/google-stt/build/.transforms/854148fb8211844869a1b1d33f9420b2/classes/classes.dex, /Users/userid/WebstormProjects/app/android/app/build/intermediates/external_libs_dex/release/out/classes2.dex

I excluded protolite-well-known-type in the /android/app/build.gradle and it looks like below but it still gives me the above error.

//android/app/build.gradle

dependencies {
// our stt project module and react-native-firebase/firestore both use protolite apparently;
implementation(project(':@react-native-firebase_firestore')) {
      exclude group: 'com.google.firebase', module: 'protolite-well-known-types'
}
implementation project(path: ':google-stt')
}

To circumvent this, I have to remove above line from the app's build.gradle and put
in /node_modules/@react-native-firebase/firestore/android/build.gradle. and build the app manually for android.

// /node_modules/@react-native-firebase/firestore/android/build.gradle

dependencies {
implementation("com.google.firebase:firebase-firestore") {
        exclude group: 'com.google.firebase', module: 'protolite-well-known-types'
  }
}

How can we solve this mergeDexRelease issue due to duplicate classes in firestore and google speech to text? So we don't have to manually update build.gradle inside the node module. FIxing it in app's build.gradle would make building .apk seamless via CI /CD

---

Project Files

Javascript

Click To Expand

package.json:

"dependencies": {

"@react-native-firebase/analytics": "^6.7.1",
"@react-native-firebase/app": "^6.7.1",
"@react-native-firebase/crashlytics": "^6.7.1",
"@react-native-firebase/firestore": "^6.7.1",
"@react-native-firebase/perf": "^6.7.1",
"@react-native-firebase/remote-config": "^6.7.1",
}

firebase.json for react-native-firebase v6:

# N/A

iOS

Click To Expand

ios/Podfile:

• I'm not using Pods
• I'm using Pods and my Podfile looks like:

# N/A

AppDelegate.m:

// N/A

---

Android

Click To Expand

Have you converted to AndroidX?

• my application is an AndroidX application?
• I am using android/gradle.settings jetifier=true for Android compatibility?
• I am using the NPM package jetifier for react-native compatibility?

android/build.gradle:

// Top-level build file where you can add configuration options common to all sub-projects/modules.

buildscript {
    ext.kotlin_version = '1.3.61'
    ext {
        buildToolsVersion = "28.0.3"
        minSdkVersion = 22
        compileSdkVersion = 28
        targetSdkVersion = 28
        ndkVersion = "21.0.6113669"
    }
    repositories {
        google()
        jcenter()
        maven {
            url 'https://maven.fabric.io/public'
        }
    }
    dependencies {
        classpath('com.android.tools.build:gradle:3.6.1')
        classpath 'com.google.gms:google-services:4.2.0'
        classpath 'io.fabric.tools:gradle:1.28.1'
        classpath "com.google.protobuf:protobuf-gradle-plugin:0.8.8"
        classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
        classpath 'com.google.firebase:perf-plugin:1.3.1'
        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
    }
}

allprojects {
    repositories {
        mavenLocal()
        maven {
            // All of React Native (JS, Obj-C sources, Android binaries) is installed from npm
            url("$rootDir/../node_modules/react-native/android")
        }
        maven {
            // Android JSC is installed from npm
            url("$rootDir/../node_modules/jsc-android/dist")
        }

        google()
        jcenter()
        maven { url 'https://jitpack.io' }
    }
}

android/app/build.gradle:

apply plugin: "com.android.application"
apply plugin: 'kotlin-android'
apply plugin: 'kotlin-android-extensions'
apply plugin: 'io.fabric'

crashlytics {
  enableNdk true
}

import com.android.build.OutputFile


project.ext.react = [
    entryFile: "index.js",
    enableHermes: false,  // clean and rebuild if changing
]

apply from: "../../node_modules/react-native/react.gradle"

/**
 * Set this to true to create two separate APKs instead of one:
 *   - An APK that only works on ARM devices
 *   - An APK that only works on x86 devices
 * The advantage is the size of the APK is reduced by about 4MB.
 * Upload all the APKs to the Play Store and people will download
 * the correct one based on the CPU architecture of their device.
 */
def enableSeparateBuildPerCPUArchitecture = false

/**
 * Run Proguard to shrink the Java bytecode in release builds.
 */
def enableProguardInReleaseBuilds = false

/**
 * The preferred build flavor of JavaScriptCore.
 *
 * For example, to use the international variant, you can use:
 * `def jscFlavor = 'org.webkit:android-jsc-intl:+'`
 *
 * The international variant includes ICU i18n library and necessary data
 * allowing to use e.g. `Date.toLocaleString` and `String.localeCompare` that
 * give correct results when using with locales other than en-US.  Note that
 * this variant is about 6MiB larger per architecture than default.
 */
def jscFlavor = 'org.webkit:android-jsc:+'

/**
 * Whether to enable the Hermes VM.
 *
 * This should be set on project.ext.react and mirrored here.  If it is not set
 * on project.ext.react, JavaScript will not be compiled to Hermes Bytecode
 * and the benefits of using Hermes will therefore be sharply reduced.
 */
def enableHermes = project.ext.react.get("enableHermes", false);

android {
    compileSdkVersion rootProject.ext.compileSdkVersion

    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }

    defaultConfig {
        applicationId "com.appname"
        minSdkVersion rootProject.ext.minSdkVersion
        targetSdkVersion rootProject.ext.targetSdkVersion
        ndkVersion rootProject.ext.ndkVersion
        versionCode 20
        versionName "1.3.1"
    }
    splits {
        abi {
            reset()
            enable enableSeparateBuildPerCPUArchitecture
            universalApk false  // If true, also generate a universal APK
            include "armeabi-v7a", "x86", "arm64-v8a", "x86_64"
        }
    }
    signingConfigs {
        debug {
            storeFile file('debug.keystore')
            storePassword 'android'
            keyAlias 'androiddebugkey'
            keyPassword 'android'
        }
    }
    buildTypes {
        debug {
            signingConfig signingConfigs.debug
        }
        release {
            minifyEnabled enableProguardInReleaseBuilds
            proguardFiles getDefaultProguardFile("proguard-android.txt"), "proguard-rules.pro"
        }
    }
    // applicationVariants are e.g. debug, release
    applicationVariants.all { variant ->
        variant.outputs.each { output ->
            // For each separate APK per architecture, set a unique version code as described here:
            // https://developer.android.com/studio/build/configure-apk-splits.html
            def versionCodes = ["armeabi-v7a": 1, "x86": 2, "arm64-v8a": 3, "x86_64": 4]
            def abi = output.getFilter(OutputFile.ABI)
            if (abi != null) {  // null for the universal-debug, universal-release variants
                output.versionCodeOverride =
                        versionCodes.get(abi) * 1048576 + defaultConfig.versionCode
            }

        }
    }
}

dependencies {
    // our stt project module and react-native-firebase/firestore both use protolite apparently;
     implementation(project(':@react-native-firebase_firestore'))
     implementation project(path: ':google-stt')

    implementation project(':react-native-vector-icons')
    implementation fileTree(dir: "libs", include: ["*.jar"])
    // implementation 'net.openid:appauth:0.7.1'
    implementation "com.facebook.react:react-native:+"  // From node_modules

    if (enableHermes) {
        def hermesPath = "../../node_modules/hermes-engine/android/";
        debugImplementation files(hermesPath + "hermes-debug.aar")
        releaseImplementation files(hermesPath + "hermes-release.aar")
    } else {
        implementation jscFlavor
    }
    implementation "androidx.core:core-ktx:+"
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"

    // react-native-screens, in support of react-navigation
    implementation 'androidx.appcompat:appcompat:1.1.0-rc01'
    implementation 'androidx.swiperefreshlayout:swiperefreshlayout:1.1.0-alpha02'
    //support for the animation
    implementation 'com.facebook.fresco:fresco:2.0.0'
    implementation 'com.facebook.fresco:animated-gif:2.0.0'


}

// Run this once to be able to run the application with BUCK
// puts all compile dependencies into folder libs for BUCK to use
task copyDownloadableDepsToLibs(type: Copy) {
    from configurations.compile
    into 'libs'
}

apply from: file("../../node_modules/@react-native-community/cli-platform-android/native_modules.gradle"); applyNativeModulesAppBuildGradle(project)



buildscript {
    repositories {
        mavenCentral()
        jcenter()
    }
}


android {
    buildTypes {
        release {
            tasks.whenTaskAdded { task ->
                if(task.name.matches(/^assemble.*Release$/)) {
                    task.finalizedBy 'signReleaseApkTask'
                }
            }
        }
        debug {
            tasks.whenTaskAdded { task ->
                if(task.name.matches(/^assemble.*Debug/) || task.name.matches(/^package.*Debug$/))
                    task.finalizedBy 'signDebugApkTask'
                if(task.name.matches(/^assemble.*DebugAndroidTest/) || task.name.matches(/^package.*DebugAndroidTest$/))
                    task.finalizedBy 'signTestApkTask'
            }
        }
    }
}

// internal repos
repositories {
    mavenCentral()
}

apply plugin: 'com.google.gms.google-services'
// the following might be able to be removed once this is merged and published: https://github.com/google/play-services-plugins/pull/95
googleServices {
    disableVersionCheck = true
}

android {
  packagingOptions {
    pickFirst 'lib/x86/libc++_shared.so'
    pickFirst 'lib/arm64-v8a/libc++_shared.so'
    pickFirst 'lib/armeabi-v7a/libc++_shared.so'
    pickFirst 'lib/x86_64/libjsc.so'
    pickFirst 'lib/arm64-v8a/libjsc.so'
    pickFirst 'lib/x86_64/libc++_shared.so'

  }
}

android/settings.gradle:

rootProject.name = 'AllSparkRN'
include ':react-native-video'
project(':react-native-video').projectDir = new File(rootProject.projectDir, '../node_modules/react-native-video/android-exoplayer')
include ':@react-native-firebase_firestore'
project(':@react-native-firebase_firestore').projectDir = new File(rootProject.projectDir, '../node_modules/@react-native-firebase/firestore/android')
include ':react-native-vector-icons', ':google-stt'
project(':react-native-vector-icons').projectDir = new File(rootProject.projectDir, '../node_modules/react-native-vector-icons/android')
apply from: file("../node_modules/@react-native-community/cli-platform-android/native_modules.gradle"); applyNativeModulesSettingsGradle(settings)
include ':app'

MainApplication.java:

// N/A

AndroidManifest.xml:

<!-- N/A -->

---

Environment

Click To Expand

react-native info output:

System:
OS: macOS Mojave 10.14.6
CPU: (12) x64 Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
Memory: 22.16 MB / 16.00 GB
Shell: 5.3 - /bin/zsh
Binaries:
Node: 10.16.0 - /usr/local/bin/node
npm: 6.9.0 - /usr/local/bin/npm
Watchman: 4.9.0 - /usr/local/bin/watchman
SDKs:
iOS SDK:
Platforms: iOS 13.0, DriverKit 19.0, macOS 10.15, tvOS 13.0, watchOS 6.0
Android SDK:
API Levels: 25, 26, 27, 28, 29
Build Tools: 28.0.3, 29.0.0, 29.0.1
System Images: android-27 | Google Play Intel x86 Atom
IDEs:
Android Studio: 3.6 AI-192.7142.36.36.6392135
Xcode: 11.0/11A420a - /usr/bin/xcodebuild
npmGlobalPackages:
react-native-cli: 2.0.1

• Platform that you're experiencing the issue on:
• iOS
• Android
• iOS but have not tested behavior on Android
• Android but have not tested behavior on iOS
• Both
• react-native-firebase version you're using that has this issue:
• e.g. 6.7.1
• Firebase module(s) you're using that has the issue:
• e.g. Instance ID
• Are you using TypeScript?
• N

---

• 👉 Check out React Native Firebase and Invertase on Twitter for updates on the library.

[mikehardy]

I don't see the stt dep? You redacted your package.json too heavily
I don't see your android files, how can I help?

[savanakbari]

I have updated the original post with more details about android files. stt dependecy is not an npm module so I listed only relevant firebase module.

There seems to be a conflict between firestore and our google speech to text. we need your help with how to make the gradle exclusions properly.

Stt is coming from here https://github.com/GoogleCloudPlatform/android-docs-samples/tree/master/speech/Speech.

[mikehardy]

I don't understand why react-native-firebase is in settings.gradle - that should not be needed. This is so project-specific I doubt you'll receive guidance on the necessary exclusions, you'll need to brute through it. If I were doing it I'd be looking through the android build tree (where the files exist in multiple spots, and at least one should be before dex tries to combine them) and I would use grep and strings utilities to try to find who was including what symbols, then play with exclusions as appropriate. It appears that STT is open source as well so you'll have a good source of truth to also perform the analysis the other direction (starting at their gradle files looking at their deps their and https://github.com/firebase/firebase-android-sdk/ and working forwards, while the build examination is working backwards).

[savanakbari]

I appreciate your suggestions. I ended up with the script to exclude some files before android starts building .apk.

Closing this issue. Thank you.