upload_to_play_store doesn't upload native debug symbols while running on CI

[rawatnaresh]

New Issue Checklist

• Updated fastlane to the latest version
• I read the Contribution Guidelines
• I read docs.fastlane.tools
• I searched for existing GitHub issues

Issue Description

I'm trying to upload Native debug symbols to playstore.

Approach 1st using mapping_paths

Initially I tried zipping debug symbols and passed them to mapping_paths like this :

  desc "Upload app to Staging PlayStore for Internal test"
  lane :deploy_stage_internal do
    build_stage_aab_file
    zip(
      path: "./android/app/build/intermediates/merged_native_libs/stageRelease/out/lib/",
      output_path: "native_debug_symbols.zip",
      verbose: true,
    )
    upload_to_play_store(
      track: "internal",
      skip_upload_images: true,
      skip_upload_screenshots: true,
      release_status: 'draft',
      mapping_paths: ["native_debug_symbols.zip", "./android/app/build/outputs/mapping/stageRelease/mapping.txt"]
    )
  end

but it resulted to an error

but zipping debug symbols and uploading them manually works fine. 👍

Approach 2nd using debugSymbolLevel 'FULL'

According to android docs

If your project builds an Android App Bundle, you can automatically include the native debug symbols file in it. To include this file in release builds, add the following to your app's build.gradle file:

android.buildTypes.release.ndk.debugSymbolLevel = { SYMBOL_TABLE | FULL }

I updated my app's build.gradle file accordingly

    buildTypes {
        debug {
            signingConfig signingConfigs.debug
        }
        release {
            // Caution! In production, you need to generate your own keystore file.
            // see https://reactnative.dev/docs/signed-apk-android.
            signingConfig signingConfigs.release
            shrinkResources false
            minifyEnabled enableProguardInReleaseBuilds
            proguardFiles getDefaultProguardFile("proguard-android.txt"), "proguard-rules.pro"

            // https://developer.android.com/studio/build/shrink-code#native-crash-support
            // Setting debugSymbolLevel to 'FULL' automatically include the native debug symbols and mappings file in aab
            // but note that it will increase the file size of your appbundle dramatically.
            ndk {
                debugSymbolLevel 'FULL'
            }

            firebaseCrashlytics {
                nativeSymbolUploadEnabled true
                unstrippedNativeLibsDir file("build/intermediates/merged_native_libs")
            }
        }
    }

Ran fastlane in Github actions, bundle uploaded successfully with mappings file but no native symbols 😕

Ran fastlane locally and I can see native debug symbols and mappings attached to the file

I'm not sure if I'm missing anything.

Command executed

bundle exec fastlane android deploy_stage_internal --env staging

Environment

 
🚫 fastlane environment 🚫
Stack



Key
Value




OS
12.6.2


Ruby
2.7.5


Bundler?
true


Git
git version 2.37.1 (Apple Git-137.1)


Installation Source
~/Documents/xx/xxx/vendor/bundle/ruby/2.7.0/bin/fastlane


Host
macOS 12.6.2 (21G320)


Ruby Lib Dir
~/.rbenv/versions/2.7.5/lib


OpenSSL Version
OpenSSL 1.1.1q  5 Jul 2022


Is contained
false


Is homebrew
false


Is installed via Fabric.app
false


Xcode Path
/Applications/Xcode.app/Contents/Developer/


Xcode Version
14.2


Swift Version
5.7.2



System Locale



Variable
Value





LANG
en_US.UTF-8
✅


LC_ALL




LANGUAGE





fastlane files:
`./fastlane/Fastfile`
default_platform(:android)

 platform :android do
  ## Readme
  desc "just regenerate fastlane/README.md"
  lane :readme do
  end

  desc 'Bump up version code - to the next highest integer value'
  lane :bumpup_version_code do 
    android_set_version_code(
      gradle_file: "./android/app/build.gradle"
    )
    git_commit( 
      path: "./android/app/build.gradle",
      message: "Android version bump Fastlane"
    )
  end

  desc 'build Staging aab file'
    lane :build_stage_aab_file do
    gradle(
      project_dir: "./android",
      task: "bundle",
      flavor: 'Stage',
      build_type: "Release",
    )
  end

  desc 'build Production aab file'
  lane :build_prod_aab_file do
  gradle(
    project_dir: "./android",
    task: "bundle",
    flavor: 'Prod',
    build_type: "Release",
  )
end

  desc "Upload app to Production PlayStore for Internal test"
  lane :deploy_prod_internal do
    build_prod_aab_file
    upload_to_play_store(
      track: "internal",
      skip_upload_images: true,
      skip_upload_screenshots: true,
      release_status: 'draft',
    )
  end

  desc "Upload app to Staging PlayStore for Internal test"
  lane :deploy_stage_internal do
    gradle(
      project_dir: "./android",
      task: "clean"
    )
    build_stage_aab_file
    upload_to_play_store(
      track: "internal",
      skip_upload_images: true,
      skip_upload_screenshots: true,
      release_status: 'draft',
    )
  end

  error do |lane, exception| 
  end
end

`./fastlane/Appfile`
## Android
package_name ENV["ANDROID_PACKAGE_NAME"]
json_key_file "./android/app/key.json"

# For more information about the Appfile, see:
# https://docs.fastlane.tools/advanced/#appfile

fastlane gems



Gem
Version
Update-Status




fastlane
2.212.0
✅ Up-To-Date



Loaded fastlane plugins:



Plugin
Version
Update-Status




fastlane-plugin-versioning_android
0.1.0
🚫 Update available



Loaded gems



Gem
Version




did_you_mean
1.4.0


bundler
2.1.4


uri
0.10.0


rake
13.0.6


CFPropertyList
3.0.3


concurrent-ruby
1.1.10


i18n
1.12.0


minitest
5.16.3


tzinfo
2.0.5


zeitwerk
2.6.6


activesupport
6.1.7


public_suffix
4.0.6


addressable
2.8.0


httpclient
2.8.3


json
2.5.1


algoliasearch
1.27.5


artifactory
3.0.15


atomos
0.1.3


aws-eventstream
1.2.0


aws-partitions
1.714.0


aws-sigv4
1.5.2


jmespath
1.6.2


aws-sdk-core
3.170.0


aws-sdk-kms
1.62.0


aws-sdk-s3
1.119.1


babosa
1.0.4


claide
1.0.3


fuzzy_match
2.0.4


nap
1.1.0


netrc
0.11.0


ffi
1.15.5


ethon
0.16.0


typhoeus
1.4.0


cocoapods-core
1.11.3


cocoapods-deintegrate
1.0.5


cocoapods-downloader
1.6.3


cocoapods-plugins
1.0.0


cocoapods-search
1.0.1


cocoapods-trunk
1.6.0


cocoapods-try
1.2.0


colored2
3.1.2


escape
0.0.4


fourflusher
2.3.1


gh_inspector
1.1.3


molinillo
0.8.0


ruby-macho
2.5.1


nanaimo
0.3.0


rexml
3.2.5


xcodeproj
1.21.0


cocoapods
1.11.3


colored
1.2


highline
2.0.3


commander
4.6.0


declarative
0.0.20


digest-crc
0.6.4


unf_ext
0.0.8.2


unf
0.1.4


domain_name
0.5.20190701


dotenv
2.8.1


emoji_regex
3.2.3


excon
0.99.0


faraday-em_http
1.0.0


faraday-em_synchrony
1.0.0


faraday-excon
1.1.0


faraday-httpclient
1.0.1


multipart-post
2.0.0


faraday-multipart
1.0.4


faraday-net_http
1.0.1


faraday-net_http_persistent
1.2.0


faraday-patron
1.0.0


faraday-rack
1.0.0


faraday-retry
1.0.3


ruby2_keywords
0.0.5


faraday
1.10.3


http-cookie
1.0.5


faraday-cookie_jar
0.0.7


faraday_middleware
1.2.0


fastimage
2.2.6


jwt
2.7.0


memoist
0.16.2


multi_json
1.15.0


os
1.1.4


signet
0.17.0


googleauth
1.3.0


mini_mime
1.1.2


trailblazer-option
0.1.2


uber
0.1.0


representable
3.2.0


retriable
3.1.2


webrick
1.8.1


google-apis-core
0.11.0


google-apis-androidpublisher_v3
0.34.0


google-apis-playcustomapp_v1
0.12.0


google-apis-iamcredentials_v1
0.17.0


google-apis-storage_v1
0.19.0


google-cloud-env
1.6.0


google-cloud-errors
1.3.0


google-cloud-core
1.6.0


google-cloud-storage
1.44.0


mini_magick
4.12.0


naturally
2.2.1


optparse
0.1.1


plist
3.6.0


rubyzip
2.3.2


security
0.1.3


simctl
1.6.10


terminal-notifier
2.0.0


unicode-display_width
1.8.0


terminal-table
1.8.0


tty-screen
0.8.1


tty-cursor
0.7.1


tty-spinner
0.9.3


word_wrap
1.0.0


rouge
2.0.7


xcpretty
0.3.0


xcpretty-travis-formatter
1.0.1


fastlane-plugin-versioning_android
0.1.0




generated on: 2023-02-20

 

[Dat-Mobile]

+1. Build with CI never get debug symbol uploaded

[lzientek]

Same issue here, I think it's because the zip function add a not wished level lib/ folder and is not valid for the playstore

[rawatnaresh]

yeah that is the issue. zip includes lib folder while zipping instead it should only include the contents of lib folder

[i10416]

Is your zip file really structured in the same way as shown bellow?

arm64-v8a/
arm64-v8a/libapp.so
armeabi-v7a/
armeabi-v7a/libapp.so
x86_64/libapp.so
...

Try zip -r syms.zip arm64-v8a armeabi-v7a x86_64 ... to zip your artifacts.

In addition, I found someone who says download artifacts makes artifacts corrupted when you use path option.
actions/upload-artifact#74 (comment)

[StefanWallin]

I want to thank you @i10416 for your question, it helped me solve this issue!

I solved this with current tools like this on Github Actions running the Fastlane action from my fastlane directory within my android directory:

myApp
├── android
│   ├── app
│   │   ├── build
│   │   │   ├── intermediates
...
│   ├── fastlane
│   │   ├── **Fastfile**

android Fastfile

    symbolsFilePath = File.join(
      Dir.pwd,
      "..",
      "native_debug_symbols.zip"
    )
    symbolsFolderPath = File.join(
      Dir.pwd,
      "..",
      "app",
      "build",
      "intermediates",
      "merged_native_libs",
      "release",
      "out",
      "lib"
    )
    system("cd #{symbolsFolderPath} && zip -r #{symbolsFilePath} .")
    upload_to_play_store(
      # ...other options ...
      mapping_paths: [mappingFilePath, symbolsFilePath]
    )

build.gradle

android {
    buildTypes {
        release {
            minifyEnabled true
            shrinkResources true
            proguardFiles getDefaultProguardFile("proguard-android-optimize.txt"), "proguard-rules.pro"
            debuggable false
            ndk {
                // Note: There is a 300 MB limit for the native debug symbols file.
                // If your debug symbols footprint is too large, use SYMBOL_TABLE
                // instead of FULL to decrease the file size.
                // Ref: https://developer.android.com/build/shrink-code#native-crash-support
                debugSymbolLevel 'FULL'
            }
        }
    }
}

Notes:
It would be nice if the fastlane zip method could support zipping like the system command with recursive and including directories inside a single directory without including that single directory.

[dhruvin22]

Alternative approach:

We can create a zip file containing the contents of the "lib" folder using GitHub actions. Afterwards, we can share the file path of the zip with Fastlane.

    - name: 📂 Create a zip file for Native Symbols. 
      shell: bash
      run: |
        cd android/app/build/intermediates/merged_native_libs/ReleaseNativeLibs/out/lib/
        zip -r symbols.zip *
        echo "symbol_file_path=android/app/build/intermediates/merged_native_libs/ReleaseNativeLibs/out/lib/symbols.zip" >> "$GITHUB_ENV"
        echo "mapping_file_path=android/app/build/outputs/mapping/Release/mapping.txt" >> "$GITHUB_ENV"

It creates a zip file named "symbols.zip" that contains the contents of the directory.
Now, the symbol_file_path and mapping_file_path can be utilized within the fastlane.

         upload_to_play_store(
             json_key: service_cred_file_path,
             package_name: ruc__app_package_id,
             aab: apk_path,
             track:'internal',
             mapping_paths: [symbol_file_path, mapping_file_path],
             release_status:'completed'
           )

[Dat-Mobile]

@StefanWallin I still get the error after following exactly your code. I run it locally

[!] Google Api Error: Invalid request - Validation of uploaded file failed.

But when I zip it manually (right click and compress them), the upload process worked 🤔

[ahna92]

@Dat-Mobile so when you zip lib folder and not the content of lib it works ?

meaning when you unzip that file do you end up with lib folder or with arm64-v8a , armeabi , armeabi-v7a ..etc ?

[krono]

I created a local fastlane plugin.

The run part of it essentially looks like this:

    class NdkZipAction < Action
      def self.run(params)
        output_path = File.expand_path(params[:output_path] || params[:ndb_path])
        ndb_path = params[:ndb_path]
        verbose = params[:verbose]

        output_path += ".zip" unless output_path.end_with?(".zip")
        UI.message("Compressing native debug symbols in #{ndb_path}...")

        output_dir = File.expand_path("..", output_path)
        FileUtils.mkdir_p(output_dir)

        # no trailing /
        ndb_path = ndb_path == '/' ? ndb_path : ndb_path.chomp(?/)
        # traverse the "out" directory
        ndb_path = ndb_path + '/out' if Dir.exist? ndb_path + '/out'
        # the "lib" directory must not be included for the ndk stuff
        ndb_path = ndb_path + '/lib' if Dir.exist? ndb_path + '/lib'
        Dir.chdir(ndb_path) do
          zip_command = ["zip", verbose ? "-r" : "-rq",  output_path, "."]
          Actions.sh(*zip_command)
        end

        UI.success("Successfully generated zip file at path '#{output_path}'")
        output_path
      end
 # .. rest of ndk_zip_action.rb etc

See here
and usage here