[Update] Cloud Firestore Xcode Build Times

[Salakar]

Hey everyone 👋

One of the key issues we wanted to look into providing a solution for was #349 - improving build times for Xcode when using Firestore.

I'm happy to say we now have something for you to try (please provide feedback) that should significantly improve your build times;

Mac mini (2018) 6 cores:

Before:    ~ 240s
After:     ~  45s

GitHub Action CI 2 cores:

Before:    ~ 551s
After:     ~ 174s

---

Why is it slow?

Currently the Firestore iOS SDK depends on some 500k lines of mostly C++ code (from dependencies such as gRPC), which when compiling as part of your Xcode build can take a long time - even more so in CI environments.

How can we speed it up?

The biggest win to improving build times would be to not have to compile all that C++ code. The way to do that would be to consume precompiled binaries, in the new XCFramework bundle type introduced to Xcode last year, luckily as part of the Firebase iOS SDK release process these bundles are generated and published in a zip containing all Firebase SDKs.

To help you consume these quicker with the least amount of changes to your codebase we've created invertase/firestore-ios-sdk-frameworks - a repository that provides a drop in replacement podspec for Firestore that consumes the precompiled binaries instead, without you needing to download 3gb of SDKs and manually extract out the Firestore SDKs from the rest of the SDKs in the zip.

⚡ Setup ⚡

To integrate, you just need to add 1 line to your ios/Podfile inside your Flutter project;

pod 'FirebaseFirestore', :git => 'https://github.com/invertase/firestore-ios-sdk-frameworks.git', :tag => '7.4.0'

Add this line inside your target 'Runner' do block in your Podfile, e.g.:

# ...
target 'Runner' do
  pod 'FirebaseFirestore', :git => 'https://github.com/invertase/firestore-ios-sdk-frameworks.git', :tag => '7.4.0'

# ...
end

No changes to the FlutterFire packages were required. See #4698 for a full example of integrating with automatic versioning based on what firebase_core uses.

Why not ship the binaries with the pub packages?

This would add ~200mb to every release. Whereas via a custom pod this is only ever downloaded once on your machine per Firebase iOS SDK version.

---

For more information on the pod and versioning please see the invertase/firestore-ios-sdk-frameworks repository.

---

⚠️ Note: if you were previously caching iOS builds on CI you may now find that when using precompiled binaries that caching is no longer required and it may actually slow down your build times by several minutes. See this comment below for more; #2751 (comment)

---

✨ Please let me know if this has helped you, and I would love to see some before and after timings if you feel like sharing ✨

[mikehardy]

This actually works for more than flutter, for react-native-firebase or anyone that integrates the firestore pod it works, I know this because my creaky old 4 core mbp takes about 5mins less to compile and get the app running. 1 line! Thank you.

[Salakar]

This actually works for more than flutter, for react-native-firebase or anyone that integrates the firestore pod it works, I know this because my creaky old 4 core mbp takes about 5mins less to compile and get the app running. 1 line! Thank you.

Yep, works for anyone using Firestore on iOS - no matter the framework 🎉

[galvin59]

Thanks for the tip.
However when I try to implement it, I get :

 [!] CocoaPods could not find compatible versions for pod "FirebaseFirestore":
      In snapshot (Podfile.lock):
        FirebaseFirestore (= 1.11.2, ~> 1.11.2)
      In Podfile:
        FirebaseFirestore (from `https://github.com/invertase/firestore-ios-sdk-frameworks.git`, tag `6.26.0`)
    None of your spec sources contain a spec satisfying the dependencies: `FirebaseFirestore (from `https://github.com/invertase/firestore-ios-sdk-frameworks.git`, tag `6.26.0`), FirebaseFirestore (= 1.11.2, ~> 1.11.2)`.
    You have either:
     * out-of-date source repos which you can update with `pod repo update` or with `pod install --repo-update`.
     * mistyped the name or version.
     * not added the source repo that hosts the Podspec to your Podfile.

I tried to pod update, but with no luck

Any idea ?

[Ehesp]

@jsoms try removing your Podfile.lock and trying again.

[orestesgaolin]

I started simple tests on codemagic and there's at least few minutes decrease in iOS build time. Unfortunately cache upload caused the total build to be still rather long.

I'm going to update this table with next results.

Flutter stable 1.17.3
Xcode 11.5

Id	Precompiled binaries	Cache	machine	cloud_firestore version	Firestore	Fetch	Installing dep	iOS	pod install	Xcode build	Total
40	🚫 (no_cache)	no	mini	0.13.4+2	6.23.0	0:00:04	0:01:43	0:30:50	278	793	0:49:34
41	🚫 (no_cache)	no	mini	0.13.4+2	6.23.0	0:00:04	0:01:09	0:26:10	212	755	0:45:10
251	🚫 (cache)	yes	mini	0.13.4+2	6.23.0	0:03:10	0:01:01	0:27:43	291	604	0:48:20
253	✅ (cache)	yes	mini	0.13.4+2	6.23.0	0:04:28	0:01:27	0:16:17	193	402	0:42:00
254	✅ (cache)	yes	mini	0.13.4+2	6.23.0	0:02:26	0:00:56	0:15:00	195	376	0:36:30
255	✅ (cache)	yes	mini	0.13.4+2	6.23.0	0:04:01	0:01:17	0:14:55	201	424	0:41:33
1_no_cache	✅ (no_cache)	no	mini	0.13.4+2	6.23.0	0:00:05	0:01:09	0:13:29	165	404	0:25:52
2_no_cache	✅ (no_cache)	no	mini	0.13.4+2	6.23.0	0:00:18	0:01:27	0:14:43	169	478	0:26:06
3_no_cache	✅ (no_cache)	no	mini	0.13.4+2	6.23.0	00:00:07	00:02:15	00:17:04	220	463	00:30:20
4_no_cache_pro	✅ (no_cache_pro)	no	pro	0.13.4+2	6.23.0	00:00:02	00:00:57	00:09:27	141	273	00:17:21
5_no_cache	✅ (no_cache)	no	mini	0.13.4+2	6.23.0	00:00:05	00:01:11	00:12:39	136	366	00:23:52

1_no_cache is completely new workflow on codemagic without any cache but with precompiled binaries!

pro means that it was run on Mac Pro

Important remark when using cache the Android build is significantly shorter (eg 15 vs 9 min). Also cache usually takes 2 min to download and 4-6 min to create and upload.

[Salakar]

@orestesgaolin what's it like without uploading build caches?

[orestesgaolin]

@Salakar I updated my comment above and it's so surprising to see that without any cache (i.e. completely clean build on new workflow) the build time dropped dramatically to about 13 min. I'm gonna run it again without any cache

[orestesgaolin]

@Salakar second build without cache was a bit longer, so it seems that it doesn't have significant effect (thus it may be completely unnecessary to use). I will be collecting data in subsequent builds and maybe we can then draw some conclusions out of it

[jmagman]

Note using xcframeworks requires CocoaPods 1.9.0 or higher. Flutter currently requires a minimum of 1.6.

@Salakar Created issue to add cocoapods_version to the podspec. invertase/firestore-ios-sdk-frameworks#1.

[ElaineSchwner]

Could you help me?

• I run 'pod deintegrate';
• Deleted Pofile.lock
• Run 'pod install'

And I got this:
[!] CocoaPods could not find compatible versions for pod "FirebaseFirestore":
In Podfile:
FirebaseFirestore (from https://github.com/invertase/firestore-ios-sdk-frameworks.git, tag 6.26.0)

cloud_firestore (from `.symlinks/plugins/cloud_firestore/ios`) was resolved to 0.0.1, which depends on
  Firebase/Firestore (~> 6.0) was resolved to 6.0.0, which depends on
    FirebaseFirestore (~> 1.3.0)

What must I do?
Thank you.

[red42]

@ElaineSchwner I solved a similar problem with pod repo update as per @jsoms ' comment.

[ElaineSchwner]

pod repo update

Solved !
Thank you!

[mafreud]

It worked well. Thanks a lot 🛩️ 😄

[RaviKavaiya]

This didn't work for me. I tried pod install, pod update...
See this:

Xcode build done.
Could not build the application for the simulator.
Error launching application on iPhone 11.

[Ehesp]

Hi @RaviKavaiya we'll need more information to help on that one. It looks like the XCode build succeeded but then fails when the app is launched - what are the errors?

[RaviKavaiya]

@Ehesp I tried running with flutter run -v. Couldn't find anything. Will give another shot later and will tell you..

Thanks

[andre-arsenault]

Build timings on my MacBook Pro (2018) 4 cores:

Before:   ~ 272s
After:    ~ 122s

A very nice speed improvement, thank you!

How can we tell which version of the iOS SDK the cloud_firestore package uses, so we can keep the tag up to date in our Podfile? I hope that will be part of the package's release notes going forward?

[ghost]

ld: library not found for -lBoringSSL-GRPC

I've raised an issue against the firestore-ios-sdk-frameworks library - referencing it here in case it helps someone else...

[farukozsoy]

On an old Macbook Pro ( 2011 - 2 cores - i7 )

before ~ 15 minutes
after ~ 9 minutes

Thanks!

[nilsreichardt]

Is there also a workaround for Flutter macOS?

[Salakar]

@AndroidNils unfortunately this is only for iOS

[zumrywahid]

Use folliwing line for

pod 'FirebaseFirestore', :git => 'https://github.com/invertase/firestore-ios-sdk-frameworks.git', :tag => '7.3.0'

for your puspec cloud_firestore: "^0.16.0+1" version .

[mikehardy]

@zumrywahid a more general recommendation (as your recommendation will go out of date as versions are released) is to consult the exact location that the firebase-ios-sdk is defined for the firebase_core package

That's right here: https://github.com/FirebaseExtended/flutterfire/blob/master/packages/firebase_core/firebase_core/ios/firebase_sdk_version.rb#L2

And it's possible to inspect tags / versions on that file in case you are not using the latest version

It is currently 7.3.0 as you mention though, yes :-)

[Python3-8]

I get this error:

[!] CocoaPods could not find compatible versions for pod "FirebaseFirestore":
  In Podfile:
    FirebaseFirestore (from `https://github.com/invertase/firestore-ios-sdk-frameworks.git`, tag `8.0.0`)

    cloud_firestore (from `.symlinks/plugins/cloud_firestore/ios`) was resolved to 2.0.0, which depends on
      Firebase/Firestore (= 7.11.0) was resolved to 7.11.0, which depends on
        FirebaseFirestore (~> 7.11.0)

NOTE: I changed the tag from 7.4.0 to 8.0.0, because 7.4.0 didn't work either.

My ios/Podfile:

target 'Runner' do
  pod 'FirebaseFirestore', :git => 'https://github.com/invertase/firestore-ios-sdk-frameworks.git', :tag => '8.0.0'
  # :tag => '7.4.0'

  use_frameworks!
  use_modular_headers!

  flutter_install_all_ios_pods File.dirname(File.realpath(__FILE__))
end

My pubspec.yaml:

dependencies:
  flutter:
    sdk: flutter
  cloud_firestore: ^2.0.0
  cupertino_icons: ^1.0.2
  email_validator: ^2.0.1
  firebase_auth: ^1.1.3
  firebase_core: ^1.1.0
  flutter_icons: ^1.1.0
  shared_preferences: ^2.0.5
  intl: ^0.17.0
  scrollable_positioned_list: ^0.1.10
  provider: ^5.0.0
  flutter_markdown: ^0.6.2
  flutter_slidable: ^0.6.0

This might help as well:
I got this error while running pod update, and here's the log:

Update all pods
Updating local specs repositories
Analyzing dependencies
Pre-downloading: `FirebaseFirestore` from `https://github.com/invertase/firestore-ios-sdk-frameworks.git`, tag `8.0.0`
FirebaseFirestore/AutodetectLeveldb: subspec would include leveldb if used directly or by default.
cloud_firestore: Using Firebase SDK version '7.11.0' defined in 'firebase_core'
firebase_auth: Using Firebase SDK version '7.11.0' defined in 'firebase_core'
firebase_core: Using Firebase SDK version '7.11.0' defined in 'firebase_core'
[!] CocoaPods could not find compatible versions for pod "FirebaseFirestore":
  In Podfile:
    FirebaseFirestore (from `https://github.com/invertase/firestore-ios-sdk-frameworks.git`, tag `8.0.0`)

    cloud_firestore (from `.symlinks/plugins/cloud_firestore/ios`) was resolved to 2.0.0, which depends on
      Firebase/Firestore (= 7.11.0) was resolved to 7.11.0, which depends on
        FirebaseFirestore (~> 7.11.0)

[!] Automatically assigning platform `iOS` with version `10.0` on target `Runner` because no platform was specified. Please specify a platform for this target in your Podfile. See `https://guides.cocoapods.org/syntax/podfile.html#platform`.

Any help would be appreciated :)

[bounty1342]

You should take the version matching the one of your package.
It screams 7.11 from the logs...

[Python3-8]

Sorry, I have absolutely no experience in iOS development. What file am I supposed to edit?

[Python3-8]

I changed this:

pod 'FirebaseFirestore', :git => 'https://github.com/invertase/firestore-ios-sdk-frameworks.git', :tag => '8.0.0'

...to this:

pod 'FirebaseFirestore', :git => 'https://github.com/invertase/firestore-ios-sdk-frameworks.git', :tag => '7.11.0'

...and ran pod update. It ran successfully and I'm gonna try building my app now.

[Python3-8]

The build still takes a long time... It took 8 minutes. Is this because it was the first build since I modified the Podfile, or do I have any other dependencies that might be causing a problem?

These are my dependencies:

dependencies:
  flutter:
    sdk: flutter
  cloud_firestore: ^2.0.0
  cupertino_icons: ^1.0.2
  email_validator: ^2.0.1
  firebase_auth: ^1.1.3
  firebase_core: ^1.1.0
  flutter_icons: ^1.1.0
  shared_preferences: ^2.0.5
  intl: ^0.17.0
  scrollable_positioned_list: ^0.1.10
  provider: ^5.0.0
  flutter_markdown: ^0.6.2
  flutter_slidable: ^0.6.0

[Python3-8]

I think it's actually working! I just built my app again to work on it a little bit, and the build finished in 82 seconds.

Launching lib/main.dart on iPhone 12 in debug mode...
Xcode build done.                                           81.8s
Connecting to VM Service at ws://127.0.0.1:58642/...
...

[valle-xyz]

You should take the version matching the one of your package.
It screams 7.11 from the logs...

Could this appear in the explanatory text? I stumbled upon it and I guess an explanation would be helpful to other people, too.

[mikehardy]

@Valentin-Seehausen hi there! If it is missing in some location that seems appropriate to you, I think in github you can just hit the little edit button on the README.md and suggest where it should be. We'd be happy to merge any reasonable change that would help users, most of our documentation is improved in that way since we (that is, the maintainers) are usually too closed to / used to things to have the perspective of fresh users anymore.

[kasvith]

Thank you, this worked well :)

Make sure to check the actual Firebase version in Podfile

[lemonbitsapp]

guys, im using SwiftPackage manager, - the build times is usually 40-50mins thru CI/CD. this is O M G. is there a setting to speed this up. Tks.

[mikehardy]

Depending on your CI/CD there are other tricks to speed it up since this doesn't do SPM yet
https://github.com/hendrikmuhs/ccache-action/ or
https://github.com/mikehardy/buildcache-action

Those are for github but the general solution design ("Install compiler cache tool, make symbolic links to the tool (or update your build definition to use it directly), save and restore compiler cache across builds") is generic and I have seen it work on bitrise as well so I know it's generally applicable.

[nathanaelneveux]

Trying to implement the automatic versioning based on what firebase_core uses (per #4698) but kinda hitting a wall.

This line of course doesn't point to anything on my system - I also can't make assumptions about where it is on other members of my teams system's:
https://github.com/FirebaseExtended/flutterfire/blob/7d806d710fa6c3968373245e0b7f6bcf9dde0d96/packages/cloud_firestore/cloud_firestore/example/ios/Podfile#L41

Trying to find a replacement for this line I've come up with this:

def firebase_core
  generated_flutter_plugins_path = File.expand_path(File.join('..', '..', '.flutter-plugins'), __FILE__)
  unless File.exist?(generated_flutter_plugins_path)
    raise "#{generated_flutter_plugins_path} must exist. If you're running pod install manually, make sure flutter pub get is executed first"
  end

  File.foreach(generated_flutter_plugins_path) do |line|
    matches = line.match(/firebase_core\=(.*)/)
    return matches[1].strip if matches
  end
  raise "firebase_core not found in #{generated_flutter_plugins_path}. Try deleting .flutter-plugins, then run flutter pub get"
end

which would allow me to reference the script file like this:

firebase_core_script = File.expand_path(File.join('ios', 'firebase_sdk_version.rb'), firebase_core)

However flutter responds like this when I ty to build:

Exception: Warning: Podfile is out of date
  This can cause issues if your application depends on plugins that do not support iOS or macOS.
  See https://flutter.dev/docs/development/packages-and-plugins/developing-packages#plugin-platforms for details.
  If you have local Podfile edits you would like to keep, see https://github.com/flutter/flutter/issues/45197 for instructions.

It seems that having any mention of .flutter-plugins in your podfile causes this message to appear and not build. So I'm at a bit of a loss as to where to find a consistent reference to the firebase_core folder.

EDIT
ok - something like changing it to '.flutter'+'-plugins' gets around the flutter test.
https://github.com/flutter/flutter/blob/e028d0f046c20c61b7c7c8a23bc456f337410658/packages/flutter_tools/lib/src/macos/cocoapods.dart#L408
But the question still stands on is there a better place I should be referencing to get the location of the firebase_core folder?

[devbrandonlong]

Went from

to

Nicely done 🙌🏽

[misteroak]

Yes, significant improvement. Went from 568s to 32s (MBP 3.1 GHz Quad-Core Intel Core i7). Thank you!

[dreadjr]

Depending on your CI/CD there are other tricks to speed it up since this doesn't do SPM yet

https://github.com/hendrikmuhs/ccache-action/ or

https://github.com/mikehardy/buildcache-action

Those are for github but the general solution design ("Install compiler cache tool, make symbolic links to the tool (or update your build definition to use it directly), save and restore compiler cache across builds") is generic and I have seen it work on bitrise as well so I know it's generally applicable.

@mikehardy do you have a bitrise.yaml example?

[mikehardy]

No, sorry. But I've heard of a successful bitrise implementation of ccache using xcodebuild directly on command line with variables LD/LD++/CLANG/CLANG++ set on the command line to the path of ccache

[Ahmadshoh]

Thanks. It works for me. True, I had to change the version from 7.14.0 to 8.10.0, if anyone has an error like that:

[!] CocoaPods could not find compatible versions for pod "FirebaseFirestore":
In Podfile:
FirebaseFirestore (from https://github.com/invertase/firestore-ios-sdk-frameworks.git, tag 8.10.0)

cloud_firestore (from `.symlinks/plugins/cloud_firestore/ios`) was resolved to 2.0.0, which depends on
  Firebase/Firestore (= 7.14.0) was resolved to 7.14.0, which depends on
    FirebaseFirestore (~> 7.14.0)

[irajeshh]

It took 1hr before & 5 mins after using this.

[mikehardy]

@irajeshh if you think that's neat (and it is!) imagine integrating ccache for your entire iOS build, so that everything (not just firestore) was only built when necessary...

[gvsakhil]

Omg this is scary my build times were more than 1hr and now it went to 10-20 secs...

I followed step 4 from this url -
https://firebase.flutter.dev/docs/firestore/overview/

[irajeshh]

@irajeshh if you think that's neat (and it is!) imagine integrating ccache for your entire iOS build, so that everything (not just firestore) was only built when necessary...

Yeah it's okay for me, Also I just tested the macOs platform, Linux is my main developement base.