Predownload all dependencies flutter needs #120
Conversation
|
Could you please rebase to include changes in 2d8c974. I'd like to check how these changes affect the final image size. |
|
@fkorotkov I have rebased it to include the changes |
|
Interesting. It grows the last layer size from |
|
You could include the android sdk/platform things in the layer this depends on. However, I think l only flutter needs exactly these ones, so I'm not sure whether that's much better https://github.com/flutter/engine/pull/20479/files |
I have built a local flutter image for myself. It's nearly The thing is that the smaller the image is, the more download and bandwidth needed for CI builds. |
|
Well, but whether to download these 6 GB while fetching an image or while performing the build doesn't really change anything, does it? I definitely see the need to add the precache even if it grows the images by several GB. Our Android CI currently takes 15 Minutes because of the required installation of the Android components; our Flutter web, desktop and iOS CI only takes 5 ~ 7 minutes in opposite. Moreover, I would like to propose to include more current sdk components too: |
Downloading while fetching the image happens only once (many CI environments cache the downloaded image), but fetching while building downloads every time. Also, one example of bundling everything in the image is full-offline (with temporary internet access) scenarios which I am struggling now. |
|
Sorry, that's exactly what I meant. The "does not matter" was meant in regard of "why shouldn't we do this at build time as there's no advantage when doing this at run time." Of course it would be a useless waste of bandwidth when downloading the sdk components again and again if we might cache by simply caching the docker image containing the components. |
💹 Working after re-arranging the tasksI could only make it work as follows # Run basic check to download Dark SDK with cache of licenses
# https://github.com/cirruslabs/docker-images-flutter/pull/120/files
RUN flutter doctor --android-licenses
RUN flutter doctor
# Avoid downloading the flutter web as we don't need
# https://stackoverflow.com/questions/58463079/how-can-i-disable-web-support-in-flutter/58463129#58463129
RUN flutter config --no-enable-web
# https://github.com/cirruslabs/docker-images-flutter/pull/120#issuecomment-852822276
RUN flutter precache
RUN sdkmanager "--update" "--verbose"
RUN sdkmanager "patcher;v4"
RUN sdkmanager "build-tools;28.0.3"
RUN sdkmanager "build-tools;30.0.2"
RUN sdkmanager "build-tools;${ANDROID_TOOLS}"
RUN sdkmanager "platform-tools"
RUN sdkmanager "platforms;android-28"
RUN sdkmanager "platforms;android-29"
RUN sdkmanager "platforms;android-${ANDROID_COMPILE_SDK}"
RUN sdkmanager "sources;android-${ANDROID_COMPILE_SDK}"
RUN sdkmanager "extras;android;m2repository"
RUN sdkmanager "extras;google;google_play_services"
RUN sdkmanager "extras;google;m2repository"Builds without LicensesWARNING: The APP_VERSION variable is not set. Defaulting to a blank string.
WARNING: The CI_COMMIT_SHORT_SHA variable is not set. Defaulting to a blank string.
WARNING: The SEMVER_TYPE variable is not set. Defaulting to a blank string.
WARNING: The VERSION_SHA variable is not set. Defaulting to a blank string.
Creating maceio-shopping-parkinglot-mobile_app-package_run ... done
[✔] 🚀
[09:58:57]: Get started using a Gemfile for fastlane https://docs.fastlane.tools/getting-started/ios/setup/#use-a-gemfile
[09:58:58]: Generating fastlane environment output, this might take a few seconds...
<details><summary>✅ fastlane environment ✅</summary>
### Stack
| Key | Value |
| --------------------------- | --------------------------------------------------------- |
| OS | debian_based 10.10 |
| Ruby | 2.7.3 |
| Bundler? | false |
| Git | git version 2.20.1 |
| Installation Source | /usr/local/bundle/bin/fastlane |
| Host | Debian GNU/Linux 10 debian_based 10.10 (5.10.47-linuxkit) |
| Ruby Lib Dir | /usr/local/lib |
| OpenSSL Version | OpenSSL 1.1.1d 10 Sep 2019 |
| Is contained | false |
| Is homebrew | false |
| Is installed via Fabric.app | false |
### System Locale
| Variable | Value | |
| -------- | ------- | - |
| LANG | C.UTF-8 | ✅ |
| LC_ALL | | |
| LANGUAGE | | |
### fastlane files:
**No Fastfile found**
**No Appfile found**
### fastlane gems
| Gem | Version | Update-Status |
| -------- | ------- | ------------- |
| fastlane | 2.199.0 | ✅ Up-To-Date |
### Loaded fastlane plugins:
**No plugins Loaded**
<details><summary><b>Loaded gems</b></summary>
| Gem | Version |
| ------------------------------- | ------------ |
| did_you_mean | 1.4.0 |
| atomos | 0.1.3 |
| CFPropertyList | 3.0.5 |
| claide | 1.0.3 |
| colored2 | 3.1.2 |
| nanaimo | 0.3.0 |
| rexml | 3.2.5 |
| xcodeproj | 1.21.0 |
| rouge | 2.0.7 |
| xcpretty | 0.3.0 |
| terminal-notifier | 2.0.0 |
| unicode-display_width | 1.8.0 |
| terminal-table | 1.8.0 |
| plist | 3.6.0 |
| public_suffix | 4.0.6 |
| addressable | 2.8.0 |
| multipart-post | 2.0.0 |
| word_wrap | 1.0.0 |
| optparse | 0.1.1 |
| tty-screen | 0.8.1 |
| tty-cursor | 0.7.1 |
| tty-spinner | 0.9.3 |
| artifactory | 3.0.15 |
| babosa | 1.0.4 |
| colored | 1.2 |
| highline | 2.0.3 |
| commander | 4.6.0 |
| excon | 0.89.0 |
| faraday-em_http | 1.0.0 |
| faraday-em_synchrony | 1.0.0 |
| faraday-excon | 1.1.0 |
| faraday-httpclient | 1.0.1 |
| faraday-net_http | 1.0.1 |
| faraday-net_http_persistent | 1.2.0 |
| faraday-patron | 1.0.0 |
| faraday-rack | 1.0.0 |
| ruby2_keywords | 0.0.5 |
| faraday | 1.8.0 |
| unf_ext | 0.0.8 |
| unf | 0.1.4 |
| domain_name | 0.5.20190701 |
| http-cookie | 1.0.4 |
| faraday-cookie_jar | 0.0.7 |
| faraday_middleware | 1.2.0 |
| fastimage | 2.2.5 |
| gh_inspector | 1.1.3 |
| json | 2.3.0 |
| mini_magick | 4.11.0 |
| naturally | 2.2.1 |
| rubyzip | 2.3.2 |
| security | 0.1.3 |
| xcpretty-travis-formatter | 1.0.1 |
| dotenv | 2.7.6 |
| simctl | 1.6.8 |
| jwt | 2.3.0 |
| uber | 0.1.0 |
| declarative | 0.0.20 |
| trailblazer-option | 0.1.2 |
| representable | 3.1.1 |
| retriable | 3.1.2 |
| mini_mime | 1.1.2 |
| memoist | 0.16.2 |
| multi_json | 1.15.0 |
| os | 1.1.4 |
| signet | 0.16.0 |
| googleauth | 1.1.0 |
| httpclient | 2.8.3 |
| webrick | 1.6.1 |
| google-apis-core | 0.4.1 |
| google-apis-playcustomapp_v1 | 0.6.0 |
| google-apis-androidpublisher_v3 | 0.14.0 |
| google-cloud-env | 1.5.0 |
| google-cloud-errors | 1.2.0 |
| google-cloud-core | 1.6.0 |
| google-apis-iamcredentials_v1 | 0.8.0 |
| google-apis-storage_v1 | 0.10.0 |
| rake | 13.0.1 |
| digest-crc | 0.6.4 |
| google-cloud-storage | 1.35.0 |
| emoji_regex | 3.2.3 |
| jmespath | 1.4.0 |
| aws-partitions | 1.540.0 |
| aws-eventstream | 1.2.0 |
| aws-sigv4 | 1.4.0 |
| aws-sdk-core | 3.124.0 |
| aws-sdk-kms | 1.52.0 |
| aws-sdk-s3 | 1.109.0 |
| uri | 0.10.0 |
| bundler | 2.2.33 |
| forwardable | 1.3.1 |
| logger | 1.4.2 |
| cgi | 0.1.0 |
| date | 3.0.0 |
| timeout | 0.1.0 |
| stringio | 0.1.0 |
| ipaddr | 1.2.2 |
| openssl | 2.1.2 |
| zlib | 1.1.0 |
| mutex_m | 0.1.0 |
| ostruct | 0.2.0 |
| strscan | 1.0.3 |
| io-console | 0.5.6 |
| delegate | 0.1.0 |
| fileutils | 1.6.0 |
| etc | 1.1.0 |
| singleton | 0.1.0 |
| open3 | 0.1.0 |
| yaml | 0.1.0 |
| psych | 3.1.0 |
</details>
*generated on:* **2021-12-12**
</details>
[09:58:58]: Take notice that this output may contain sensitive information, or simply information that you don't want to make public.
[09:58:58]: Open https://github.com/fastlane/fastlane/issues/new to submit a new issue ✅
INFO: version for package is the following
1.5.59
version: 1.5.59
INFO: Flutter env
[✓] Flutter (Channel unknown, 1.27.0-8.0.pre, on Linux, locale C.UTF-8)
• Flutter version 1.27.0-8.0.pre at /home/supercash/flutter
• Framework revision b7d4806243 (10 months ago), 2021-02-19 09:22:45 -0800
• Engine revision 6993cb229b
• Dart version 2.13.0 (build 2.13.0-30.0.dev)
[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
• Android SDK at /home/supercash/Android/sdk
• Platform android-30, build-tools 30.0.3
• ANDROID_SDK_ROOT = /home/supercash/Android/sdk
• Java binary at: /usr/local/openjdk-8/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_292-b10)
• All Android licenses accepted.
[!] Android Studio (not installed)
• Android Studio not found; download from https://developer.android.com/studio/index.html
(or visit https://flutter.dev/docs/get-started/install/linux#android-setup for detailed instructions).
[!] Connected device
! No devices available
! Doctor found issues in 2 categories.
INFO: Getting dependencies...
Running "flutter pub get" in supercash-mobile... 23.5s
INFO: Generating app bundle with the version in pubspec.yaml
Building without sound null safety
For more information see https://dart.dev/null-safety/unsound-null-safety
Removed unused resources: Binary resource data reduced from 532KB to 489KB: Removed 7%
Running Gradle task 'bundleRelease'...
Running Gradle task 'bundleRelease'... Done 463.7s
✓ Built build/app/outputs/bundle/release/app-release.aab (40.9MB).
INFO: Generating app apk with version in pubspec.yaml
Building without sound null safety
For more information see https://dart.dev/null-safety/unsound-null-safety
Running Gradle task 'assembleRelease'...
Running Gradle task 'assembleRelease'... Done 178.8s
✓ Built build/app/outputs/flutter-apk/app-release.apk (41.8MB). |
|
I mount the following folders into the container, then no more download is needed after first build.
|
This PR fixes the compilation always being slow. #115
Issue was caused by missing platforms android 28 and 29. And also by build-tools version 28 being missing. So flutter autodownloaded it. This predownloads it as part of the docker image.