Android SDK Updates and Bug Fixes #337

Merged
merged 6 commits into from Jul 3, 2014

Projects

None yet

6 participants

@gildegoma
Member

Points to discuss with @joshk:

  • go for 0413363 ?
  • preinstall android-L, the L Developer Preview.
  • maybe reduce the VM size, by installing less (or even no) system images. Note that preinstalling popular system images shaves build time for projects that are using related emulators.
  • Already activate maven-rescue recipe ? This part is not final, but it might be interesting to gather some first feedbacks during the beta phase. See travis-ci/travis-ci#2280.
  • Update the docs (android-wait-for-emulator, new versions).
gildegoma added some commits Jun 27, 2014
@gildegoma gildegoma Android SDK: Bug fixes and Version Updates
- store helper scripts out of android sdk directory
  (fix travis-ci/travis-ci#2279)

- change license pattern in android-update-sdk helper script
  (fix travis-ci/travis-ci#2466)

- upgrade to Android SDK 23 and use new system images identifiers

- ship android-wait-for-emulator helper script
  (usage to be described in docs.travis-ci.com)

Note: maven-rescue recipe is still a work in progress, and for this
reason it is disabled for now.
a167d0c
@gildegoma gildegoma Android SDK: accept all the licenses by default 0413363
@joshk joshk was assigned by BanzaiMan Jun 30, 2014
@BanzaiMan
Member

An off-topic question: How useful is Food Critic warnings at the moment?

@gildegoma
Member

@BanzaiMan Good point! These warnings were intially thought as reminders before deciding for each rule whether to fix it, or ignore it. By not doing this triage for long time, I agree that they became over time annoying and useless. If it is okay for you, I disable them directly in this branch to avoid extra merge/rebase work (babc1a2) for this "little" thing.

PS: I'll keep in mind to enable some of this checks ;-)

@joshk
Member
joshk commented Jul 1, 2014

@gildegoma any downsides to me merging this in?

@gildegoma
Member

@joshk you can merge and we'll fine tune later, but I'd prefer to clarify the major doubts during this pull request review:

  • Don't pre-install the system images ? That will increase build time and some projects will need to adapt their .travis.yml (which was to be expected during the beta phase).
  • Confirm that we'll accept any license by default. I don't see any downsides.
  • Maybe force tools upgrade during chef provisioning, though I prefer to claim the upgrade by an explicit bump of default['android-sdk']['version'].

I put some arguments in favor to not pre-install too much things in travis-ci/travis-ci#2470 (comment). If we agree on this "head style" approach, I update this PR branch accordingly (and try to create a PR for the travis documentation updates).

The rest (possible pre-installation of android-L or activation of maven-rescue chef recipe) can wait.

@gildegoma
Member

@joshk with 8de5ea0, I propose to provide a lighter VM, forcing each project to specify its whole set of dependencies. I hope we get soon some user feedbacks to consolidate this approach.

@gildegoma
Member

We started a very interesting chat with @smarek on IRC. Please @joshk wait a bit before building the new Android VM.

@smarek
smarek commented Jul 2, 2014

@gildegoma

Pre-installing components is bad idea in my opinion, as the compileSdk, buildTools and M2/Extra repositories versions change quite often.

Few notes:

  • Definitely don't include sysimages in VM, that's huge overhead
  • Include SDK preinstalled, only if it's installation would took more than few seconds (now not needed, because of caching system included previously)
  • Don't include Extra or M2 repositories installed in VM, because they are updated frequently
  • Include documentation on how to accept all licenses (or simple switch to accept them all)
  • Add two configuration blocks
    • android:buildTools or android:buildToolsVersion to contain string version of build tools (instead of specifying build-tools-x.y.z in android:components)
    • android:compileSdk or android:compileSdkVersion to contain number version of SDK to be installed (usually projects don't require multiple SDK versions, and if they do, they can always specify them in android:components)
    • Inspiration is took from Gradle configuration file, which is for me the most transparent way to configure Android build so far
  • Allow the buildscript to be specified manually, to override auto-detection

Bug
Even with build.gradle and gradlew files present, I was forced to invoke ./gradlew clean assemble check in travis config (see https://github.com/loopj/android-async-http/blob/master/.travis.yml#L15)

My idea of .travis.yml configuration for Android project is this:

language: android
jdk: openjdk7
android:
  # will automatically invoke `./gradlew assemble` and `./gradlew check`
  # or whatever the default command chain will be
  # could be gradle,gradlew,ant,maven - required to state the buildscript manually
  buildscript: gradlew # optional
  # or buildToolsVersion, could be 19, 19.0, 19.1, 20.0, 20
  buildTools: 19.1 # required
  # or compileSdkVersion, could be 11,14,18,19,20 etc..
  compileSdk: 19 # required
  components: # optional
    - extra-android-support
    - extra-android-m2repository
  # this declaration should be stronger than android:licenses configuration
  # so if provided, there should be warning if android:licenses is provided too
  acceptAllLicenses: true # optional
@gildegoma
Member

@smarek thanks a bundle for all these inputs! It is of great help ❤️

Below my comments in response...

Include documentation on how to accept all licenses

The new behavior will be to accept all licenses by default (see 0413363). The android.licenses can be used as optional whitelist alternative. So no need for acceptAllLicenses addition.

Pre-installing components is bad idea in my opinion, as the compileSdk, buildTools and M2/Extra repositories versions change quite often.

Good to get a confirmation on that point. We identified the problem with buildTools, but now it seems that there are more components in that situation.

Include SDK preinstalled, only if it's installation would took more than few seconds

Just to be 100% sure, are you referring to the android-... components?
Assuming the "use a single sdk version, but build for multiple targets" practice (pointed out in travis-ci/travis-ci#1395 (comment)), most of the time a project will only request to install one SDK, which is not that big deal to install on the fly.

Even with build.gradle and gradlew files present, I was forced to invoke ./gradlew clean assemble check in travis config

When Gradle (or Gradle wrapper) is detected, the default build script should be gradle build connectedCheck (or ./gradlew build connectedCheck). We thought that build target includes assemble and check (see travis-ci/travis-build#238 (comment)). Is it not equivalent? Note that clean is not required as git repository is freshly cloned. I am very interested to provide an optimal default script... help to improve this is absolutely welcome :)

or buildToolsVersion, could be 19, 19.0, 19.1, 20.0, 20

buildTools: 19.1 # required

or compileSdkVersion, could be 11,14,18,19,20 etc..

compileSdk: 19 # required

If I understand well, you have maybe two things in mind:

  1. Encourage to use only one build-tools version and one sdk version. From the example:
    • buildTools: 19.1 will trigger the installation of the component build-tools-19.1.0
    • compileSdk: 19 will trigger the installation of the component android-19
  2. Do you also expect to export these two settings (buildTools and compileSdk versions) in such a manner that the build tool chain can be configured by these values?

If it's only about the first point, I think that it is enough at this stage to expect (but not require) that the android.components list is well composed (without useless dependencies).

Allow the buildscript to be specified manually, to override auto-detection

the general script: key in .travis.yml does that already.

As a summary, I show here how I think that .travis.yml should look like after this second iteration:

language: android
android:
  components:
# Uncomment the lines below if you want to upgrade to latest release of Android SDK Tools
#    - platform-tools
#    - tools

# No components are pre-installed, all dependencies must be declared
    - build-tools-19.1.0              # buildTools version
    - android-19                      # compileSdk version
    - extra-android-support
    - extra-android-m2repository
    - sys-img-armeabi-v7a-android-19  # needed to create a testing emulator

# Emulator: Create, Start and Wait 
before_script:
    - echo no | android create avd --force -n test -t android-19 --abi armeabi-v7a
    - emulator -avd test -no-skin -no-audio -no-window &
    - android-wait-for-emulator # this helper script is now installed on the Travis VM
    - adb shell input keyevent 82 &

# will automatically invoke the "best" gradle, ant or maven command
# otherwise, override it by uncommenting line below.
# script: ./gradlew ...

@smarek how does it sound to you for a second round of this beta phase?

@gildegoma
Member

After our last discussion with @joshk, it is confirmed that Travis infrastructure can afford to download the Android dependencies for each build. In this second Beta round, we'll gather feedbacks about how is build time impacted by this new approach.

So, I force-rebased this branch with following procedure in mind:

  • an urgent "bug fix" VM image will be produced as soon as possible. This VM will still include the same pre-installed components for backwards compatibility concerns (but no new components, see d6773f9)
  • I propose that we'll officially deprecate these preinstalled components (via a Blog post, but not via travis-build or travis-lint warnings or alike. It's a beta, he!). People should adapt their .travis.yml so that it declares the complete list of depending components via android.components key.
  • Later (in August) we'll probably start to ship a "light VM" image (without preinstalled components). That will potential break builds for project that were not adapted as described above.
@gildegoma gildegoma merged commit 434edac into master Jul 3, 2014

1 check passed

continuous-integration/travis-ci The Travis CI build passed
Details
@gildegoma gildegoma deleted the gc-update-android-sdk branch Jul 3, 2014
@gildegoma gildegoma added a commit to travis-ci/docs-travis-ci-com that referenced this pull request Jul 3, 2014
@gildegoma gildegoma Update Android Guide (rough and dirty)
These changes correspond to travis-ci/travis-cookbooks#337
19c0216
@gildegoma gildegoma referenced this pull request in travis-ci/docs-travis-ci-com Jul 3, 2014
Merged

Update Android Guide #91

@ChristianBecker

Could you provide a timeline for the release of the bug fix VM image?

@joshk
Member
joshk commented Jul 11, 2014

We are working on a set of updates at the moment and hope to have a public timeline to share soon.

Sorry for the delays.

@BanzaiMan
Member

Does anyone have a public Android app that I can test this update on? (One that is failing on Travis is best.)

@ChristianBecker

You can check out Triple-T/simpleprovider@889aff4. The referenced commit is the one, that fixed the problem, so by reverting it, the Travis build should fail.

@barbeau
barbeau commented Jul 16, 2014

@BanzaiMan https://github.com/OneBusAway/onebusaway-android master branch is currently failing on Travis, due the license issue and a dependency on the Google APIs. Latest failing master build is Travis Build 74.

@BanzaiMan
Member

@ChristianBecker @barbeau Thank you!

I tested OneBusAway/onebusaway-android@aa2656c but it appears that it gets stuck in ./wait_for_emulator when the emulator actually starts.

https://gist.github.com/BanzaiMan/643851d26f478630a75e

Which is past the current spot where the build fails, so that is something. :-D

@barbeau
barbeau commented Jul 16, 2014

@BanzaiMan The ./wait_for_emulator script used in that project is the one recommended on the Travis Android page - https://github.com/andrewhr/rxjava-android-example/blob/master/ci/wait_for_emulator. It will eventually exit the script, but it has to wait for the emulator to fully boot to the lockscreen, which takes a while. See my comments here for details.

See this Travis build (prior to the release of the newest Android SDK version) for an example of a successful run.

@gildegoma gildegoma added a commit that referenced this pull request Nov 24, 2014
@gildegoma gildegoma Android SDK: minor cleanup
Remove a forgotten TODO that initially cames via
#337
and d6773f9
4c2eb28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment