Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rework:android:Move build system from ant to gradle #553

Merged
merged 33 commits into from May 2, 2018
Merged

Conversation

@jandegr
Copy link
Contributor

@jandegr jandegr commented Apr 27, 2018

The aim is to allow building with gradle for Android

  • builds a multi-arch apk in one container on CI in less time than one of the 2 containers now.
  • can be imported as an Androidstudio project as-is, CI builds the resources as well but there are no provisions to do so locally, for now the fastest way is to grab the resources from an existing apk
  • gradle/androidstudio is the only tool supported by Google.
    tested on CircleCI and win10, will probably work as-is on linux and osX

feel free to comment

jandegr
Copy link
Contributor

jandegr commented on 8b0a647 Apr 24, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was working fine, thx, but a better solution was to use librsvg2-bin as in the linux build, so it now builds a combined apk for arm and x86 in one container in 5 minutes instead of 20 minutes with inkscape.

@jkoan
Copy link
Member

@jkoan jkoan commented Apr 30, 2018

This Pr is also quiet useful for F-Droid because thy currently have some problems to build our app ;)

@jkoan
Copy link
Member

@jkoan jkoan commented May 1, 2018

@jandegr could you give me some hints how I can set up the Navit project within androidstudio?

CMakeLists.txt Outdated
set_with_reason(graphics/android "Android detected, NDK:${ANDROID_NDK_API_VERSION}, API:${ANDROID_API_VERSION}" TRUE)
set_with_reason(speech/android "Android detected, NDK:${ANDROID_NDK_API_VERSION}, API:${ANDROID_API_VERSION}" TRUE)
set_with_reason(vehicle/android "Android detected, NDK:${ANDROID_NDK_API_VERSION}, API:${ANDROID_API_VERSION}" TRUE)
set_with_reason(graphics/android "Android detected TRUE)
Copy link
Member

@jkoan jkoan May 1, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ending quotes are missing

@jandegr
Copy link
Contributor Author

@jandegr jandegr commented May 1, 2018

[DRAFT] How to imoprt it as an Androidstudio project

  • provide the sources of this branch in you androidstudioprojects folder
  • provide the resources, either by doing the same conversions as in the CI script if you're on Linux or by grabbing them from a prebuilt apk as a shortcut on Linux or the most practical way on Windows. If you provide prebuilt icons from a prebuilt apk you may get an error because you have duplicates on the two icons that already are present in the resource folders drawable-...dpi, just remove those duplicates if needed. https://github.com/navit-gps/navit/tree/Android_gradle/navit/android/res/drawable-hdpi

Maybe we could remove those in the sources and have them processed from the icons folder during the build just like all the other icons.

  • in the startupscreen select "Import project (Gradle, Eclipse ADT, etc.)"
  • point it to the folder containing the sources
  • ...
  • restart Androidstudio, triggers some necessery step (sync ?)(not needed if the run button appears right away)
  • now the run button became available and you can load it onto a device connected over USB or run it on an AVD, or just build an apk or ...

@pgrandin
Copy link
Contributor

@pgrandin pgrandin commented May 1, 2018

@jkoan and @jandegr : that androidstudio doc would be a nice addition to the wiki ;)

jandegr added 4 commits May 1, 2018
Build Android with NLS even if the box does not have gettext for processing translations. Allows builds on platforms with NLS where the resources are provided preprocessed.
@jandegr jandegr changed the title [WIP]Android:gradle rework:android:Move build system from ant to grandle May 2, 2018
@jandegr jandegr merged commit c6e644e into trunk May 2, 2018
6 checks passed
@jandegr jandegr deleted the Android_gradle branch May 2, 2018
@mvglasow
Copy link
Contributor

@mvglasow mvglasow commented Jul 21, 2018

I’ve started a small writeup at https://wiki.navit-project.org/index.php/Android_development#With_gradle (focus is on building Navit on a Linux box without Android Studio installed).

I recently tried to build with gradlew and have found two issues with the build:

  • Both vehicle/gpsd and map/garmin cause it to fail with errors. Are they supposed to be included in the Android build at all? They currently get enabled because the respective libraries are found, but I’m not sure if these results are accurate for cross-compilation. Maybe you can have a look at that. I managed to get the build to complete without errors by editing CMakeLists.txt and disabling these two specifically for Android.
  • After running gradlew build and gradlew installDebug, Navit installs fine but all icons are missing from the internal GUI (menu items, menu title bar, dialogs) as well as from the map. OSD icons display, though. The CircleCI build is OK, just my local gradle build is having issues.

@lains
Copy link
Contributor

@lains lains commented Jul 22, 2018

I had the same issue with gradle, and also had to disable detection of gpsd or I was getting an failure during compilation on #include "gps.h"
At some stage, my build was also missing files, the best way to make sure you have everything is to manually perform all the steps in .circleci/config.yml

@lains
Copy link
Contributor

@lains lains commented Jul 22, 2018

Maybe if gradle is the way to go (and cmake support is dropped for Android builds), the build script should be updated, and all the pre- and post-build commands that are currently in config.yml could be moved to a shell script, so that when building outside of circleCI, it is just a matter of running one script.

@mvglasow
Copy link
Contributor

@mvglasow mvglasow commented Jul 22, 2018

Thanks for the heads-up, I’d tried to find the CI stuff but missed the .circleci dir.

To sum it all up, there seems to be a lot of stuff which has to be done outside of gradle, though the old CMake setup would do everything automatically… not great. Gradle gurus, is there any way to have gradle trigger these things?

Otherwise, yes, a build script for Android is probably the best way to go. However, I’d advise keeping build dependency installation out of the build script—check if dependencies are there, and if not, exit with an error telling the user how to install them. Optionally, add another script to install all build dependencies.

@jkoan
Copy link
Member

@jkoan jkoan commented Jul 22, 2018

For now this should do the trick:
(please adapt at least the code location as needed)

WORKDIR=$(mktemp -d)
cd $WORKDIR
git clone https://github.com/navit-gps/navit
export JVM_OPTS="-Xmx3200m"
export GRADLE_OPTS='-Dorg.gradle.jvmargs="-Xmx2048m -XX:+HeapDumpOnOutOfMemoryError"'
cd navit
sudo apt-get install cmake gettext libsaxonb-java librsvg2-bin
cmake ./ -Dvehicle/gpsd_dbus:BOOL=FALSE -Dsvg2png_scaling:STRING=-1,24,32,48,64,96,128 -Dsvg2png_scaling_nav:STRING=-1,24,32,48,64,96,128 -Dsvg2png_scaling_flag:STRING=-1,24,32,64,96
cd navit/icons
make
mkdir ../android/res/drawable-nodpi
rename 'y/A-Z/a-z/' *
cp *.png ../android/res/drawable-nodpi
cd ../../
cd po
make
mkdir ../navit/android/res/raw
rename 'y/A-Z/a-z/' *
cp *.mo ../navit/android/res/raw
cd ../
cd navit
saxonb-xslt -s:navit_shipped.xml -xsl:xslt/android.xslt -o:navitldpi.xml OSD_SIZE=1 ICON_SMALL=24 ICON_MEDIUM=32 ICON_BIG=48
saxonb-xslt -s:navit_shipped.xml -xsl:xslt/android.xslt -o:navitmdpi.xml OSD_SIZE=1.33 ICON_SMALL=32 ICON_MEDIUM=48 ICON_BIG=64
saxonb-xslt -s:navit_shipped.xml -xsl:xslt/android.xslt -o:navithdpi.xml OSD_SIZE=2 ICON_SMALL=48 ICON_MEDIUM=64 ICON_BIG=96
saxonb-xslt -s:navit_shipped.xml -xsl:xslt/android.xslt -o:navitxhdpi.xml OSD_SIZE=2.67 ICON_SMALL=64 ICON_MEDIUM=96 ICON_BIG=128
saxonb-xslt -s:navit_shipped.xml -xsl:xslt/android.xslt -o:navitxxhdpi.xml OSD_SIZE=4 ICON_SMALL=96 ICON_MEDIUM=128 ICON_BIG=192
saxonb-xslt -s:navit_shipped.xml -xsl:xslt/android.xslt -o:navitxxxhdpi.xml OSD_SIZE=5.33 ICON_SMALL=128 ICON_MEDIUM=192 ICON_BIG=256
mv *dpi.xml android/res/raw
cd ../
chmod +x ./gradlew
./gradlew -v
sdkmanager ndk-bundle
./gradlew assembleDebug
ln -s navit/navit.dtd navit.dtd
mkdir /home/circleci/code/test-results
./gradlew lint test

echo Build finished
echo Apk should be in "navit/android/build/outputs/apk"
find navit/android/build/outputs/apk -iname "*.apk" 

@mvglasow
Copy link
Contributor

@mvglasow mvglasow commented Jul 22, 2018

Just tried the following:

  • saved this script as scripts/build-android-gradle.sh (shebang added)
  • removed all apt-get, sdkmanager and git clone occurrences (no installations from a build script, and the script is meant to be run from an already existing repo)

I get an error about rename not being found, and the following message from gradle:

> Task :navit:android:mergeDebugResources FAILED
/home/michael/src/navit/navit/android/res/raw/zh_TW.mo: Error: 'T' is not a valid file-based resource name character: File-based resource names must contain only lowercase a-z, 0-9, or underscore
/home/michael/src/navit/navit/android/res/drawable-nodpi/country_US_96_96.png: Error: 'U' is not a valid file-based resource name character: File-based resource names must contain only lowercase a-z, 0-9, or underscore

Something isn’t working yet…

Also, I would suggest exiting as soon as a command fails, something like

command_which_may_fail || exit 1

Advantages:

  • no waste of dev time and CPU cycles if something early in the chain fails
  • continuing after a failure usually does not produce a usable result
  • running more stuff after a failure clutters up the display, making the actual error harder to spot

Lastly, running this from the root of the source tree pollutes the source tree quite badly, any easy way to avoid that?

@jkoan
Copy link
Member

@jkoan jkoan commented Jul 22, 2018

The issue about rename not working seems to let gradle fail... The rename should only rename every file to lower case

For early error detection you can add set - e

And currently there's no way to do other than that, but I am working on getting it done inside the gradle build so all extra garbage will be gone.

@mvglasow
Copy link
Contributor

@mvglasow mvglasow commented Jul 22, 2018

Sounds good, thanks. sudo apt-get install rename turned out to fix the issue, and now at least I'm getting a working build. Now I'm working on a cleanup script…

@mvglasow
Copy link
Contributor

@mvglasow mvglasow commented Jul 22, 2018

About the errors in components: map/garmin throws the following error:

  FAILED: cd /home/michael/src/navit/navit/android/.externalNativeBuild/cmake/debug/x86/navit/map/garmin && gentypes /home/michael/src/navit/navit/map/garmin/garmintypes.txt /home/michael/src/navit/navit/android/.externalNativeBuild/cmake/debug/x86/navit/map/garmin/g2nbuiltin.h
  /bin/sh: 1: gentypes: not found
  ninja: build stopped: subcommand failed.

vehicle/gpsd complains about not being able to find gps.h, this should probably be disabled on Android

@lains lains changed the title rework:android:Move build system from ant to grandle rework:android:Move build system from ant to gradle Jul 23, 2018
@lains
Copy link
Contributor

@lains lains commented Jul 23, 2018

@mvglasow, the fix for gps.h (and garmin gentypes) that you mentionned in https://wiki.navit-project.org/index.php/Android_development#With_gradle should probably be submitted as a new pull request that adds the following two lines to CMakeLists.txt as you suggested in the Wiki (BTW thanks for the tip):

set_with_reason(vehicle/gpsd "Android detected" FALSE)
set_with_reason(map/garmin "Android detected" FALSE)

This way, it will improve the build with gradle when done locally (outside of CircleCI).

@mvglasow
Copy link
Contributor

@mvglasow mvglasow commented Jul 23, 2018

Agreed for vehicle/gpsd, which we probably don’t need on Android. I’ll add that as soon as I get to it.

As for map/garmin, I’d like some more background knowledge: Currently, the component is enabled or disabled based on whether a library is found. Question—is there an Android port of the library? (Have we supported garmin maps on Android before?) If the answer is that we never had Android support for garmin maps, then I’d agree to simply disable it in the build. Otherwise, the error is a bug in the toolchain which would need to be fixed.

@jkoan
Copy link
Member

@jkoan jkoan commented Jul 23, 2018

I think we have never enforced the support for it but also never said it's not supported since the map is loaded directly from the file system over the c code.
This is what I got in mind, but I am not totally sure about it. So for Garmin we could repair it or totally drop the support, because I don't know if it's used anymore. OpenStreetMaps are quite good and they are probably the best tested map driver we have.

But for now I think exactly in android it's the best to disable it.

@mvglasow
Copy link
Contributor

@mvglasow mvglasow commented Jul 25, 2018

Added in 9f57fe1 and 0e0b7a5, respectively.

@mvglasow
Copy link
Contributor

@mvglasow mvglasow commented Aug 5, 2018

As for the build script, I had to modify the saxonb-xslt calls by adding -strip:none. Without it, it would mangle the XML files, stripping a bunch of whitespace characters in various places. The man page states that not stripping whitespace is default behavior, but that doesn’t seem to be the case everywhere. Adding the command line switch fixed this for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

6 participants