Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rework:android:Move build system from ant to gradle #553

Merged
merged 33 commits into from May 2, 2018

Conversation

@jandegr
Copy link
Contributor

commented Apr 27, 2018

The aim is to allow building with gradle for Android

  • builds a multi-arch apk in one container on CI in less time than one of the 2 containers now.
  • can be imported as an Androidstudio project as-is, CI builds the resources as well but there are no provisions to do so locally, for now the fastest way is to grab the resources from an existing apk
  • gradle/androidstudio is the only tool supported by Google.
    tested on CircleCI and win10, will probably work as-is on linux and osX

feel free to comment

jandegr and others added 17 commits Apr 23, 2018
@jandegr

This comment has been minimized.

Copy link
Contributor

commented on 8b0a647 Apr 24, 2018

this was working fine, thx, but a better solution was to use librsvg2-bin as in the linux build, so it now builds a combined apk for arm and x86 in one container in 5 minutes instead of 20 minutes with inkscape.

jandegr added 7 commits Apr 24, 2018
@jkoan

This comment has been minimized.

Copy link
Member

commented Apr 30, 2018

This Pr is also quiet useful for F-Droid because thy currently have some problems to build our app ;)

@jkoan

This comment has been minimized.

Copy link
Member

commented May 1, 2018

@jandegr could you give me some hints how I can set up the Navit project within androidstudio?

jandegr added 2 commits May 1, 2018
set_with_reason(graphics/android "Android detected, NDK:${ANDROID_NDK_API_VERSION}, API:${ANDROID_API_VERSION}" TRUE)
set_with_reason(speech/android "Android detected, NDK:${ANDROID_NDK_API_VERSION}, API:${ANDROID_API_VERSION}" TRUE)
set_with_reason(vehicle/android "Android detected, NDK:${ANDROID_NDK_API_VERSION}, API:${ANDROID_API_VERSION}" TRUE)
set_with_reason(graphics/android "Android detected TRUE)

This comment has been minimized.

Copy link
@jkoan

jkoan May 1, 2018

Member

Ending quotes are missing

jandegr and others added 2 commits May 1, 2018
@jandegr

This comment has been minimized.

Copy link
Contributor Author

commented May 1, 2018

[DRAFT] How to imoprt it as an Androidstudio project

  • provide the sources of this branch in you androidstudioprojects folder
  • provide the resources, either by doing the same conversions as in the CI script if you're on Linux or by grabbing them from a prebuilt apk as a shortcut on Linux or the most practical way on Windows. If you provide prebuilt icons from a prebuilt apk you may get an error because you have duplicates on the two icons that already are present in the resource folders drawable-...dpi, just remove those duplicates if needed. https://github.com/navit-gps/navit/tree/Android_gradle/navit/android/res/drawable-hdpi

Maybe we could remove those in the sources and have them processed from the icons folder during the build just like all the other icons.

  • in the startupscreen select "Import project (Gradle, Eclipse ADT, etc.)"
  • point it to the folder containing the sources
  • ...
  • restart Androidstudio, triggers some necessery step (sync ?)(not needed if the run button appears right away)
  • now the run button became available and you can load it onto a device connected over USB or run it on an AVD, or just build an apk or ...
@pgrandin

This comment has been minimized.

Copy link
Contributor

commented May 1, 2018

@jkoan and @jandegr : that androidstudio doc would be a nice addition to the wiki ;)

jandegr added 4 commits May 1, 2018
Update CMakeLists.txt
Build Android with NLS even if the box does not have gettext for processing translations. Allows builds on platforms with NLS where the resources are provided preprocessed.

@jandegr jandegr changed the title [WIP]Android:gradle rework:android:Move build system from ant to grandle May 2, 2018

@jandegr jandegr merged commit c6e644e into trunk May 2, 2018

6 checks passed

ci/circleci: build_android Your tests passed on CircleCI!
Details
ci/circleci: build_linux Your tests passed on CircleCI!
Details
ci/circleci: build_tomtom_minimal Your tests passed on CircleCI!
Details
ci/circleci: build_tomtom_plugin Your tests passed on CircleCI!
Details
ci/circleci: build_win32 Your tests passed on CircleCI!
Details
ci/circleci: build_wince Your tests passed on CircleCI!
Details

@jandegr jandegr deleted the Android_gradle branch May 2, 2018

@mvglasow

This comment has been minimized.

Copy link
Contributor

commented Jul 21, 2018

I’ve started a small writeup at https://wiki.navit-project.org/index.php/Android_development#With_gradle (focus is on building Navit on a Linux box without Android Studio installed).

I recently tried to build with gradlew and have found two issues with the build:

  • Both vehicle/gpsd and map/garmin cause it to fail with errors. Are they supposed to be included in the Android build at all? They currently get enabled because the respective libraries are found, but I’m not sure if these results are accurate for cross-compilation. Maybe you can have a look at that. I managed to get the build to complete without errors by editing CMakeLists.txt and disabling these two specifically for Android.
  • After running gradlew build and gradlew installDebug, Navit installs fine but all icons are missing from the internal GUI (menu items, menu title bar, dialogs) as well as from the map. OSD icons display, though. The CircleCI build is OK, just my local gradle build is having issues.
@lains

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2018

I had the same issue with gradle, and also had to disable detection of gpsd or I was getting an failure during compilation on #include "gps.h"
At some stage, my build was also missing files, the best way to make sure you have everything is to manually perform all the steps in .circleci/config.yml

@lains

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2018

Maybe if gradle is the way to go (and cmake support is dropped for Android builds), the build script should be updated, and all the pre- and post-build commands that are currently in config.yml could be moved to a shell script, so that when building outside of circleCI, it is just a matter of running one script.

@mvglasow

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2018

Thanks for the heads-up, I’d tried to find the CI stuff but missed the .circleci dir.

To sum it all up, there seems to be a lot of stuff which has to be done outside of gradle, though the old CMake setup would do everything automatically… not great. Gradle gurus, is there any way to have gradle trigger these things?

Otherwise, yes, a build script for Android is probably the best way to go. However, I’d advise keeping build dependency installation out of the build script—check if dependencies are there, and if not, exit with an error telling the user how to install them. Optionally, add another script to install all build dependencies.

@jkoan

This comment has been minimized.

Copy link
Member

commented Jul 22, 2018

For now this should do the trick:
(please adapt at least the code location as needed)

WORKDIR=$(mktemp -d)
cd $WORKDIR
git clone https://github.com/navit-gps/navit
export JVM_OPTS="-Xmx3200m"
export GRADLE_OPTS='-Dorg.gradle.jvmargs="-Xmx2048m -XX:+HeapDumpOnOutOfMemoryError"'
cd navit
sudo apt-get install cmake gettext libsaxonb-java librsvg2-bin
cmake ./ -Dvehicle/gpsd_dbus:BOOL=FALSE -Dsvg2png_scaling:STRING=-1,24,32,48,64,96,128 -Dsvg2png_scaling_nav:STRING=-1,24,32,48,64,96,128 -Dsvg2png_scaling_flag:STRING=-1,24,32,64,96
cd navit/icons
make
mkdir ../android/res/drawable-nodpi
rename 'y/A-Z/a-z/' *
cp *.png ../android/res/drawable-nodpi
cd ../../
cd po
make
mkdir ../navit/android/res/raw
rename 'y/A-Z/a-z/' *
cp *.mo ../navit/android/res/raw
cd ../
cd navit
saxonb-xslt -s:navit_shipped.xml -xsl:xslt/android.xslt -o:navitldpi.xml OSD_SIZE=1 ICON_SMALL=24 ICON_MEDIUM=32 ICON_BIG=48
saxonb-xslt -s:navit_shipped.xml -xsl:xslt/android.xslt -o:navitmdpi.xml OSD_SIZE=1.33 ICON_SMALL=32 ICON_MEDIUM=48 ICON_BIG=64
saxonb-xslt -s:navit_shipped.xml -xsl:xslt/android.xslt -o:navithdpi.xml OSD_SIZE=2 ICON_SMALL=48 ICON_MEDIUM=64 ICON_BIG=96
saxonb-xslt -s:navit_shipped.xml -xsl:xslt/android.xslt -o:navitxhdpi.xml OSD_SIZE=2.67 ICON_SMALL=64 ICON_MEDIUM=96 ICON_BIG=128
saxonb-xslt -s:navit_shipped.xml -xsl:xslt/android.xslt -o:navitxxhdpi.xml OSD_SIZE=4 ICON_SMALL=96 ICON_MEDIUM=128 ICON_BIG=192
saxonb-xslt -s:navit_shipped.xml -xsl:xslt/android.xslt -o:navitxxxhdpi.xml OSD_SIZE=5.33 ICON_SMALL=128 ICON_MEDIUM=192 ICON_BIG=256
mv *dpi.xml android/res/raw
cd ../
chmod +x ./gradlew
./gradlew -v
sdkmanager ndk-bundle
./gradlew assembleDebug
ln -s navit/navit.dtd navit.dtd
mkdir /home/circleci/code/test-results
./gradlew lint test

echo Build finished
echo Apk should be in "navit/android/build/outputs/apk"
find navit/android/build/outputs/apk -iname "*.apk" 
@mvglasow

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2018

Just tried the following:

  • saved this script as scripts/build-android-gradle.sh (shebang added)
  • removed all apt-get, sdkmanager and git clone occurrences (no installations from a build script, and the script is meant to be run from an already existing repo)

I get an error about rename not being found, and the following message from gradle:

> Task :navit:android:mergeDebugResources FAILED
/home/michael/src/navit/navit/android/res/raw/zh_TW.mo: Error: 'T' is not a valid file-based resource name character: File-based resource names must contain only lowercase a-z, 0-9, or underscore
/home/michael/src/navit/navit/android/res/drawable-nodpi/country_US_96_96.png: Error: 'U' is not a valid file-based resource name character: File-based resource names must contain only lowercase a-z, 0-9, or underscore

Something isn’t working yet…

Also, I would suggest exiting as soon as a command fails, something like

command_which_may_fail || exit 1

Advantages:

  • no waste of dev time and CPU cycles if something early in the chain fails
  • continuing after a failure usually does not produce a usable result
  • running more stuff after a failure clutters up the display, making the actual error harder to spot

Lastly, running this from the root of the source tree pollutes the source tree quite badly, any easy way to avoid that?

@jkoan

This comment has been minimized.

Copy link
Member

commented Jul 22, 2018

The issue about rename not working seems to let gradle fail... The rename should only rename every file to lower case

For early error detection you can add set - e

And currently there's no way to do other than that, but I am working on getting it done inside the gradle build so all extra garbage will be gone.

@mvglasow

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2018

Sounds good, thanks. sudo apt-get install rename turned out to fix the issue, and now at least I'm getting a working build. Now I'm working on a cleanup script…

@mvglasow

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2018

About the errors in components: map/garmin throws the following error:

  FAILED: cd /home/michael/src/navit/navit/android/.externalNativeBuild/cmake/debug/x86/navit/map/garmin && gentypes /home/michael/src/navit/navit/map/garmin/garmintypes.txt /home/michael/src/navit/navit/android/.externalNativeBuild/cmake/debug/x86/navit/map/garmin/g2nbuiltin.h
  /bin/sh: 1: gentypes: not found
  ninja: build stopped: subcommand failed.

vehicle/gpsd complains about not being able to find gps.h, this should probably be disabled on Android

@lains lains changed the title rework:android:Move build system from ant to grandle rework:android:Move build system from ant to gradle Jul 23, 2018

@lains

This comment has been minimized.

Copy link
Contributor

commented Jul 23, 2018

@mvglasow, the fix for gps.h (and garmin gentypes) that you mentionned in https://wiki.navit-project.org/index.php/Android_development#With_gradle should probably be submitted as a new pull request that adds the following two lines to CMakeLists.txt as you suggested in the Wiki (BTW thanks for the tip):

set_with_reason(vehicle/gpsd "Android detected" FALSE)
set_with_reason(map/garmin "Android detected" FALSE)

This way, it will improve the build with gradle when done locally (outside of CircleCI).

@mvglasow

This comment has been minimized.

Copy link
Contributor

commented Jul 23, 2018

Agreed for vehicle/gpsd, which we probably don’t need on Android. I’ll add that as soon as I get to it.

As for map/garmin, I’d like some more background knowledge: Currently, the component is enabled or disabled based on whether a library is found. Question—is there an Android port of the library? (Have we supported garmin maps on Android before?) If the answer is that we never had Android support for garmin maps, then I’d agree to simply disable it in the build. Otherwise, the error is a bug in the toolchain which would need to be fixed.

@jkoan

This comment has been minimized.

Copy link
Member

commented Jul 23, 2018

I think we have never enforced the support for it but also never said it's not supported since the map is loaded directly from the file system over the c code.
This is what I got in mind, but I am not totally sure about it. So for Garmin we could repair it or totally drop the support, because I don't know if it's used anymore. OpenStreetMaps are quite good and they are probably the best tested map driver we have.

But for now I think exactly in android it's the best to disable it.

@mvglasow

This comment has been minimized.

Copy link
Contributor

commented Jul 25, 2018

Added in 9f57fe1 and 0e0b7a5, respectively.

@mvglasow

This comment has been minimized.

Copy link
Contributor

commented Aug 5, 2018

As for the build script, I had to modify the saxonb-xslt calls by adding -strip:none. Without it, it would mangle the XML files, stripping a bunch of whitespace characters in various places. The man page states that not stripping whitespace is default behavior, but that doesn’t seem to be the case everywhere. Adding the command line switch fixed this for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.