Skip to content
This repository has been archived by the owner on May 27, 2024. It is now read-only.

Modifying XML libraries for Android

Jump to bottom
Sean Barbeau edited this page Sep 27, 2017 · 38 revisions

To be clear - we don't really recommend use an XML feed in an Android app if there is a JSON alternative available.

JSON is far less verbose than XML, and therefore can save a significant amount of processing, data transmission, and battery life when compared to the resources required to process the same information represented in XML.

However, if you no choice but to use XML, or if you want to benchmark how long XML processing takes to process vs. JSON or other formats, then the below instructions should help you convert XML libraries required to get the Jackson JSON and XML Processor up and running on Android to parse XML.

The Problem - XML parser implementation on Android

Including XML parsing libraries in your Android app is tricky. Most streaming Java XML parsers are based on the Java Specification Request (JSR) 173 - Streaming API for XML (StAX) Java standard. However, StAX is not natively supported by Android.

We can try to include a library in our Android app for a pure StAX implementation that doesn't rely on the desktop JDK libraries, but then we run into another problem: Android doesn't allow you to include libraries that are in the protected Java namespace, including the javax.xml.* namespace.

If you try to include these restricted javax.xml.* namespaces in your app, you'll get a message like:

[2012-09-10 17:51:02 - SiriRestClient] Dx warning: Ignoring InnerClasses attribute for an anonymous inner class
(org.codehaus.stax2.XMLStreamLocation2$1) that doesn't come with an associated EnclosingMethod attribute. This class was probably produced by a compiler that did not target the modern .class file format. The recommended solution is to recompile the class from source, using an up-to-date compiler and without specifying any "-target" type options. The consequence of ignoring this warning is that reflective operations on this class will incorrectly indicate that it is *not* an inner class.
[2012-09-10 17:51:04 - SiriRestClient] Dx trouble processing "javax/xml/stream/EventFilter.class":

Ill-advised or mistaken usage of a core class (java.* or javax.*) when not building a core library.

This is often due to inadvertently including a core library file in your application's project, when using an IDE (such as Eclipse). If you are sure you're not intentionally defining a core class, then this is the most likely explanation of what's going on.

However, you might actually be trying to define a class in a core namespace, the source of which you may have taken, for example, from a non-Android virtual machine project. This will most assuredly not work. At a minimum, it jeopardizes the compatibility of your app with future versions of the platform.
It is also often of questionable legality.

If you really intend to build a core library -- which is only appropriate as part of creating a full virtual machine distribution, as opposed to compiling an application -- then use the "--core-library" option to suppress this error message.

If you go ahead and use "--core-library" but are in fact building an application, then be forewarned that your application will still fail to build or run, at some point. Please be prepared for angry customers who find, for example, that your application ceases to function once they upgrade their operating system. You will be to blame for this problem.

If you are legitimately using some code that happens to be in a core package, then the easiest safe alternative you have is to repackage that code. That is, move the classes in question into your own package namespace. This means that they will never be in conflict with core system classes. JarJar is a tool that may help you in this endeavor. If you find that you cannot do this, then that is an indication that the path you are on will ultimately lead to pain, suffering, grief, and lamentation.

[2012-09-10 17:51:04 - SiriRestClient] Dx 1 error; aborting
[2012-09-10 17:51:04 - SiriRestClient] Conversion to Dalvik format failed with error 1

So far, there doesn't seem to be an easy way of executing the Dx "--core-library" parameter when using Eclipse.

This creates a problem, since by definition of JSR 173 StAX, StAX implementations include classes in the javax.xml.* namespace.

This issue effectively prevents you from using an XML parser such as Jackson JSON and XML Processor on Android with the default JARs that are build for use on the Java desktop JDK/JRE platform.

The Solution - Jar Jar Links

We use the open-source project Jar Jar Links, or JarJar for short, to modify our XML libraries that will be bundled with our app so that they will run on Android.

We can tell JarJar to find the namespaces in the XML library JAR files that we provide that conflict with Android (e.g., javax.xml.stream.*), and have JarJar rename them to something that doesn't conflict (e.g., edu.usf.cutr.javax.xml.stream.*). Android will then accept the libraries that don't conflict.

Here's a zip file that includes everything you'll need to convert the included XML library JAR files to their Android-compatible versions, including:

  • JarJar 1.3 JAR file - main executable for JarJar that does the renaming magic.
  • rules.txt - file that includes our renaming rules (see below)
  • XML parsing libraries required by Jackson:
    • StAX API - JSR 173 API interface definitions. Also see Wikipedia StAX entry for more info.
    • StAX2 API - Experimental expansion of the original StAX API used by Aalto
    • Aalto - StAX 1 and 2 XML parser implementation
    • jackson-dataformat-xml - integrates the above XML parser APIs and implementations with the core Jackson project

The generate_android_jarsv21.bat batch file is included to automate the conversion process using JarJar for multiple XML library JARs at once.

How do I use JarJar to get Android-compatible XML files?

  1. Download the zip file mentioned above, and unzip it.
  2. Make sure you have an output folder in the same directory as the included generate_android_jars.bat script.
  3. Type generate_android_jars.bat at the command prompt and hit enter to run the batch file to convert multiple JARs at once. You'll see this output on your screen:
D:\JarJar\Jackson>generate_android_jars.bat
Generating Android-compatible XML library JARs...
Installing Android-compatible XML library JARs in Maven repo at 'Git Projects\cutr-mvn-repo'...
...(a lot of Maven output)
Done!  Check the /output directory for your Android-compatible XML library JARs, and a local maven respository at '\Git Projects\cutr-mvn-repo' for the artifacts.
Artifacts can be updated in the main CUTR Maven repository by pushing contents of cutr-mvn-repo folder to this Github project:
https://github.com/CUTR-at-USF/cutr-mvn-repo
echo ----------------------------

You should now have a set of JARs in the output folder that are compatible with Android, as well as these artifacts installed in your local Maven repository! We also output the files to another directory so they can easily be pushed to our Maven repository on Github.

How did you set up JarJar to do the XML library conversion?

JarJar uses a series of rules to change the namespaces in the JAR files.

Here's the contents of our rules.txt file that renames all occurences of javax.xml.stream.* to edu.usf.cutr.javax.xml.stream.*:

rule javax.xml.stream.** edu.usf.cutr.javax.xml.stream.@1

The batch file automates multiple JAR conversions with JarJar at once and then does the Maven install:

@echo off
REM This batch file uses JarJar (http://code.google.com/p/jarjar/) to convert StAX and other XML libraries
REM so they are compatible with Android.

REM Change any updated library versions in below variables
set jackson-ver=2.1.2
set stax2-ver=3.1.1
set stax-ver=1.0-2
set aalto-ver=0.9.8

REM Delete existing files
del output\*-android.jar

REM Generate Android-compatible XML library JARs
echo Generating Android-compatible XML library JARs...
java -jar jarjar-1.3.jar process rules.txt jackson-dataformat-xml-%jackson-ver%.jar output\jackson-dataformat-xml-android-%jackson-ver%.jar
java -jar jarjar-1.3.jar process rules.txt stax2-api-%stax2-ver%.jar output\stax2-api-android-%stax2-ver%.jar
java -jar jarjar-1.3.jar process rules.txt stax-api-%stax-ver%.jar output\stax-api-android-%stax-ver%.jar
java -jar jarjar-1.3.jar process rules.txt aalto-xml-%aalto-ver%.jar output\aalto-xml-android-%aalto-ver%.jar

REM Install these artifacts in your local Maven repository - check for error after each command
REM NOTE - Depending on whether each version is a SNAPSHOT or RELEASE, 
REM make sure its being deployed to the correct directory by changing the last path parameter in each line below!!!!

echo Installing Android-compatible XML library JARs in Maven repo at 'Git Projects\cutr-mvn-repo'...
call mvn install:install-file -DgroupId=edu.usf.cutr.android.xml -DartifactId=jackson-dataformat-xml-android -Dversion=%jackson-ver% -Dfile=output\jackson-dataformat-xml-android-%jackson-ver%.jar -Dpackaging=jar -DgeneratePom=true -DlocalRepositoryPath="/Git Projects/cutr-mvn-repo/releases"
if not "%ERRORLEVEL%" == "0" exit /b
call mvn install:install-file -DgroupId=edu.usf.cutr.android.xml -DartifactId=stax2-api-android -Dversion=%stax2-ver% -Dfile=output\stax2-api-android-%stax2-ver%.jar -Dpackaging=jar -DgeneratePom=true -DlocalRepositoryPath="/Git Projects/cutr-mvn-repo/releases"
if not "%ERRORLEVEL%" == "0" exit /b
call mvn install:install-file -DgroupId=edu.usf.cutr.android.xml -DartifactId=stax-api-android -Dversion=%stax-ver% -Dfile=output\stax-api-android-%stax-ver%.jar -Dpackaging=jar -DgeneratePom=true -DlocalRepositoryPath="/Git Projects/cutr-mvn-repo/releases"
if not "%ERRORLEVEL%" == "0" exit /b
call mvn install:install-file -DgroupId=edu.usf.cutr.android.xml -DartifactId=aalto-xml-android -Dversion=%aalto-ver% -Dfile=output\aalto-xml-android-%aalto-ver%.jar -Dpackaging=jar -DgeneratePom=true -DlocalRepositoryPath="/Git Projects/cutr-mvn-repo/releases"

echo ----------------------------
echo Done!  Check the /output directory for your Android-compatible XML library JARs, and a local maven respository at '\Git Projects\cutr-mvn-repo' for the artifacts.
echo Artifacts can be updated in the main CUTR Maven repository by pushing contents of cutr-mvn-repo folder to this Github project:
echo https://github.com/CUTR-at-USF/cutr-mvn-repo
echo ----------------------------
pause

Using XML Libraries for Android, modified by CUTR

We make the modified XML libraries available via Maven if you want to use them in your own project.

Put the following in your pom.xml to use them:

<dependencies>              
    <!-- SIRI POJOs -->
    <dependency>
        <artifactId>onebusaway-siri-api-v13-pojos</artifactId>            
        <groupId>edu.usf.cutr.siri</groupId>
        <version>1.0.0</version>
    </dependency>
        
    <!-- Using Jackson for JSON parsing -->
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-core</artifactId>
        <version>2.1.2</version>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.1.2</version>
    </dependency>
    
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-annotations</artifactId>
        <version>2.1.2</version>
    </dependency>	
    <!-- Use Jackson for XML parsing  -->
    <!-- So many problems with XML and Android...
         Below XML libraries have been modified using JarJar
         and are NOT the same as their J2SE counterparts,
         hence the added "-android" to the artifactId
         See:
         https://github.com/CUTR-at-USF/SiriRestClient/wiki/Modifying-XML-libraries-for-Android
          -->        
    <dependency>
        <groupId>edu.usf.cutr.android.xml</groupId>
        <artifactId>jackson-dataformat-xml-android</artifactId>
        <version>2.1.2</version>
    </dependency>
    <dependency>
        <groupId>edu.usf.cutr.android.xml</groupId>
        <artifactId>stax2-api-android</artifactId>
        <version>3.1.1</version>
    </dependency>
    <dependency>
        <groupId>edu.usf.cutr.android.xml</groupId>
        <artifactId>stax-api-android</artifactId>
        <version>1.0-2</version>
    </dependency>
    <dependency>
        <groupId>edu.usf.cutr.android.xml</groupId>
        <artifactId>aalto-xml-android</artifactId>
        <version>0.9.8</version>
    </dependency>        
</dependencies>    
    
<repositories>
  <!-- CUTR Android XML libraries Releases -->
  <repository>
    <id>cutr-releases</id>
    <url>https://github.com/CUTR-at-USF/cutr-mvn-repo/raw/master/releases</url>
  </repository>      
</repositories>

What's next?

Clone this wiki locally