Error parsing XML: not well-formed (invalid token)

[iBotPeaches]

Original issue 776 created by kefir500 on 2015-03-04T10:57:38.000Z:

What steps will reproduce the problem?

• Download and recompile this APK: https://www.dropbox.com/s/916e1t25k6notia/locus.apk?dl=1

What is the expected output? What do you see instead?

warning: string 'pref_map_show_center_cross' has no default translation.
warning: string 'pref_map_show_center_cross_desc' has no default translation.

C:\Apktool\res\layout\abc_action_menu_layout.xml:2: error: Error parsing XML: not well-formed (invalid token)

C:\Apktool\res\layout\abc_action_mode_bar.xml:2: error: Error parsing XML: not well-formed (invalid token)

C:\Apktool\res\layout\abc_activity_chooser_view.xml:2: error: Error parsing XML: not well-formed (invalid token)

--- And 5 more similar errors ---

C:\Apktool\res\layout\fragment_track_record.xml:24: error: No resource identifier found for attribute 'pw_text' in package 'android'

C:\Apktool\res\layout\fragment_track_record.xml:24: error: No resource identifier found for attribute 'pw_textSize' in package 'android'

C:\Apktool\res\layout\fragment_track_record.xml:24: error: No resource identifier found for attribute 'pw_barColor' in package 'android'

-- And >10 similar errors ---

C:\Apktool\res\layout\view_color_picker_basic_tab.xml:5: error: Error parsing XML: duplicate attribute

What version of the product are you using? On what operating system?

• Apktool RC4
• aapt.exe (from the Apktool GitHub repository)

Please provide any additional information below.
Seems that XML tags are corrupt. Tried to fix these XML tags manually - no luck. Also tried different aapt and Apktool releases - still the same. Removing "framework/1.apk" does not help.

[iBotPeaches]

Comment #1 originally posted by connor.tumbleson on 2015-03-11T12:25:18.000Z:

Yeah no idea what this is. There are some strange characters in the layout files.

<o.ᴗ android:gravity

Dumping out the resources via aapt

ibotpeaches@raganok:~/Downloads/Apktool/Bug776$ aapt d xmltree locus.apk res/layout/abc_action_menu_layout.xml
N: android=http://schemas.android.com/apk/res/android
N: app=http://schemas.android.com/apk/res-auto
E: o.ᴗ (line=17)

Yields the same weird character. Needs more investigation.

[iBotPeaches]

Comment #2 originally posted by Liquid.St.Ray on 2015-03-13T23:20:00.000Z:

The unknow tag name is actually the full qualified name of a custom UI class, which has been obfuscated. I suffered the same issue and I found the symbol in smali files.
I think this could be a common case for lots of commerical apps.

[iBotPeaches]

Comment #3 originally posted by connor.tumbleson on 2015-03-13T23:21:51.000Z:

Is this from a certain ROM/OEM then? How does an obfuscated resource work without being de-obfuscated prior to use?

Its not like Android knows this mapping of obfuscation -> real resource names.

[iBotPeaches]

Comment #4 originally posted by Liquid.St.Ray on 2015-03-13T23:52:04.000Z:

Should be some unexpected characters was introduced in the obfuscation and AAPT is complaining about it.

I tried to rename a custom UI component class using unexpected characters and suffered the same issue when building the APK file.

That APK could be packed with custom built AAPT I think.

[iBotPeaches]

Comment #5 originally posted by Liquid.St.Ray on 2015-03-14T00:18:07.000Z:

DexGuard could generate non-ASCII characters for class/method/field names, which is not supported by stock AAPT.

[iBotPeaches]

Comment #6 originally posted by connor.tumbleson on 2015-03-14T00:20:05.000Z:

aapt is all about resources. It doesn't manage the source (java) at all, so those points aren't relatable.

[iBotPeaches]

Comment #7 originally posted by Liquid.St.Ray on 2015-03-14T00:25:04.000Z:

You can claim a custom UI class (TextView for example) and reference it in the layout xml file. In this case it would involve compiled resoruce file.

Once that class was obfuscate by something like DexGuard, the class reference in layout xml file would be modified as well.

[iBotPeaches]

Comment #8 originally posted by connor.tumbleson on 2015-03-14T00:50:24.000Z:

Ah true true. Custom UI classes then are just localized mapping in an application. So it doesn't matter if the naming is Foo or some crazy UTF8 character.

The question remains AAPT is obviously not liking it. So as you said a custom AAPT must be used. With that being said, do you have any information on this? I looked around there website (Dexguard/Proguard) and couldn't find anything.

[iBotPeaches]

Comment #9 originally posted by alkorware on 2015-03-14T18:33:01.000Z:

it's 100% dexguard's obfuscation.

[ddccffvv]

Would it be possible to work around this by just renaming the classes to unique [a-Z*] names and updating the references (during/after decompilation)?

[jack16099]

Is this issue fixed? I think I've seen some modded apps which gives the same error on latest version of apktool and there's those illegal characters in the manifest. If I'm not wrong that it's modded, how did they do it if apktool can't?

[warren-bank]

though this issue is closed, I recently encountered this behavior with apktool 2.4.1

to summarize, the problem traced back to the fact that the decoded apk contained smali filenames with unicode characters.

as a workaround, I wrote a small nodejs command-line utility to rename all smali unicode filenames, and remap all references elsewhere in the project.

wrote it in a day.. might not solve everyone's woes, but I hope that some may find it useful; worked for me.

[pgp]

Same issue here, I tested the same apk (titanium backup 7.6.0.1) linked in #1389 , using apktool 2.5.0 ... Gonna try @warren-bank script for now

[warren-bank]

I'm in a virtualized environment right now, so I can't check my notes (on modding TB) until later.. but I do remember TB having some unique issues. I believe that more recent versions decompile with missing resource directories, in addition to unicode issues. I had to iteratively test older versions until I found the last one to decompile without any missing resource directories. It was that version that I was able to successfully round-trip: decompile, (optionally) modify smali source files, run my script, recompile, and run in Android without any installation or runtime errors.

[pgp]

Actually, on another apk that had this issue, I managed to make recompile work (and also to correctly run on the emulator the recompiled apk after resigning it) by replacing: <id with: <item type="id" in the res/values/ids.xml file.
On the above mentioned titanium backup apk, the inconsistent files I edited so far using this strategy are:

res/values/ids.xml
res/values/layouts.xml
res/values/drawables.xml

but now apktool complains about res/values/colors.xml, it seems because in this case the xml tag has no content (apart from the name property).
Still, it is unclear to me why apktool generates these xml files with inconsistent schema.

Disclaimer: what I'm trying to do is recompiling some apks I don't trust, after having removed INTERNET permission from manifest (this avoids the burden of installing 3rd party firewalls for having the same functionality, since android doesn't have a decent way of providing firewall feature at the OS level)

[warren-bank]

As an aside.. we should really set up some kind of repo:

• with a branch for each:
  ${apk_package_id}/${apk_version_code}/${apk_sha1_hash}/${apktool_version}
• and share our patched smali files
• or better yet..
  • just diffs
  • since apktool output is deterministic given:
    • identical input apk files
    • identical versions of apktool
  • and identical apk files can be guaranteed by SHA1 hash

I don't know about any of you, but:

• I'd never run an apk that isn't signed by its original author
  • which rules out running apks modded by anyone other than myself
• I'm sitting on a huge database of diffs that I'd be willing to share
  • which would allow others to apply my exact same changes to a trusted/original version
• I really don't want to get into any kind of legal trouble for doing so..

[pgp]

Actually, I started experimenting with apktool just for curiosity, so that when I found those apk that gave errors during recompilation, I also tried to reproduce this by using an obfuscation dictionary in proguard rules in one application of mine: I thought these errors were due to usage of non-ascii characters in proguard mappings, yet the apk generated for my application decompiled and recompiled fine even with the proguard rules modification.
Now I'm even more curious to know how to reproduce this scenario in my source code, so any hint of possible proguard settings responsible of this issue is welcome.

[pgp]

BTW: you were talking about modifying smali files, it's clearly not my case, so this issue should be related to something else at least.