Apk built with dexguard is not getting decoded. #1014

Closed
greadercha opened this Issue Aug 3, 2015 · 6 comments

Comments

Projects
None yet
2 participants
@greadercha

Hi,

I am working on project where I need to decode apks built by our dev team as part of build processing.
I wrote script using apktool to decode apks which works fine with normal apks and proguarded apks.
But its giving diff errors for apks built using Dexguard Enterprise.

Eventhough errors are diff, I suspect underlying problem is same, apktool is not able to read resource table.

output of team apk:

I: Using Apktool 2.0.0 on team_app-release.apk
I: Loading resource table...
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 3
        at brut.androlib.res.decoder.ARSCDecoder.readConfig(ARSCDecoder.java:187)
        at brut.androlib.res.decoder.ARSCDecoder.readType(ARSCDecoder.java:157)
        at brut.androlib.res.decoder.ARSCDecoder.readPackage(ARSCDecoder.java:114)
        at brut.androlib.res.decoder.ARSCDecoder.readTable(ARSCDecoder.java:78)
        at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:47)
        at brut.androlib.res.AndrolibResources.getResPackagesFromApk(AndrolibResources.java:538)
        at brut.androlib.res.AndrolibResources.loadMainPkg(AndrolibResources.java:63)
        at brut.androlib.res.AndrolibResources.getResTable(AndrolibResources.java:55)
        at brut.androlib.Androlib.getResTable(Androlib.java:64)
        at brut.androlib.ApkDecoder.setTargetSdkVersion(ApkDecoder.java:209)
        at brut.androlib.ApkDecoder.decode(ApkDecoder.java:92)
        at brut.apktool.Main.cmdDecode(Main.java:165)
        at brut.apktool.Main.main(Main.java:81)

output of test apk:

I: Using Apktool 2.0.0 on test_app-release.apk
I: Loading resource table...
Exception in thread "main" brut.androlib.AndrolibException: Multiple resources: spec=0x7f070000 id/ic_launcher, config=[DEFAULT]
        at brut.androlib.res.data.ResConfig.addResource(ResConfig.java:63)
        at brut.androlib.res.data.ResConfig.addResource(ResConfig.java:56)
        at brut.androlib.res.decoder.ARSCDecoder.readEntry(ARSCDecoder.java:218)
        at brut.androlib.res.decoder.ARSCDecoder.readConfig(ARSCDecoder.java:189)
        at brut.androlib.res.decoder.ARSCDecoder.readType(ARSCDecoder.java:157)
        at brut.androlib.res.decoder.ARSCDecoder.readPackage(ARSCDecoder.java:114)
        at brut.androlib.res.decoder.ARSCDecoder.readTable(ARSCDecoder.java:78)
        at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:47)
        at brut.androlib.res.AndrolibResources.getResPackagesFromApk(AndrolibResources.java:538)
        at brut.androlib.res.AndrolibResources.loadMainPkg(AndrolibResources.java:63)
        at brut.androlib.res.AndrolibResources.getResTable(AndrolibResources.java:55)
        at brut.androlib.Androlib.getResTable(Androlib.java:64)
        at brut.androlib.ApkDecoder.setTargetSdkVersion(ApkDecoder.java:209)
        at brut.androlib.ApkDecoder.decode(ApkDecoder.java:92)
        at brut.apktool.Main.cmdDecode(Main.java:165)
        at brut.apktool.Main.main(Main.java:81)

@greadercha greadercha changed the title from Apk built with dexguard is getting decoded. to Apk built with dexguard is not getting decoded. Aug 3, 2015

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Aug 4, 2015

Owner

and holy cow this is an improvement over previous versions of Dexguard.

This enterprise version obfuscates/encrypts resources. I'm not sure how it does this, unless it decrypts them on the fly, but apktool's responsibility is simply decoding the apk. I will see what I can do.

Owner

iBotPeaches commented Aug 4, 2015

and holy cow this is an improvement over previous versions of Dexguard.

This enterprise version obfuscates/encrypts resources. I'm not sure how it does this, unless it decrypts them on the fly, but apktool's responsibility is simply decoding the apk. I will see what I can do.

@iBotPeaches iBotPeaches added this to the 2.0.3 milestone Oct 12, 2015

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Dec 15, 2015

Owner

Made progress on this after committing - eabb7d8

Exception in thread "main" brut.androlib.AndrolibException: Multiple res specs: attr/
    at brut.androlib.res.data.ResTypeSpec.addResSpec(ResTypeSpec.java:78)
    at brut.androlib.res.decoder.ARSCDecoder.readEntry(ARSCDecoder.java:248)
    at brut.androlib.res.decoder.ARSCDecoder.readTableType(ARSCDecoder.java:212)
    at brut.androlib.res.decoder.ARSCDecoder.readTableTypeSpec(ARSCDecoder.java:154)
    at brut.androlib.res.decoder.ARSCDecoder.readTablePackage(ARSCDecoder.java:116)
    at brut.androlib.res.decoder.ARSCDecoder.readTableHeader(ARSCDecoder.java:78)
    at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:47)
    at brut.androlib.res.AndrolibResources.getResPackagesFromApk(AndrolibResources.java:544)
    at brut.androlib.res.AndrolibResources.loadMainPkg(AndrolibResources.java:63)
    at brut.androlib.res.AndrolibResources.getResTable(AndrolibResources.java:55)
    at brut.androlib.Androlib.getResTable(Androlib.java:66)
    at brut.androlib.ApkDecoder.setTargetSdkVersion(ApkDecoder.java:198)
    at brut.androlib.ApkDecoder.decode(ApkDecoder.java:96)
    at brut.apktool.Main.cmdDecode(Main.java:165)
    at brut.apktool.Main.main(Main.java:81)

This is another creative tweak. All resources have different IDs, but their names aren't visible to me, so they are set to null. Therefore 2nd resource is a duplicate. Must be some more UTF8 changes vs ASCII.

Owner

iBotPeaches commented Dec 15, 2015

Made progress on this after committing - eabb7d8

Exception in thread "main" brut.androlib.AndrolibException: Multiple res specs: attr/
    at brut.androlib.res.data.ResTypeSpec.addResSpec(ResTypeSpec.java:78)
    at brut.androlib.res.decoder.ARSCDecoder.readEntry(ARSCDecoder.java:248)
    at brut.androlib.res.decoder.ARSCDecoder.readTableType(ARSCDecoder.java:212)
    at brut.androlib.res.decoder.ARSCDecoder.readTableTypeSpec(ARSCDecoder.java:154)
    at brut.androlib.res.decoder.ARSCDecoder.readTablePackage(ARSCDecoder.java:116)
    at brut.androlib.res.decoder.ARSCDecoder.readTableHeader(ARSCDecoder.java:78)
    at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:47)
    at brut.androlib.res.AndrolibResources.getResPackagesFromApk(AndrolibResources.java:544)
    at brut.androlib.res.AndrolibResources.loadMainPkg(AndrolibResources.java:63)
    at brut.androlib.res.AndrolibResources.getResTable(AndrolibResources.java:55)
    at brut.androlib.Androlib.getResTable(Androlib.java:66)
    at brut.androlib.ApkDecoder.setTargetSdkVersion(ApkDecoder.java:198)
    at brut.androlib.ApkDecoder.decode(ApkDecoder.java:96)
    at brut.apktool.Main.cmdDecode(Main.java:165)
    at brut.apktool.Main.main(Main.java:81)

This is another creative tweak. All resources have different IDs, but their names aren't visible to me, so they are set to null. Therefore 2nd resource is a duplicate. Must be some more UTF8 changes vs ASCII.

@iBotPeaches iBotPeaches modified the milestones: 2.1.0, 2.0.3 Dec 15, 2015

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Dec 25, 2015

Owner

A update for this. As my last update said, there are resources which names decode to an empty value. Which might be a decode failure of some crazy UTF8 char, or literally an empty value.

So there are 2 problems that arise here.

  1. Resources have the same name, which is just their type and then an empty value. So after we read one String, then its name is string/ (since no name was found). This means the next String that is read becomes string/ which throws the Multiple res specs failure.
  2. Empty strings can't be regex`d matched. Easy fix.

I patched these, but this introduces an APK that is obviously not like the original, but hey the decode works. That counts for something right?

So then I started looking at the dummy resources. The values of the resources are legit and used sometime, but the name of such resource is bogus and thus replaced with APKTOOL_DUMMYVAL_{HEX_RESID}. The source (smali) though goes by resourceId which is in tact, the only place that used the name of the resource was the resource files and those are stored by resId. Only during decode do we replace with the human readable name value, which at this point becomes the dummy value we create.

So can we build the apk back? Not all, we then run into the #885 bug (Which means a custom aapt change is used to allow such characters)

Merry Christmas, btw.

Owner

iBotPeaches commented Dec 25, 2015

A update for this. As my last update said, there are resources which names decode to an empty value. Which might be a decode failure of some crazy UTF8 char, or literally an empty value.

So there are 2 problems that arise here.

  1. Resources have the same name, which is just their type and then an empty value. So after we read one String, then its name is string/ (since no name was found). This means the next String that is read becomes string/ which throws the Multiple res specs failure.
  2. Empty strings can't be regex`d matched. Easy fix.

I patched these, but this introduces an APK that is obviously not like the original, but hey the decode works. That counts for something right?

So then I started looking at the dummy resources. The values of the resources are legit and used sometime, but the name of such resource is bogus and thus replaced with APKTOOL_DUMMYVAL_{HEX_RESID}. The source (smali) though goes by resourceId which is in tact, the only place that used the name of the resource was the resource files and those are stored by resId. Only during decode do we replace with the human readable name value, which at this point becomes the dummy value we create.

So can we build the apk back? Not all, we then run into the #885 bug (Which means a custom aapt change is used to allow such characters)

Merry Christmas, btw.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Dec 26, 2015

Owner

You can decode applications with this patch - f932394

No recompile yet due to #885

Owner

iBotPeaches commented Dec 26, 2015

You can decode applications with this patch - f932394

No recompile yet due to #885

@iBotPeaches iBotPeaches closed this Feb 4, 2016

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Feb 4, 2016

Owner

oops. wrong issue.

Owner

iBotPeaches commented Feb 4, 2016

oops. wrong issue.

@iBotPeaches iBotPeaches reopened this Feb 4, 2016

@iBotPeaches iBotPeaches modified the milestones: 2.1.1, 2.1.0 Mar 27, 2016

@iBotPeaches iBotPeaches modified the milestones: 2.1.2, 2.1.1 May 6, 2016

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Aug 6, 2016

Owner

Since the last thing in this bug is actually Bug #885, I'm closing this to help clean up the tracker. My guess is that once Android N drops and aapt2, I'll have to rediscover all my patches to the binary and at that point I can patch it to allow non-standard characters.

Owner

iBotPeaches commented Aug 6, 2016

Since the last thing in this bug is actually Bug #885, I'm closing this to help clean up the tracker. My guess is that once Android N drops and aapt2, I'll have to rediscover all my patches to the binary and at that point I can patch it to allow non-standard characters.

@iBotPeaches iBotPeaches closed this Aug 6, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment