New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Breaks on decoding new Google Search apk #773

Closed
iBotPeaches opened this Issue Mar 18, 2015 · 20 comments

Comments

Projects
None yet
1 participant
@iBotPeaches
Owner

iBotPeaches commented Mar 18, 2015

Original issue 664 created by archon810 on 2014-08-06T19:08:32.000Z:

http://www.androidfilehost.com/?fid=23610159112651142

I: Loading resource table...
Exception in thread "main" brut.androlib.AndrolibException: Could not decode ars
c file
at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:54)
at brut.androlib.res.AndrolibResources.getResPackagesFromApk(AndrolibRes
ources.java:604)
at brut.androlib.res.AndrolibResources.loadMainPkg(AndrolibResources.jav
a:74)
at brut.androlib.res.AndrolibResources.getResTable(AndrolibResources.jav
a:66)
at brut.androlib.Androlib.getResTable(Androlib.java:49)
at brut.androlib.ApkDecoder.decode(ApkDecoder.java:93)
at brut.apktool.Main.cmdDecode(Main.java:169)
at brut.apktool.Main.main(Main.java:85)
Caused by: java.io.IOException: Expected: 0x001c0001, got: 0x00000000
at brut.util.ExtDataInput.skipCheckInt(ExtDataInput.java:48)
at brut.androlib.res.decoder.StringBlock.read(StringBlock.java:43)
at brut.androlib.res.decoder.ARSCDecoder.readPackage(ARSCDecoder.java:95
)
at brut.androlib.res.decoder.ARSCDecoder.readTable(ARSCDecoder.java:81)
at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:49)
... 7 more

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #1 originally posted by connor.tumbleson on 2014-08-06T19:41:30.000Z:

hmmm.

(for me)
https://github.com/android/platform_frameworks_base/commits/6e2fb587d71631bd29fdb844a6451d68e2144337/include/androidfw/ResourceTypes.h

Never seen it fail this early before. Its looking for the header for a familiar chunk and not finding it. Looking at the last few commits to AOSP, I don't see anything that has directly affected this chunk data, so not sure at the moment.

Owner

iBotPeaches commented Mar 18, 2015

Comment #1 originally posted by connor.tumbleson on 2014-08-06T19:41:30.000Z:

hmmm.

(for me)
https://github.com/android/platform_frameworks_base/commits/6e2fb587d71631bd29fdb844a6451d68e2144337/include/androidfw/ResourceTypes.h

Never seen it fail this early before. Its looking for the header for a familiar chunk and not finding it. Looking at the last few commits to AOSP, I don't see anything that has directly affected this chunk data, so not sure at the moment.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #2 originally posted by connor.tumbleson on 2014-09-26T11:43:04.000Z:

Issue 680 has been merged into this issue.

Owner

iBotPeaches commented Mar 18, 2015

Comment #2 originally posted by connor.tumbleson on 2014-09-26T11:43:04.000Z:

Issue 680 has been merged into this issue.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #3 originally posted by connor.tumbleson on 2014-09-26T11:45:45.000Z:

This issue is now top priority. It seems the newest updates of Google apps are breaking on this.

Data shifted somewhere and is breaking things. AOSP isn't helping locate this so its gonna be a fun weekend.

Owner

iBotPeaches commented Mar 18, 2015

Comment #3 originally posted by connor.tumbleson on 2014-09-26T11:45:45.000Z:

This issue is now top priority. It seems the newest updates of Google apps are breaking on this.

Data shifted somewhere and is breaking things. AOSP isn't helping locate this so its gonna be a fun weekend.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #4 originally posted by e11880@motorola.com on 2014-09-30T21:36:47.000Z:

Is there a workaround for this issue?

Owner

iBotPeaches commented Mar 18, 2015

Comment #4 originally posted by e11880@motorola.com on 2014-09-30T21:36:47.000Z:

Is there a workaround for this issue?

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #5 originally posted by pzhang9999 on 2014-09-30T21:39:05.000Z:

Is there a workaround for this issue? Thx.

Owner

iBotPeaches commented Mar 18, 2015

Comment #5 originally posted by pzhang9999 on 2014-09-30T21:39:05.000Z:

Is there a workaround for this issue? Thx.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #6 originally posted by connor.tumbleson on 2014-09-30T21:55:22.000Z:

# 5 - Not yet, I do have a commit in process with good progress though. All updates regarding it will be posted in this issue.

(Now that you've responded you will get updates about changes). Others may "star" the issue to receive updates instead of responding.

Owner

iBotPeaches commented Mar 18, 2015

Comment #6 originally posted by connor.tumbleson on 2014-09-30T21:55:22.000Z:

# 5 - Not yet, I do have a commit in process with good progress though. All updates regarding it will be posted in this issue.

(Now that you've responded you will get updates about changes). Others may "star" the issue to receive updates instead of responding.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #7 originally posted by nico.de.ceulaer on 2014-10-02T09:14:36.000Z:

I'm hitting the same problem on another Android application provided by Google.
Please let me know if I can help out in fixing this issue by testing possible fixes.

Owner

iBotPeaches commented Mar 18, 2015

Comment #7 originally posted by nico.de.ceulaer on 2014-10-02T09:14:36.000Z:

I'm hitting the same problem on another Android application provided by Google.
Please let me know if I can help out in fixing this issue by testing possible fixes.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #8 originally posted by connor.tumbleson on 2014-10-02T19:13:53.000Z:

<empty>

Owner

iBotPeaches commented Mar 18, 2015

Comment #8 originally posted by connor.tumbleson on 2014-10-02T19:13:53.000Z:

<empty>

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #9 originally posted by connor.tumbleson on 2014-10-05T05:01:40.000Z:

[Update for myself - notes]

PhoneSky and GooogleSearch have the exact problem. After the first ResChunk_header (after we pull table strings, and package count). We iterate into the first package and get id / name of package successfully.

We pull typeStrings, lastPublicType, keyStrings and lastPublicKey perfectly fine. The problem comes when trying to pull the typeNames * specName ResStringPools.

The skipIntCheck we have fails, like it should. In the case of Phonesky, the pointer is at 0x5B5C68 looking for 0x01001C. When in reality the header it is looking for is located at 0x5B5D40, 0xD8 (216) away.

Manually setting the correct pointer as you debug fixes each string block, one by one. So just setting the correct pointer fixes the problem completely. The problem is understanding why there is this extra data in only new applications and why its not documented anywhere: https://github.com/android/platform_frameworks_base/blob/master/include/androidfw/ResourceTypes.h#L177

Building smaller applications to reduce overhead in hex editing HUGE google apps cannot replicate this. L apks, wear apks, etc. Creating any sort of check is useless since the data ranges in size and location constantly. We have to figure out what this new data is in between the ResTable_package and before the ResStringPool_header starts.

Running PhoneSky again and comparing it to the ResTable_package https://github.com/android/platform_frameworks_base/blob/master/include/androidfw/ResourceTypes.h#L801 we get these variables correctly. So the start of the ResTable_pacakge can be pulled without error on these apks.

128 / 0x7F
com.android.vending
typeStrings = 288
lastPublicType = 15
keyStrings = 500
lastPublicKey = 3000

My guess is that our assumption of all ResTable_package having an immediate ResStringPool right after is incorrect. We assume it, but AOSP lists 3 other possibilities that can be in that header "type" positon.

enum {
RES_NULL_TYPE = 0x0000,
RES_STRING_POOL_TYPE = 0x0001,
RES_TABLE_TYPE = 0x0002,
RES_XML_TYPE = 0x0003,
....
}

Assuming something other than a StringPool our first header after the ResTable_package would be RES_NULL_TYPE, since 0x5B5C68 returns 0x0000. Though this is just a theory since the struct of RES_NULL_TYPE, RES_TABLE_TYPE & RES_XML_TYPE are unknown to me currently.

Owner

iBotPeaches commented Mar 18, 2015

Comment #9 originally posted by connor.tumbleson on 2014-10-05T05:01:40.000Z:

[Update for myself - notes]

PhoneSky and GooogleSearch have the exact problem. After the first ResChunk_header (after we pull table strings, and package count). We iterate into the first package and get id / name of package successfully.

We pull typeStrings, lastPublicType, keyStrings and lastPublicKey perfectly fine. The problem comes when trying to pull the typeNames * specName ResStringPools.

The skipIntCheck we have fails, like it should. In the case of Phonesky, the pointer is at 0x5B5C68 looking for 0x01001C. When in reality the header it is looking for is located at 0x5B5D40, 0xD8 (216) away.

Manually setting the correct pointer as you debug fixes each string block, one by one. So just setting the correct pointer fixes the problem completely. The problem is understanding why there is this extra data in only new applications and why its not documented anywhere: https://github.com/android/platform_frameworks_base/blob/master/include/androidfw/ResourceTypes.h#L177

Building smaller applications to reduce overhead in hex editing HUGE google apps cannot replicate this. L apks, wear apks, etc. Creating any sort of check is useless since the data ranges in size and location constantly. We have to figure out what this new data is in between the ResTable_package and before the ResStringPool_header starts.

Running PhoneSky again and comparing it to the ResTable_package https://github.com/android/platform_frameworks_base/blob/master/include/androidfw/ResourceTypes.h#L801 we get these variables correctly. So the start of the ResTable_pacakge can be pulled without error on these apks.

128 / 0x7F
com.android.vending
typeStrings = 288
lastPublicType = 15
keyStrings = 500
lastPublicKey = 3000

My guess is that our assumption of all ResTable_package having an immediate ResStringPool right after is incorrect. We assume it, but AOSP lists 3 other possibilities that can be in that header "type" positon.

enum {
RES_NULL_TYPE = 0x0000,
RES_STRING_POOL_TYPE = 0x0001,
RES_TABLE_TYPE = 0x0002,
RES_XML_TYPE = 0x0003,
....
}

Assuming something other than a StringPool our first header after the ResTable_package would be RES_NULL_TYPE, since 0x5B5C68 returns 0x0000. Though this is just a theory since the struct of RES_NULL_TYPE, RES_TABLE_TYPE & RES_XML_TYPE are unknown to me currently.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #10 originally posted by connor.tumbleson on 2014-10-05T17:23:19.000Z:

I believe this is fixed as of this: #92

Just needed to sleep on it. It works for the broken apks provided. I'm going to run a few more tests, then slice an RC2 release today if all goes well.

Owner

iBotPeaches commented Mar 18, 2015

Comment #10 originally posted by connor.tumbleson on 2014-10-05T17:23:19.000Z:

I believe this is fixed as of this: #92

Just needed to sleep on it. It works for the broken apks provided. I'm going to run a few more tests, then slice an RC2 release today if all goes well.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #11 originally posted by connor.tumbleson on 2014-10-05T18:20:43.000Z:

0912ac6

Owner

iBotPeaches commented Mar 18, 2015

Comment #11 originally posted by connor.tumbleson on 2014-10-05T18:20:43.000Z:

0912ac6

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #12 originally posted by archon810 on 2014-10-05T19:10:56.000Z:

Yay!

Owner

iBotPeaches commented Mar 18, 2015

Comment #12 originally posted by archon810 on 2014-10-05T19:10:56.000Z:

Yay!

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #13 originally posted by archon810 on 2014-10-05T20:18:20.000Z:

Can we download the updated version somewhere?

Owner

iBotPeaches commented Mar 18, 2015

Comment #13 originally posted by archon810 on 2014-10-05T20:18:20.000Z:

Can we download the updated version somewhere?

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #14 originally posted by archon810 on 2014-10-06T01:05:08.000Z:

Tried rc2. Got a crash here: https://gist.github.com/a5a2c68b8a722dd5f7c8.

APK is here: http://www.apkmirror.com/apk/google-inc/google-play-newsstand/google-play-newsstand-3-3-2-apk/.

Owner

iBotPeaches commented Mar 18, 2015

Comment #14 originally posted by archon810 on 2014-10-06T01:05:08.000Z:

Tried rc2. Got a crash here: https://gist.github.com/a5a2c68b8a722dd5f7c8.

APK is here: http://www.apkmirror.com/apk/google-inc/google-play-newsstand/google-play-newsstand-3-3-2-apk/.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #15 originally posted by Michael.j.Churchward on 2014-12-16T10:59:45.000Z:

tried RC3 with SystemUI.apk out of samsung SGS5 build here: https://dl.dropboxusercontent.com/u/14028045/SystemUI.apk

got this which i think is the same problem?

C:\Dev\apktool>java -jar apktool.jar d -f c:\dev\systemui\systemui.apk
I: Using Apktool 2.0.0-RC3 on systemui.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: C:\Users\Michael\apktool\framework\1.apk
Exception in thread "main" java.lang.ClassCastException: brut.androlib.res.data.value.ResStringValue cannot be cast to brut.androlib.res.data.value.ResAttr
at brut.androlib.res.decoder.ResAttrDecoder.decode(ResAttrDecoder.java:36)
at brut.androlib.res.decoder.AXmlResourceParser.getAttributeValue(AXmlResourceParser.java:369)
at org.xmlpull.v1.wrapper.classic.XmlPullParserDelegate.getAttributeValue(XmlPullParserDelegate.java:69)
at org.xmlpull.v1.wrapper.classic.StaticXmlSerializerWrapper.writeStartTag(StaticXmlSerializerWrapper.java:267)
at org.xmlpull.v1.wrapper.classic.StaticXmlSerializerWrapper.event(StaticXmlSerializerWrapper.java:211)
at brut.androlib.res.decoder.XmlPullStreamDecoder$1.event(XmlPullStreamDecoder.java:83)
at brut.androlib.res.decoder.XmlPullStreamDecoder.decode(XmlPullStreamDecoder.java:141)
at brut.androlib.res.decoder.XmlPullStreamDecoder.decodeManifest(XmlPullStreamDecoder.java:153)
at brut.androlib.res.decoder.ResFileDecoder.decodeManifest(ResFileDecoder.java:134)
at brut.androlib.res.AndrolibResources.decode(AndrolibResources.java:296)
at brut.androlib.Androlib.decodeResourcesFull(Androlib.java:131)
at brut.androlib.ApkDecoder.decode(ApkDecoder.java:101)
at brut.apktool.Main.cmdDecode(Main.java:165)
at brut.apktool.Main.main(Main.java:81)

Owner

iBotPeaches commented Mar 18, 2015

Comment #15 originally posted by Michael.j.Churchward on 2014-12-16T10:59:45.000Z:

tried RC3 with SystemUI.apk out of samsung SGS5 build here: https://dl.dropboxusercontent.com/u/14028045/SystemUI.apk

got this which i think is the same problem?

C:\Dev\apktool>java -jar apktool.jar d -f c:\dev\systemui\systemui.apk
I: Using Apktool 2.0.0-RC3 on systemui.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: C:\Users\Michael\apktool\framework\1.apk
Exception in thread "main" java.lang.ClassCastException: brut.androlib.res.data.value.ResStringValue cannot be cast to brut.androlib.res.data.value.ResAttr
at brut.androlib.res.decoder.ResAttrDecoder.decode(ResAttrDecoder.java:36)
at brut.androlib.res.decoder.AXmlResourceParser.getAttributeValue(AXmlResourceParser.java:369)
at org.xmlpull.v1.wrapper.classic.XmlPullParserDelegate.getAttributeValue(XmlPullParserDelegate.java:69)
at org.xmlpull.v1.wrapper.classic.StaticXmlSerializerWrapper.writeStartTag(StaticXmlSerializerWrapper.java:267)
at org.xmlpull.v1.wrapper.classic.StaticXmlSerializerWrapper.event(StaticXmlSerializerWrapper.java:211)
at brut.androlib.res.decoder.XmlPullStreamDecoder$1.event(XmlPullStreamDecoder.java:83)
at brut.androlib.res.decoder.XmlPullStreamDecoder.decode(XmlPullStreamDecoder.java:141)
at brut.androlib.res.decoder.XmlPullStreamDecoder.decodeManifest(XmlPullStreamDecoder.java:153)
at brut.androlib.res.decoder.ResFileDecoder.decodeManifest(ResFileDecoder.java:134)
at brut.androlib.res.AndrolibResources.decode(AndrolibResources.java:296)
at brut.androlib.Androlib.decodeResourcesFull(Androlib.java:131)
at brut.androlib.ApkDecoder.decode(ApkDecoder.java:101)
at brut.apktool.Main.cmdDecode(Main.java:165)
at brut.apktool.Main.main(Main.java:81)

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #16 originally posted by feedprograms.com on 2015-01-11T03:49:09.000Z:

Can we download the updated version somewhere?
http://www.feedprograms.com

Owner

iBotPeaches commented Mar 18, 2015

Comment #16 originally posted by feedprograms.com on 2015-01-11T03:49:09.000Z:

Can we download the updated version somewhere?
http://www.feedprograms.com

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #17 originally posted by Aviron707 on 2015-01-12T17:41:08.000Z:

Got exactly same problem and same errors one by one as Michael got with Samsung Note 4 when trying to decode SystemUI.apk by using RC3.

Owner

iBotPeaches commented Mar 18, 2015

Comment #17 originally posted by Aviron707 on 2015-01-12T17:41:08.000Z:

Got exactly same problem and same errors one by one as Michael got with Samsung Note 4 when trying to decode SystemUI.apk by using RC3.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #18 originally posted by alberto.pedrosa@tuapli.com on 2015-02-15T15:38:12.000Z:

i´ve the same issue, windows 8.1 and the last version of apktool, i tried with apkmanager and same result. Please help us! :(

Owner

iBotPeaches commented Mar 18, 2015

Comment #18 originally posted by alberto.pedrosa@tuapli.com on 2015-02-15T15:38:12.000Z:

i´ve the same issue, windows 8.1 and the last version of apktool, i tried with apkmanager and same result. Please help us! :(

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #19 originally posted by bastanijoon on 2015-02-19T16:50:40.000Z:

Funny thing is if you rename any of the resources xml files to "AndroidManifext.xml" and just zip this xml file as an apk then the apktool will decode it :)

Owner

iBotPeaches commented Mar 18, 2015

Comment #19 originally posted by bastanijoon on 2015-02-19T16:50:40.000Z:

Funny thing is if you rename any of the resources xml files to "AndroidManifext.xml" and just zip this xml file as an apk then the apktool will decode it :)

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Mar 18, 2015

Owner

Comment #20 originally posted by bastanijoon on 2015-02-19T16:52:54.000Z:

Funny thing is if you rename any of the resources xml files to "AndroidManifest.xml" and just zip this xml file as an apk then the apktool will decode it :)

Owner

iBotPeaches commented Mar 18, 2015

Comment #20 originally posted by bastanijoon on 2015-02-19T16:52:54.000Z:

Funny thing is if you rename any of the resources xml files to "AndroidManifest.xml" and just zip this xml file as an apk then the apktool will decode it :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment