Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Android O Dev Preview Support #1453

Closed
argraur opened this issue Mar 21, 2017 · 36 comments · Fixed by #1493
Closed

Android O Dev Preview Support #1453

argraur opened this issue Mar 21, 2017 · 36 comments · Fixed by #1493
Assignees
Labels
Milestone

Comments

@argraur
Copy link

argraur commented Mar 21, 2017

Information

  1. Apktool Version (Apktool v2.2.2) -
  2. Operating System (Windows) -
  3. APK From? (ROM) -

Stacktrace/Logcat


C:\Users\Александр\Desktop\bullhead-opp1.170223.012-factory-338a95b1\bullhead-opp1.170223.012\system_\framework>java -jar apktool_2.2.2.jar if framework-res.apk
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: 111
        at java.lang.String.substring(Unknown Source)
        at brut.androlib.res.decoder.StringBlock.getHTML(StringBlock.java:162)
        at brut.androlib.res.decoder.ARSCDecoder.readValue(ARSCDecoder.java:315)
        at brut.androlib.res.decoder.ARSCDecoder.readEntry(ARSCDecoder.java:241)
        at brut.androlib.res.decoder.ARSCDecoder.readTableType(ARSCDecoder.java:226)
        at brut.androlib.res.decoder.ARSCDecoder.readTableTypeSpec(ARSCDecoder.java:156)
        at brut.androlib.res.decoder.ARSCDecoder.readTablePackage(ARSCDecoder.java:118)
        at brut.androlib.res.decoder.ARSCDecoder.readTableHeader(ARSCDecoder.java:80)
        at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:47)
        at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:39)
        at brut.androlib.res.AndrolibResources.installFramework(AndrolibResources.java:640)
        at brut.androlib.res.AndrolibResources.installFramework(AndrolibResources.java:622)
        at brut.androlib.Androlib.installFramework(Androlib.java:683)
        at brut.apktool.Main.cmdInstallFramework(Main.java:241)
        at brut.apktool.Main.main(Main.java:87)

java -jar apktool_2.2.2.jar if framework-res.apk

@iBotPeaches
Copy link
Owner

and thus the first Android O report was made :)

Yes, I know.

@argraur
Copy link
Author

argraur commented Mar 21, 2017

Hope you will fix it soon bro ;)

@argraur
Copy link
Author

argraur commented Mar 22, 2017

Also ApkTool cannot parse SettingsGoogle and SystemUI:

java -jar apktool_2.2.1.jar d SettingsGoogle.apk
I: Using Apktool 2.2.1 on SettingsGoogle.apk
I: Loading resource table...
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: 227
        at java.lang.String.substring(Unknown Source)
        at brut.androlib.res.decoder.StringBlock.getHTML(StringBlock.java:155)
        at brut.androlib.res.decoder.ARSCDecoder.readValue(ARSCDecoder.java:315)
        at brut.androlib.res.decoder.ARSCDecoder.readEntry(ARSCDecoder.java:241)
        at brut.androlib.res.decoder.ARSCDecoder.readTableType(ARSCDecoder.java:226)
        at brut.androlib.res.decoder.ARSCDecoder.readTableTypeSpec(ARSCDecoder.java:156)
        at brut.androlib.res.decoder.ARSCDecoder.readTablePackage(ARSCDecoder.java:118)
        at brut.androlib.res.decoder.ARSCDecoder.readTableHeader(ARSCDecoder.java:80)
        at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:47)
        at brut.androlib.res.AndrolibResources.getResPackagesFromApk(AndrolibResources.java:558)
        at brut.androlib.res.AndrolibResources.loadMainPkg(AndrolibResources.java:72)
        at brut.androlib.res.AndrolibResources.getResTable(AndrolibResources.java:64)
        at brut.androlib.Androlib.getResTable(Androlib.java:68)
        at brut.androlib.ApkDecoder.setTargetSdkVersion(ApkDecoder.java:193)
        at brut.androlib.ApkDecoder.decode(ApkDecoder.java:102)
        at brut.apktool.Main.cmdDecode(Main.java:166)
        at brut.apktool.Main.main(Main.java:81)

java -jar apktool_2.2.1.jar d SystemUI.apk -f
I: Using Apktool 2.2.1 on SystemUI.apk
I: Loading resource table...
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: 240
        at java.lang.String.substring(Unknown Source)
        at brut.androlib.res.decoder.StringBlock.getHTML(StringBlock.java:162)
        at brut.androlib.res.decoder.ARSCDecoder.readValue(ARSCDecoder.java:315)
        at brut.androlib.res.decoder.ARSCDecoder.readEntry(ARSCDecoder.java:241)
        at brut.androlib.res.decoder.ARSCDecoder.readTableType(ARSCDecoder.java:226)
        at brut.androlib.res.decoder.ARSCDecoder.readTableTypeSpec(ARSCDecoder.java:156)
        at brut.androlib.res.decoder.ARSCDecoder.readTablePackage(ARSCDecoder.java:118)
        at brut.androlib.res.decoder.ARSCDecoder.readTableHeader(ARSCDecoder.java:80)
        at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:47)
        at brut.androlib.res.AndrolibResources.getResPackagesFromApk(AndrolibResources.java:558)
        at brut.androlib.res.AndrolibResources.loadMainPkg(AndrolibResources.java:72)
        at brut.androlib.res.AndrolibResources.getResTable(AndrolibResources.java:64)
        at brut.androlib.Androlib.getResTable(Androlib.java:68)
        at brut.androlib.ApkDecoder.setTargetSdkVersion(ApkDecoder.java:193)
        at brut.androlib.ApkDecoder.decode(ApkDecoder.java:102)
        at brut.apktool.Main.cmdDecode(Main.java:166)
        at brut.apktool.Main.main(Main.java:81)

@alice-mkh
Copy link

Same here.

@iBotPeaches
Copy link
Owner

Initial research shows that I won't be able to fix anything until official release. There is a new method of how text is stored (UTF8/UTF16) and the easiest way is to inspect commits to AOSP during release to patch the changes in.

Configurations and qualifiers look the same at initial glance of the framework file provided, so its much more low level which requires AOSP code access or days upon days of black box work.

@iBotPeaches iBotPeaches self-assigned this Mar 25, 2017
@iBotPeaches iBotPeaches added this to the 2.2.3 milestone Mar 25, 2017
@Furniel
Copy link
Contributor

Furniel commented Mar 30, 2017

Fixed few apktool exceptions(1, 2, 3). With this fixes apktool can install android O frameworks and decode android O apks(only known resources). Of course there will be a lot of warnings that unknown resources were skipped and such apks can't be compiled back(some resources will be missing), but i think that it's better than nothing.

PS: Didn't create pull request cause i'm not sure that it's good idea pull this to master.

@iBotPeaches
Copy link
Owner

Thanks!

1 scares me a bit, because I think the real fix for that is higher up the chain and that is simply catching the failure point. 2,3 look fine in my initial look but then again I don't think they will be a problem once 1 is properly fixed

@argraur
Copy link
Author

argraur commented Apr 4, 2017

Hey Google release Android O DP 1 source: https://android.googlesource.com/platform/manifest/+/refs/heads/android-o-preview-1

@iBotPeaches
Copy link
Owner

aosp-mirror/platform_frameworks_base@76da37e - No longer need AndroidManifest.xml in frameworks. Host constraint for them is gone.

Need to dig more. Haven't found any changes to StringPool

@argraur
Copy link
Author

argraur commented Apr 5, 2017

I saw last pull request.. So apktool now works with O?

@iBotPeaches
Copy link
Owner

Negative, but progress.

@argraur
Copy link
Author

argraur commented Apr 5, 2017

Btw I built and tested, nothing changed:
java -jar apktool-cli.jar if framework-res.apk
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: 111

@iBotPeaches
Copy link
Owner

The resource in question is

        resource 0x0104029c android:string/kg_reordering_delete_drop_target_text: t=0x03 d=0x000155d8 (s=0x0008 r=0x00)

Which reports the raw value as

‏‮Forgot‬‏ ‏‮your‬‏ ‏‮username‬‏ ‏‮or‬‏ ‏‮password?‬\n.‏‮Visit‬‏ ‏‮google.com/accounts/recover‬‏‏‮y‬‏

Apktool is failing on this due to an improper style decode. So not actually related to Android O at all. I'll get this solved soon. There aren't many tests written for this part of the codebase so I need to be careful patching this low level area.

@alice-mkh
Copy link

Umm, why is its value backwards?

@iBotPeaches
Copy link
Owner

It was during the ar-rXB, which is some form of Arabic. I posted wrong resource above. This is the correct one.

       resource 0x0104028e android:string/kg_login_account_recovery_hint: t=0x03 d=0x000001c5 (s=0x0008 r=0x00)
          (string8) "‏‮Forgot‬‏ ‏‮your‬‏ ‏‮username‬‏ ‏‮or‬‏ ‏‮password?‬‏\n‏‮Visit‬‏ ‏‮google.com/accounts/recover‬‏‏‮y‬‏‏‮.‬‏"

@iBotPeaches
Copy link
Owner

Okay, a WIP commit is up. I don't like the fix. Basically we have a UTF8 string, so our "length" count of it is 105, when in fact taking into consideration the true size of UTF8 chars we are looking at a size of 155 using Guava's (Utf8.encodeLength(raw)). For some reason, the system believes a style start exists at 111, so of course NullPointerException.

I don't know how to manipulate a UTF8 string correctly, w/ sub strings etc. I need to do some research into the proper fix for this. There is my WIP and #1 of @Furniel above both do something similar to this. I might look into why Apktool thinks a style exists in this string, unless a hyperlink is the style.

The message is

Forgot your username or password?
Visit google.com/accounts/recovery.

@iBotPeaches
Copy link
Owner

Next Problem.

    E: uses-sdk (line=0)
      A: android:minSdkVersion(0x0101020c)="O" (Raw: "O")
      A: android:targetSdkVersion(0x01010270)="O" (Raw: "O")

Never seen a letter used before for version. Apktool will fail trying to parse this as a number.

@RonAmadeo
Copy link

RonAmadeo commented May 5, 2017

You rock iBotPeaches! Not sure if this helps, but in past Developer Previews, letters have been used for min and target SDK versions and Apktool has never had a problem with it.

Attached is the Monospace APK compiled against the Android N Developer Preview, which works fine in APKtool:
E: uses-sdk (line=7)
A: android:minSdkVersion(0x0101020c)="N" (Raw: "N")
A: android:targetSdkVersion(0x01010270)="N" (Raw: "N")

Also in the zip is the Google Messages apk from the N preview, which is min 23, target n. It also works fine.

N apks.zip

@iBotPeaches
Copy link
Owner

    E: uses-sdk (line=7)
      A: android:minSdkVersion(0x0101020c)="N" (Raw: "N")
      A: android:targetSdkVersion(0x01010270)="N" (Raw: "N")

Underwood still has the same trouble as the O apks I have.

➜  N.apks apktool b com.underwood.monospace_2.0.1-33_minAPIN\(nodpi\)
I: Using Apktool 2.2.3-3e552c-SNAPSHOT
Exception in thread "main" java.lang.NumberFormatException: For input string: "N"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Integer.parseInt(Integer.java:580)
	at java.lang.Integer.parseInt(Integer.java:615)
	at brut.androlib.Androlib.build(Androlib.java:282)
	at brut.androlib.Androlib.build(Androlib.java:263)
	at brut.apktool.Main.cmdBuild(Main.java:231)
	at brut.apktool.Main.main(Main.java:84)

Easy to patch, strange that this is the first time I hit it. I downgraded to 2.2.1 which also had the problem.

➜  N.apks java -jar apktool_2.2.1.jar b com.underwood.monospace_2.0.1-33_minAPIN\(nodpi\)
I: Using Apktool 2.2.1
Exception in thread "main" java.lang.NumberFormatException: For input string: "N"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Integer.parseInt(Integer.java:580)
	at java.lang.Integer.parseInt(Integer.java:615)
	at brut.androlib.Androlib.build(Androlib.java:283)
	at brut.androlib.Androlib.build(Androlib.java:264)
	at brut.apktool.Main.cmdBuild(Main.java:227)
	at brut.apktool.Main.main(Main.java:84)
➜  N.apks 

I probably should have clarified. The decode is working fine, its the rebuild that is currently broken with O dev preview, and I guess N preview. I'll have some patches out this weekend for this original issue and this new, but old Developer Preview bug.

@iBotPeaches
Copy link
Owner

Okay, support for preview N and O APIs added. A small lookup table to map them to respective API levels, as we need to know the numerical value for building purposes (at the source and resource level). I never tested if aapt can take N/O, but I know for the source rebuild I need the numerical API.

Next error.

W: /home/ibotpeaches/Downloads/Apktool/Bug1453/framework-res_o/res/values/public.xml:1893: error: invalid symbol: 'default'
W: /home/ibotpeaches/Downloads/Apktool/Bug1453/framework-res_o/res/values-en-rXA/strings.xml:566: error: Error parsing XML: mismatched tag
W: /home/ibotpeaches/Downloads/Apktool/Bug1453/framework-res_o/res/values-ar-rXB/strings.xml:559: error: Error parsing XML: mismatched tag

The first one is a fake problem. AOSP strictly states you cannot use default as a resource name as stated in the valid_symbol_name function in the O Preview - https://github.com/android/platform_frameworks_base/blob/o-preview/tools/aapt/AaptAssets.cpp#L1622

Yet, it has one (in the official framework)

    <public type="id" name="default" id="0x0102020d" />
framework-res_o/res/values/attrs.xml:        <enum name="default" value="0" />

Not sure why its doing that, when its own aapt binary would prevent it. That is Android for you, maybe system frameworks aren't bound by those rules.

I guess since Apktool's job is to simply maintain the original apk, it should completely ignore reserved keywords. I'll need to patch all aapt binaries for this. Much like I did here for do and if - iBotPeaches/platform_frameworks_base@e37c8ea

The next two problems are probably a bug in Apktool's decoder. The error is about mismatched tags and we can see from the string

    <string name="keyguard_password_entry_touch_hint">[<font size="17">Ţåþ ţö ţýþé þåššŵöŕð one two three]</string>

It indeed is missing the closing </font> tag. I'll have to add a failing test case and debug our style decoder.

@iBotPeaches iBotPeaches changed the title Android O: ApkTool can not parse new framework-res.apk Android O Dev Preview Support May 8, 2017
@alice-mkh
Copy link

Or maybe the missing tag is intentional? The ęń-řXÅ ļóçăĺè is for debugging anyway, maybe you shouldn't even try to parse tags that are in the value strings?

@alice-mkh
Copy link

Also, is there a reason why values/en-rXA is missing from O Preview framework sources?
https://github.com/android/platform_frameworks_base/tree/o-preview/core/res/res

@iBotPeaches
Copy link
Owner

@Exalm I believe the debug strings are made on the build process stage with the aapt flag -c if my memory is correct, so they wouldn't be laying in the application.

@alice-mkh
Copy link

alice-mkh commented May 8, 2017

Ah, ok.
Looking at values/strings.xml, it seems that the closing tag is there:
< string name="keyguard_password_entry_touch_hint">< font size="17">Tap to type password< /font>< /string>
(Added spaces so that Github doesn't eat the tags)
So I was wrong about it.

@iBotPeaches
Copy link
Owner

I spent some time this morning looking at this. The failure in question has a style at index 1, ending at index 36 consisting of style 0xB703, which in the case of this application is font size="17". This, much like bold, italic, etc wraps the contents of the string (or substring) depending on the rule.

This is 1 style, so no depth. So according to our rules, string

[Ţåþ ţö ţýþé þåššŵöŕð one two three]

Should start at the 1 index, so we write out the 0 index char like so

[

Then we hit our style attribute at index 1.

[<font size="17">

So far so good, now this is where things mess up. Apktool currently knows the end tag is at index 36, but the string length is 36, so it writes out the remainder of the string due to a <= check.

[<font size="17">Ţåþ ţö ţýþé þåššŵöŕð one two three]

Now our position in the string is 36, so the string is complete without closing the tag. This is where things get interesting though. Like one of the original reports in this thread. This is an UTF8 encoded string, so the string length may be 36, but the byte count is 53 bytes.

So perhaps the index is in relation to the byte index and not string index, this would mean that 36 in byte, knowing that these weird characters take up 2 bytes per character would be the end of the funky chars. So taking a substring by the byte index value would become

[<font size="17">Ţåþ ţö ţýþé þåššŵöŕð</font> one two three]

Which is valid, and looking at the English translation

    <string name="keyguard_password_entry_touch_hint"><font size="17">Tap to type password</font></string>

Is a perfect match. I need to revert this commit - f3536ef and introduce UTF8 safe string operations.

Which means we need

  1. A safe length check for UTF8 and not. I think Guava and Sun have Utf8.encodedLength() for this.
  2. Next we need a substring that is UTF8 safe. I think we can use codePointAt and offsetByCodePoint which are methods on the string. This would return the code point and not the char point in the string.

Patching those two correctly should fix a lot more bugs than hacky if statements around these functions.

@argraur
Copy link
Author

argraur commented May 15, 2017

Hey, iBotPeaches I have removed all XA / XB translations.
Now we have another error.

W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:1893: error: invalid symbol: 'default'
W:
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6142: error: Public symbol array/config_autoBrightnessButtonBacklightValues declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6144: error: Public symbol array/config_autoBrightnessLcdBacklightValues declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6145: error: Public symbol array/config_autoBrightnessLevels declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6146: error: Public symbol array/config_autoRotationTiltTolerance declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6147: error: Public symbol array/config_calendarDateVibePattern declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6153: error: Public symbol array/config_clockTickVibePattern declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6154: error: Public symbol array/config_contextClickVibePattern declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6157: error: Public symbol array/config_defaultNotificationVibePattern declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6158: error: Public symbol array/config_defaultPinnerServiceFiles declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6160: error: Public symbol array/config_disabledUntilUsedPreinstalledCarrierApps declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6161: error: Public symbol array/config_disabledUntilUsedPreinstalledImes declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6162: error: Public symbol array/config_dynamicHysteresisBrightLevels declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6163: error: Public symbol array/config_dynamicHysteresisDarkLevels declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6165: error: Public symbol array/config_emergency_mcc_codes declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6166: error: Public symbol array/config_ephemeralResolverPackage declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6167: error: Public symbol array/config_globalActionsList declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6168: error: Public symbol array/config_gpsParameters declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6169: error: Public symbol array/config_keyboardTapVibePattern declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6170: error: Public symbol array/config_locationProviderPackageNames declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6171: error: Public symbol array/config_longPressVibePattern declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6172: error: Public symbol array/config_lteDbmThresholds declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6174: error: Public symbol array/config_mobile_tcp_buffers declared here is not defined.
Exception in thread "main" W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6176: error: Public symbol array/config_networkRecommendationPackageNames declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6177: error: Public symbol array/config_nonBlockableNotificationPackages declared here is not defined.
Exception in thread "main" W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6176: error: Public symbol array/config_networkRecommendationPackageNames declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6177: error: Public symbol array/config_nonBlockableNotificationPackages declared here is not defined.
brut.androlib.AndrolibException: brut.androlib.AndrolibException: brut.common.BrutException: could not exec (exit code = 1): [C:\Users\8523~1\AppData\Local\Temp\brut_util_Jar_5038231197882018220.tmp, p, --forced-package-id, 1, --min-sdk-version, O, --target-sdk-version, O, --version-code, 25, --version-name, O, --no-version-vectors, -F, C:\Users\8523~1\AppData\Local\Temp\APKTOOL6842162282118715500.tmp, -x, -0, arsc, -0, arsc, -S, C:\Users\Александр\Desktop\bullhead-opp1.170223.012\framework-res\res, -M, C:\Users\Александр\Desktop\bullhead-opp1.170223.012\framework-res\AndroidManifest.xml]
        at brut.androlib.Androlib.buildResourcesFull(Androlib.java:493)
        at brut.androlib.Androlib.buildResources(Androlib.java:427)
        at brut.androlib.Androlib.build(Androlib.java:326)
        at brut.androlib.Androlib.build(Androlib.java:264)
        at brut.apktool.Main.cmdBuild(Main.java:231)
        at brut.apktool.Main.main(Main.java:84)
Caused by: brut.androlib.AndrolibException: brut.common.BrutException: could not exec (exit code = 1): [C:\Users\8523~1\AppData\Local\Temp\brut_util_Jar_5038231197882018220.tmp, p, --forced-package-id, 1, --min-sdk-version, O, --target-sdk-version, O, --version-code, 25, --version-name, O, --no-version-vectors, -F, C:\Users\8523~1\AppData\Local\Temp\APKTOOL6842162282118715500.tmp, -x, -0, arsc, -0, arsc, -S, C:\Users\Александр\Desktop\bullhead-opp1.170223.012\framework-res\res, -M, C:\Users\Александр\Desktop\bullhead-opp1.170223.012\framework-res\AndroidManifest.xml]
        at brut.androlib.res.AndrolibResources.aaptPackage(AndrolibResources.java:441)
        at brut.androlib.Androlib.buildResourcesFull(Androlib.java:479)
        ... 5 more
Caused by: brut.common.BrutException: could not exec (exit code = 1): [C:\Users\8523~1\AppData\Local\Temp\brut_util_Jar_5038231197882018220.tmp, p, --forced-package-id, 1, --min-sdk-version, O, --target-sdk-version, O, --version-code, 25, --version-name, O, --no-version-vectors, -F, C:\Users\8523~1\AppData\Local\Temp\APKTOOL6842162282118715500.tmp, -x, -0, arsc, -0, arsc, -S, C:\Users\Александр\Desktop\bullhead-opp1.170223.012\framework-res\res, -M, C:\Users\Александр\Desktop\bullhead-opp1.170223.012\framework-res\AndroidManifest.xml]
        at brut.util.OS.exec(OS.java:95)
        at brut.androlib.res.AndrolibResources.aaptPackage(AndrolibResources.java:435)
        ... 6 more
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6178: error: Public symbol array/config_notificationFallbackVibePattern declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6179: error: Public symbol array/config_notificationSignalExtractors declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6182: error: Public symbol array/config_protectedNetworks declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6183: error: Public symbol array/config_safeModeDisabledVibePattern declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6184: error: Public symbol array/config_safeModeEnabledVibePattern declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6186: error: Public symbol array/config_scrollBarrierVibePattern declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6190: error: Public symbol array/config_statusBarIcons declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6191: error: Public symbol array/config_system_condition_providers declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6192: error: Public symbol array/config_telephonyHardware declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6194: error: Public symbol array/config_tether_bluetooth_regexs declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6132: error: Public symbol array/config_tether_upstream_types declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6196: error: Public symbol array/config_tether_usb_regexs declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6197: error: Public symbol array/config_tether_wifi_regexs declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6201: error: Public symbol array/config_virtualKeyVibePattern declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6202: error: Public symbol array/config_vvmSmsFilterRegexes declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6134: error: Public symbol array/emailAddressTypes declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6206: error: Public symbol array/imAddressTypes declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6135: error: Public symbol array/imProtocols declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6207: error: Public symbol array/maps_starting_lat_lng declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6208: error: Public symbol array/maps_starting_zoom declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6209: error: Public symbol array/networkAttributes declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6210: error: Public symbol array/network_switch_type_name declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6211: error: Public symbol array/networks_not_clear_data declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6212: error: Public symbol array/no_ems_support_sim_operators declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6136: error: Public symbol array/organizationTypes declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6137: error: Public symbol array/phoneTypes declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6138: error: Public symbol array/postalAddressTypes declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6213: error: Public symbol array/preloaded_color_state_lists declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6214: error: Public symbol array/preloaded_drawables declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6215: error: Public symbol array/preloaded_freeform_multi_window_drawables declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6216: error: Public symbol array/radioAttributes declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6217: error: Public symbol array/resolver_target_actions_pin declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6218: error: Public symbol array/resolver_target_actions_unpin declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6219: error: Public symbol array/sim_colors declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6220: error: Public symbol array/special_locale_codes declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6221: error: Public symbol array/special_locale_names declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6222: error: Public symbol array/supported_locales declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6223: error: Public symbol array/wfcOperatorErrorAlertMessages declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6224: error: Public symbol array/wfcOperatorErrorNotificationMessages declared here is not defined.
W: C:\Users\└ыхъёрэфЁ\Desktop\bullhead-opp1.170223.012\framework-res\res\values\public.xml:6225: error: Public symbol array/wfcSpnFormats declared here is not defined.

@iBotPeaches
Copy link
Owner

@argraur The first line is the error I detailed in this response - #1453 (comment)

In some situations, and I believe in this one, fixing that error will resolve the rest. You do bring up a good point though, rXB/rXA don't exist in the source application as they are generated at runtime via

   --pseudo-localize
       generate resources for pseudo-locales (en-XA and ar-XB).

So to @Exalm, confirmation on that. I didn't remember the parameter, but thought aapt was responsible for that. So what we could actually do here is hard delete psuedo locales during decode (if we find them), and then pass that parameter to aapt during build to generate them.

I think thats only a band aid on the UTF8 String problem, but just another avenue to think about. I have some good progress on a new UTF8 String class that handles length calculation and sub strings, but it's still a bit buggy. I should have something soon.

@iBotPeaches
Copy link
Owner

iBotPeaches commented May 19, 2017

Okay, had a little time this morning.

I've fixed the problem with AOSP using reserved keywords. Since apktool's job is to match the original application as much as possible, it shouldn't be responsible for enforcing keywords. If the original application uses them, it should as well. What that means on the host device among these different reserved keywords - I have no idea - iBotPeaches/platform_frameworks_base@c81b389

I need to build new platform binaries (unix 32/36, win, osx 32/64) still, but the patch for aapt is already committed.

The problem above with psuedo locales is a bit more complicated than I thought. Originally I thought the start/end index was the byte position and not character position, but that makes no sense if you process it. Imagine a character that represents 2 bytes like þ which is 2 bytes, but 1 character.

Imagine two of those characters, that would mean 4 bytes or 2 characters. If the start index was 1 that makes absolutely 0 sense at the byte level because you are not splitting a character in half to insert the style. So it makes sense that the start/end index for styles are actually codepoint indexes.

This makes sense, but still has the edge case with 2 issues.

  1. A style start/end index outside the "codepoint length" of a string. This isn't the case if you treat the length of a string as its byte size/length, but this train of thought does not work for all situations.

  2. A style end index at end of codepoint length of string. IE string is 36 characters and the end of style is 36. This introduces an edge case in apktool where it generates the beginning style tag and not end.

I originally wanted to strip psuedo locales from decompiled application and restore them during build, but that still has the problem with 1 above. This needs a lot more research and perhaps rewriting of the style decoder so for now, bandaid fixes - 390ecae6c569228

@argraur
Copy link
Author

argraur commented May 22, 2017

Hmm @iBotPeaches, I think that we can forget about pseudo-locales for now and focus on other issues (with public.xml that I showed in my previous post)

@iBotPeaches
Copy link
Owner

iBotPeaches commented May 23, 2017

Like a record, more time this morning. Attached are a few commits that fix the reserved resource name and a proper fix for the psuedo locale problem. This led us to another error during rebuild.

  1. d979247

  2. 59df689

➜  Bug1453 apktool b framework-res_o 
I: Using Apktool 2.2.3-390eca-SNAPSHOT
W: Could not find sources
I: Checking whether resources has changed...
I: Building resources...
W: /home/ibotpeaches/Downloads/Apktool/Bug1453/framework-res_o/res/values/public.xml:6141: error: Public symbol array/config_autoBrightnessButtonBacklightValues declared here is not defined.
W: /home/ibotpeaches/Downloads/Apktool/Bug1453/framework-res_o/res/values/public.xml:6143: error: Public symbol array/config_autoBrightnessLcdBacklightValues declared here is not defined.
W: /home/ibotpeaches/Downloads/Apktool/Bug1453/framework-res_o/res/values/public.xml:6144: error: Public symbol array/config_autoBrightnessLevels declared here is not defined.
...

So I looked for the that first resource.

➜  framework-res_o grep -r -i 'config_autoBrightnessButtonBacklightValues' *
res/values/public.xml:    <public type="array" name="config_autoBrightnessButtonBacklightValues" id="0x01070008" />
res/values/arrays.xml:    <style name="config_autoBrightnessButtonBacklightValues" />

Why is a style in the arrays.xml file?

<?xml version="1.0" encoding="utf-8"?>
<resources>
    <style name="config_gpsParameters" />
    <style name="emailAddressTypes" />
    <style name="imProtocols" />
    <style name="organizationTypes" />
    <style name="phoneTypes" />
    <style name="postalAddressTypes" />
    <array name="config_keySystemUuidMapping" />
    <array name="carrier_properties" />
    <array name="common_nicknames" />
    <style name="config_autoBrightnessButtonBacklightValues" />

Doh. No idea, next thing to patch. For now, find/replace all style with array and

➜  Bug1453 apktool b framework-res_o
I: Using Apktool 2.2.3-390eca-SNAPSHOT
W: Could not find sources
I: Checking whether resources has changed...
I: Building resources...
I: Building apk file...
I: Copying unknown files/dir...
➜  Bug1453 

I don't have an Android O supported device to actually test this yet, but it builds which is progress.

@argraur
Copy link
Author

argraur commented May 24, 2017

Hey I have Nexus 5X, I can test that. PM me on Telegram

iBotPeaches added a commit that referenced this issue May 24, 2017
 - #1453
 - temporarily cast unknown enum (0) to ResArray
@iBotPeaches
Copy link
Owner

With the commit above. This application can now be decoded and rebuilt with no work-around. I will confirm this functionality on both my Windows/Mac computers soon and if those pass with my recent aapt binary changes. This will be closed and marked as fixed.

We might have to re-explore when Android O really drops, but O Preview is working for this specific application.

@argraur
Copy link
Author

argraur commented May 24, 2017

Add smali and baksmali v2.2.1 because that version introduce initial support for .vdex files. #1515

@argraur
Copy link
Author

argraur commented May 24, 2017

@iBotPeaches after framework-res.apk editing ( changing 2 bools from false to true ) device doesn't boot.

@iBotPeaches
Copy link
Owner

I will need a device capable of running O to debug that further. The time it takes for me to get requested information and react upon it is simply too long when doing remote work over a problem. Decode only support will have to do for now. Closing this in favor for more specific new issues.

@alice-mkh
Copy link

What about emulator?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants