Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Resource not found trying to recompile with no changes #2980

Closed
purpleman2k opened this issue Jan 17, 2023 · 12 comments · Fixed by #3067
Closed

[BUG] Resource not found trying to recompile with no changes #2980

purpleman2k opened this issue Jan 17, 2023 · 12 comments · Fixed by #3067
Labels
ARSC Bug

Comments

@purpleman2k
Copy link

purpleman2k commented Jan 17, 2023 •
edited

Information

  1. Apktool Version 2.7.0
  2. Operating System Windows
  3. APK From? Playstore

When trying to de/recompile gboard with no changes, error message indicates that some resources that are referenced in the xmls are not found (I assume 'declared'). Judging by the name of the referenced resource it appears that the name was obfuscated -- that's fine, I don't need the original name, but I see that the actual resource it's complaining about (with the obfuscated name) doesn't exist in the relevant XML where it needs to be declared.

For example, "id/0_resource_name_obfuscated" which should be declared in ids.xml just isn't there.

Stacktrace/Logcat

C:\Users\home\Android>apktool d -f gboard126.apk -o "C:\Users\home\Android\gboard126"
I: Using Apktool 2.7.0 on gboard126.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: C:\Users\home\AppData\Local\apktool\framework\1.apk
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Baksmaling classes.dex...
I: Baksmaling classes2.dex...
I: Baksmaling classes3.dex...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
I: Copying META-INF/services directory

C:\Users\home\Android>apktool b -f --use-aapt2 -d gboard126
I: Framework installed to: C:\Users\home\AppData\Local\apktool\framework\1.apk
I: Using Apktool 2.7.0
I: Smaling smali folder into classes.dex...
I: Smaling smali_classes2 folder into classes2.dex...
I: Smaling smali_classes3 folder into classes3.dex...
I: Building resources...
I: Using aapt2 - setting 'debuggable' attribute to 'true' in AndroidManifest.xml
W: C:\Users\home\Android\gboard126\res\values\attrs.xml:632: error: resource id/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:id/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\attrs.xml:1876: error: resource id/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:id/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\dimens.xml:43: error: resource dimen/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:dimen/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:926: error: resource font/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:font/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:926: error: resource font/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:font/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:1204: error: resource font/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:font/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:1204: error: resource font/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:font/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:2406: error: resource dimen/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:dimen/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:2407: error: resource dimen/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:dimen/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:2406: error: resource dimen/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:dimen/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:2407: error: resource dimen/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:dimen/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:4172: error: resource attr/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:attr/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:4173: error: resource attr/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:attr/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:4172: error: resource attr/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:attr/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:4173: error: resource attr/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:attr/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:4285: error: resource attr/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:attr/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:4286: error: resource attr/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:attr/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:4285: error: resource attr/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:attr/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:4286: error: resource attr/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:attr/0_resource_name_obfuscated) not found.
W: C:\Users\home\Android\gboard126\res\values\styles.xml:5428: error: style attribute 'attr/0_resource_name_obfuscated (aka com.google.android.inputmethod.latin:attr/0_resource_name_obfuscated)' not found.
brut.androlib.AndrolibException: brut.common.BrutException: could not exec (exit code = 1): [C:\Users\ITAYNA~1\AppData\Local\Temp\brut_util_Jar_39724244080885536784648859253930036022.tmp, link, -o, C:\Users\ITAYNA~1\AppData\Local\Temp\APKTOOL7747251418954349550.tmp, --package-id, 127, --min-sdk-version, 23, --target-sdk-version, 33, --version-code, 102451614, --version-name, 12.6.02.491625702-beta-arm64-v8a, --no-auto-version, --no-version-vectors, --no-version-transitions, --no-resource-deduping, --allow-reserved-package-id, -e, C:\Users\ITAYNA~1\AppData\Local\Temp\APKTOOL4362795320748915216.tmp, -0, arsc, -I, C:\Users\home\AppData\Local\apktool\framework\1.apk, --manifest, C:\Users\home\Android\gboard126\AndroidManifest.xml, C:\Users\home\Android\gboard126\build\resources.zip]
Unknown error.

Steps to Reproduce

  1. decompile using apktool
  2. recompile with no changes (I used aapt2)

Frameworks

N/A

APK

If this APK can be freely shared, please upload/attach a link to it.
https://m.apkpure.com/gboard-the-google-keyboard/com.google.android.inputmethod.latin/download/102451614-APK

Questions to ask before submission

  1. Have you tried apktool d, apktool b without changing anything? Yes
  2. If you are trying to install a modified apk, did you resign it? N/A
  3. Are you using the latest apktool version? Yes
@iBotPeaches
Copy link
Owner

Not sure what is happening here. All resources are named - 0_resource_name_obfuscated, so the first one gets the name, but the remaining are conflicting resource names and thus become a different iteration.

    <attr name="APKTOOL_DUPLICATE_attr_0x7f04027b">
        <flag name="0_resource_name_obfuscated" value="0x00000002" />
        <flag name="APKTOOL_DUPLICATE_id_0x7f0b0003" value="0x00001000" />
        <flag name="APKTOOL_DUPLICATE_id_0x7f0b0004" value="0x00000008" />
        <flag name="APKTOOL_DUPLICATE_id_0x7f0b0006" value="0x00010000" />
        <flag name="APKTOOL_DUPLICATE_id_0x7f0b0009" value="0x00000001" />
        <flag name="APKTOOL_DUPLICATE_id_0x7f0b000d" value="0x00000004" />
    </attr>

So in a regular application - we trust the named attributes. I wonder if we have some sort logic issue here with renaming of attributes in such a way that we mark the original attribute (0_resource_name_obfuscated) as DUPLICATE resources further on in processing leading to the mismatch.

That however, is not valid. There is only 1 reference for that ID

➜  2980 grep -r -i '0x7f0b0000' *
2980/res/values/public.xml:    <public type="id" name="0_resource_name_obfuscated" id="0x7f0b0000" />
➜  2980

However, we key the resources by the type, but look for them via the name specific to that type. So that might be our cause, since we might have the id type marked as original resource, but then as your issue suggests the attrs, dimens and styles would have the duplicate resource name.

@iBotPeaches
Copy link
Owner

Nah wasn't that. Will backlog this, not sure.

    <public type="anim" name="0_resource_name_obfuscated" id="0x7f010002" />
    <public type="animator" name="0_resource_name_obfuscated" id="0x7f020000" />
    <public type="array" name="0_resource_name_obfuscated" id="0x7f030000" />
    <public type="attr" name="0_resource_name_obfuscated" id="0x7f040000" />
    <public type="bool" name="0_resource_name_obfuscated" id="0x7f050000" />
    <public type="color" name="0_resource_name_obfuscated" id="0x7f060000" />
    <public type="dimen" name="0_resource_name_obfuscated" id="0x7f070000" />
    <public type="drawable" name="0_resource_name_obfuscated" id="0x7f080000" />
    <public type="font" name="0_resource_name_obfuscated" id="0x7f090000" />
    <public type="fraction" name="0_resource_name_obfuscated" id="0x7f0a0000" />
    <public type="id" name="0_resource_name_obfuscated" id="0x7f0b0000" />
    <public type="integer" name="0_resource_name_obfuscated" id="0x7f0c0000" />
    <public type="interpolator" name="0_resource_name_obfuscated" id="0x7f0d0000" />
    <public type="layout" name="0_resource_name_obfuscated" id="0x7f0e0000" />
    <public type="menu" name="0_resource_name_obfuscated" id="0x7f100002" />
    <public type="mipmap" name="0_resource_name_obfuscated" id="0x7f110001" />
    <public type="plurals" name="0_resource_name_obfuscated" id="0x7f120000" />
    <public type="raw" name="0_resource_name_obfuscated" id="0x7f130000" />
    <public type="string" name="0_resource_name_obfuscated" id="0x7f140001" />
    <public type="style" name="0_resource_name_obfuscated" id="0x7f150000" />
    <public type="xml" name="0_resource_name_obfuscated" id="0x7f170000" />

@purpleman2k
Copy link
Author

Thank you for your attention to this, your work is much appreciated!

@Tomoms
Copy link

Tomoms commented Feb 12, 2023

A very similar issue happens when trying to decompile and recompile (no changes) the latest version (34.4.16-21) of the Play Store (single apk, no bundle, downloaded from ApkMirror).

I: Building resources...
W: /mnt/data/android/play_store_34.4.16-21/res/values/attrs.xml:90: error: resource id/0_resource_name_obfuscated (aka com.android.vending:id/0_resource_name_obfuscated) not found.
W: /mnt/data/android/play_store_34.4.16-21/res/values/attrs.xml:2048: error: resource id/0_resource_name_obfuscated (aka com.android.vending:id/0_resource_name_obfuscated) not found.
W: /mnt/data/android/play_store_34.4.16-21/res/values/dimens.xml:41: error: resource dimen/0_resource_name_obfuscated (aka com.android.vending:dimen/0_resource_name_obfuscated) not found.
W: /mnt/data/android/play_store_34.4.16-21/res/values/styles.xml:1473: error: resource font/0_resource_name_obfuscated (aka com.android.vending:font/0_resource_name_obfuscated) not found.
W: /mnt/data/android/play_store_34.4.16-21/res/values/styles.xml:1473: error: resource font/0_resource_name_obfuscated (aka com.android.vending:font/0_resource_name_obfuscated) not found.
W: /mnt/data/android/play_store_34.4.16-21/res/values/styles.xml:1771: error: resource font/0_resource_name_obfuscated (aka com.android.vending:font/0_resource_name_obfuscated) not found.
W: /mnt/data/android/play_store_34.4.16-21/res/values/styles.xml:1771: error: resource font/0_resource_name_obfuscated (aka com.android.vending:font/0_resource_name_obfuscated) not found.
W: /mnt/data/android/play_store_34.4.16-21/res/values/styles.xml:3123: error: resource dimen/0_resource_name_obfuscated (aka com.android.vending:dimen/0_resource_name_obfuscated) not found.
W: /mnt/data/android/play_store_34.4.16-21/res/values/styles.xml:3124: error: resource dimen/0_resource_name_obfuscated (aka com.android.vending:dimen/0_resource_name_obfuscated) not found.
W: /mnt/data/android/play_store_34.4.16-21/res/values/styles.xml:3123: error: resource dimen/0_resource_name_obfuscated (aka com.android.vending:dimen/0_resource_name_obfuscated) not found.
W: /mnt/data/android/play_store_34.4.16-21/res/values/styles.xml:3124: error: resource dimen/0_resource_name_obfuscated (aka com.android.vending:dimen/0_resource_name_obfuscated) not found.
W: /mnt/data/android/play_store_34.4.16-21/res/values/styles.xml:4858: error: style attribute 'attr/0_resource_name_obfuscated (aka com.android.vending:attr/0_resource_name_obfuscated)' not found.
W: /mnt/data/android/play_store_34.4.16-21/res/values/styles.xml:4910: error: style attribute 'attr/0_resource_name_obfuscated (aka com.android.vending:attr/0_resource_name_obfuscated)' not found.
W: error: failed linking references.
brut.androlib.AndrolibException: brut.common.BrutException: could not exec (exit code = 1): [/tmp/brut_util_Jar_146934250591862605138441301672493754285.tmp, link, -o, /tmp/APKTOOL8074483388451500670.tmp, --package-id, 127, --min-sdk-version, 21, --target-sdk-version, 33, --version-code, 83441610, --version-name, 34.4.16-21 [0] [PR] 507539062, --no-auto-version, --no-version-vectors, --no-version-transitions, --no-resource-deduping, --allow-reserved-package-id, -e, /tmp/APKTOOL10209900601815719859.tmp, -0, arsc, -I, /home/tfonda/.local/share/apktool/framework/1.apk, --manifest, /mnt/data/android/play_store_34.4.16-21/AndroidManifest.xml, /mnt/data/android/play_store_34.4.16-21/build/resources.zip]

@reubenscratton
Copy link

The build tool aapt2 has an optimization flag named --collapse-resource-names. When used, all resource IDs are assigned the same name, 0_resource_name_obfuscated. Obviously this breaks Resources.getIdentifier() but if you can live without that, it's often an optimization worth having.

The workaround I have is to add the following line to constructor ofbrut.androlib.res.data.ResResSpec :

    `if ("0_resource_name_obfuscated".equals(name)) name = null;`

i.e. duplicate the existing logic for the case where the resource name is "(name removed)". This lets apktool generate unique dummy IDs instead of hundreds of clashes.

With this change I am now able to recompile the latest Google Chrome .apk.

@iBotPeaches
Copy link
Owner

thanks @reubenscratton for the tip on aapt2 feature. Will help to make a POC and tweak Apktool for this.

@reubenscratton
Copy link

No problem, glad to help. Thanks for creating such an awesomely useful tool.

FWIW Google's own apps will often be the first to use new tools changes that might break apktool. If you have an automated testing pipeline it might be worth scripting something that downloads the latest chrome/play/etc .apks and rebuilds them. Just a thought.

@purpleman2k
Copy link
Author

@reubenscratton thank you for finding this!

I just built apktool with the change you suggested in brut.androlib.res.data.ResResSpec and I can confirm gboard now succesfully roundtrips.

@iBotPeaches iBotPeaches mentioned this issue May 16, 2023
Merged
@iBotPeaches
Copy link
Owner

Thanks for tip. Started a PR for this change to support the various forms that aapt uses to strip attribute names.

#3067

@reubenscratton
Copy link

@reubenscratton thank you for finding this!

I just built apktool with the change you suggested in brut.androlib.res.data.ResResSpec and I can confirm gboard now succesfully roundtrips.

Hi @purpleman2k, does your modified gboard actually run? I've been working on rebuilding Google Chrome and am in a world of pain. For some reason XML resources aren't loading properly - the core framework seems unable to resolve any attribute IDs to values. This could be because this fix needs more work, or it could be a consequence of my changing the package name (unavoidable cos Chrome is a system app).

@purpleman2k
Copy link
Author

@reubenscratton
Sort of. At first it would crash because there's a certificate check in the code itself. Once I bypassed it in the smali code and rebuilt it would run, but wasn't fully functional. I could run the app and browse through the settings, but was unable to add any keyboard languages, if I try I just get a never-ending loading bar and it doesn't show any available languages.
In the logcat I found this error:

05-15 14:44:30.373 10597 21714 E GoogleApiManager: Failed to get service from broker. 
05-15 14:44:30.373 10597 21714 E GoogleApiManager: java.lang.SecurityException: GoogleCertificatesRslt: not allowed: pkg=com.testing.android.inputmethod.latin, sha256=[1e08a903aef9c3a721510b64ec764d01d3d094eb954161b62544ea8f187b5953], atk=false, ver=231613045.true (go/gsrlt)
05-15 14:44:30.373 10597 21714 E GoogleApiManager: 	at android.os.Parcel.createExceptionOrNull(Parcel.java:3040)
05-15 14:44:30.373 10597 21714 E GoogleApiManager: 	at android.os.Parcel.createException(Parcel.java:3024)
05-15 14:44:30.373 10597 21714 E GoogleApiManager: 	at android.os.Parcel.readException(Parcel.java:3007)
05-15 14:44:30.373 10597 21714 E GoogleApiManager: 	at android.os.Parcel.readException(Parcel.java:2949)
05-15 14:44:30.373 10597 21714 E GoogleApiManager: 	at fyp.r(PG:15)
05-15 14:44:30.373 10597 21714 E GoogleApiManager: 	at fic.run(PG:27)
05-15 14:44:30.373 10597 21714 E GoogleApiManager: 	at android.os.Handler.handleCallback(Handler.java:942)
05-15 14:44:30.373 10597 21714 E GoogleApiManager: 	at android.os.Handler.dispatchMessage(Handler.java:99)
05-15 14:44:30.373 10597 21714 E GoogleApiManager: 	at android.os.Looper.loopOnce(Looper.java:240)
05-15 14:44:30.373 10597 21714 E GoogleApiManager: 	at android.os.Looper.loop(Looper.java:351)
05-15 14:44:30.373 10597 21714 E GoogleApiManager: 	at android.os.HandlerThread.run(HandlerThread.java:67)

But I can't be sure if this error is the actual issue and if it is -- is it a result of something in apktool.

If anyone has insight, that'd be awesome.

Thanks!

@reubenscratton
Copy link

It doesn't look like an apktool issue so far. You must have changed the package ID (to 'com.testing.android.inputmethod.latin'), but presumably GBoard requires Google Play Services to run and the relevant system process doesn't recognise your app. You may need to set up API access in the Play Console... see https://developers.google.com/android/guides/client-auth for what needs doing. You'll have to take a guess as to what Google APIs GBoard needs.

iBotPeaches added a commit that referenced this issue May 20, 2023
@iBotPeaches
docs: update changelog for #2980
1d609fb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ARSC Bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: support 0_resource_name_obfuscated iBotPeaches/Apktool
4 participants
@iBotPeaches @reubenscratton @purpleman2k @Tomoms