Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support applications with multiple packages. #2514

Open
peterpt opened this issue Feb 25, 2021 · 24 comments
Open

Support applications with multiple packages. #2514

peterpt opened this issue Feb 25, 2021 · 24 comments
Labels

Comments

@peterpt
Copy link

peterpt commented Feb 25, 2021

Apktool Latest
OS is linux

Believe it or not , if i decompile an apk and try to recompile it again without any changes in code it fails .

I try using a twitter apk file .
I will leave here the link :
https://www.mediafire.com/file/82w10791m4xrjaf/Twitter.apk/file

You can decompile it , but you are unable to recompile it again even if no changes were made in the code .
Update , forgot to write the logs , here they are :

The log :

----------------------------------Decompiling log--------------------------
I: Using Apktool 2.4.1 on twitter.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: /root/.local/share/apktool/framework/1.apk
W: Could not decode attr value, using undecoded value instead: ns=android, name=resource, value=0x7e010000
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Baksmaling classes.dex...
I: Baksmaling classes2.dex...
I: Baksmaling classes3.dex...
I: Baksmaling classes4.dex...
I: Baksmaling classes5.dex...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
I: Copying META-INF/services directory
---------------------------------------------------------------------------
Recompiling log : 
I: Using Apktool 2.4.1
I: Checking whether sources has changed...
I: Smaling smali folder into classes.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes4 folder into classes4.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes5 folder into classes5.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes2 folder into classes2.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes3 folder into classes3.dex...
I: Checking whether resources has changed...
I: Building resources...
W: /opt/testapk/twitter/AndroidManifest.xml:831: error: Error: No resource type specified (at 'resource' with value '@2113994752').
W: 
brut.androlib.AndrolibException: brut.common.BrutException: could not exec (exit code = 1): [/tmp/brut_util_Jar_12790044432587942505.tmp, p, --forced-package-id, 127, --min-sdk-version, 21, --target-sdk-version, 30, --version-code, 28820000, --version-name, 8.82.0-release.00, --no-version-vectors, -F, /tmp/APKTOOL4379167829085885196.tmp, -e, /tmp/APKTOOL8538338056184325237.tmp, -0, arsc, -I, /root/.local/share/apktool/framework/1.apk, -S, /opt/testapk/twitter/res, -M, /opt/testapk/twitter/AndroidManifest.xml]|

@ayrox99
Copy link

ayrox99 commented Feb 26, 2021

Yeah got the same thing :
image

The apk I'im trying on : https://drive.google.com/file/d/17BtPR8A9OwkUrEvO82m8NP8lwUx3JqMX/view?usp=sharing

@peterpt
Copy link
Author

peterpt commented Feb 26, 2021

Best way to check if any changes can be made in an apk is , decompile and try to recompile , if recompile fails without any changes in the code of apk then apktool will never be able to recompile that apk no matter what you do .
I used in twitter.apk the 2.4.1 version but 2.5.0 was also giving the same issues , so the log is from 2.4.1 .

@iBotPeaches
Copy link
Owner

Believe it or not , if i decompile an apk and try to recompile it again without any changes in code it fails .

I would believe it, you are disassembling an application and trying to remake it. That isn't easy but any stretch of the imagination.


@archos-sa - Your error is not related.


If we take the bugged resource.

        <meta-data android:name="com.android.vending.splits" android:resource="@2113994752"/>

It looks bad. That isn't a valid reference.

W: /home/ibotpeaches/Downloads/Apktool/2514/Twitter/AndroidManifest.xml:831: error: '@2113994752' is incompatible with attribute resource (attr) reference.
W: error: failed processing manifest.

So lets check aapt.

      E: meta-data (line=0)
        A: android:name(0x01010003)="com.android.vending.splits" (Raw: "com.android.vending.splits")
        A: android:value(0x01010025)=@0x7e010000
      E: meta-data (line=0)
        A: android:name(0x01010003)="com.android.vending.derived.apk.id" (Raw: "com.android.vending.derived.apk.id")
        A: android:value(0x01010024)=(type 0x10)0x1

Hmm, its a reference to a resourceId. Lets look for it.

➜  2514 aapt d resources Twitter.apk| grep '0x7e010000'
      spec resource 0x7e010000 com.twitter.android.webrtcnative:xml/splits0: flags=0x00000000
        resource 0x7e010000 com.twitter.android.webrtcnative:xml/splits0: t=0x03 d=0x00000bbd (s=0x0008 r=0x00)
➜  2514 

We could not decode that resource which explains all of this.

W: Could not decode attr value, using undecoded value instead: ns=android, name=resource, value=0x7e010000

@iBotPeaches
Copy link
Owner

Okay, this is a bug with our handling of multiple package apks. We always assumed manifest resources were on 7F, but these are on 7E. However, we don't properly load enough data into the attribute parser for manifest at that point in the code to have a mapping of pkgId to name, so this will take some changes.

@iBotPeaches
Copy link
Owner

Yeah no way I see easily to resolve this without major research. Multiple packages in one resources.arsc. If we store them differently, they have the same resources. So we'd need to overhaul extraction code to isolate to one package.

I can fix the decoding error and properly resolve the reference, but then we get:

➜  2514 apktool b Twitter --use-aapt2
I: Using Apktool 2.5.1-d1c006-SNAPSHOT
I: Checking whether resources has changed...
I: Building resources...
W: /home/ibotpeaches/Downloads/Apktool/2514/Twitter/AndroidManifest.xml:831: error: resource com.twitter.android.webrtcnative:xml/splits0 not found.
W: error: failed processing manifest.

aapt2 can't build it since its a non-standard framework.

@iBotPeaches
Copy link
Owner

➜  2514 aapt d resources Twitter.apk | grep 'Package Group' -C 10
Package Groups (2)
Package Group 0 id=0x7f packageCount=1 name=com.twitter.android
  Package 0 id=0x7f name=com.twitter.android
    type 0 configCount=1 entryCount=117
      spec resource 0x7f010000 com.twitter.android:anim/abc_fade_in: flags=0x00000000
      spec resource 0x7f010001 com.twitter.android:anim/abc_fade_out: flags=0x00000000
      spec resource 0x7f010002 com.twitter.android:anim/abc_grow_fade_in_from_bottom: flags=0x00000000
      spec resource 0x7f010003 com.twitter.android:anim/abc_popup_enter: flags=0x00000000
      spec resource 0x7f010004 com.twitter.android:anim/abc_popup_exit: flags=0x00000000
      spec resource 0x7f010005 com.twitter.android:anim/abc_shrink_fade_out_from_bottom: flags=0x00000000
      spec resource 0x7f010006 com.twitter.android:anim/abc_slide_in_bottom: flags=0x00000000
      spec resource 0x7f010007 com.twitter.android:anim/abc_slide_in_top: flags=0x00000000
--
        resource 0x7f160025 com.twitter.android:xml/standalone_badge_gravity_top_start: t=0x03 d=0x00000bc1 (s=0x0008 r=0x00)
        resource 0x7f160026 com.twitter.android:xml/standalone_badge_offset: t=0x03 d=0x00000bc2 (s=0x0008 r=0x00)
        resource 0x7f160027 com.twitter.android:xml/sync_data_adapter: t=0x03 d=0x00000bc3 (s=0x0008 r=0x00)
        resource 0x7f160028 com.twitter.android:xml/trends_pref: t=0x03 d=0x00000bc4 (s=0x0008 r=0x00)
        resource 0x7f160029 com.twitter.android:xml/tweet_prefs: t=0x03 d=0x00000bc5 (s=0x0008 r=0x00)
        resource 0x7f16002a com.twitter.android:xml/twitter_external_file_provider_paths: t=0x03 d=0x00000bc6 (s=0x0008 r=0x00)
        resource 0x7f16002b com.twitter.android:xml/twitter_internal_file_provider_paths: t=0x03 d=0x00000bc7 (s=0x0008 r=0x00)
        resource 0x7f16002c com.twitter.android:xml/two_factor_auth_settings: t=0x03 d=0x00000bc8 (s=0x0008 r=0x00)
      config v22:
        resource 0x7f160021 com.twitter.android:xml/shortcuts: t=0x03 d=0x00000c0b (s=0x0008 r=0x00)
Package Group 1 id=0x7e packageCount=1 name=com.twitter.android.webrtcnative
  DynamicRefTable entryCount=1:
    0x7e -> com.twitter.android.webrtcnative

  Package 0 id=0x7e name=com.twitter.android.webrtcnative
    type 0 configCount=1 entryCount=1
      spec resource 0x7e010000 com.twitter.android.webrtcnative:xml/splits0: flags=0x00000000
      config (default):
        resource 0x7e010000 com.twitter.android.webrtcnative:xml/splits0: t=0x03 d=0x00000bbd (s=0x0008 r=0x00)

tldr - need to research dynamic refs more and how these function.

  1. Should we auto split them into different packages / thus resources.arsc during framework install?
  2. Automatically move resources to 7F?
  3. Handle automatic loading of non-framework id packages during recompilation

@iBotPeaches iBotPeaches added this to the v2.5.1 milestone Mar 4, 2021
@iBotPeaches
Copy link
Owner

This works perfectly for Twitter, given the following patch above.

  1. Use this patch.
  2. Install the application itself as a package (rename to 126 - 0x7e)
  3. Decode application
  4. Rebuild application

The way this works is that Apktool extracts needed frameworks (1, 126) as requirements but skips 127 (0x7F) as that is private. So when you try and build you get a framework missing exception. That is why installing the application itself, but lying around the pgkId works. So when aapt2 is looking for resources the -I include parameter includes the old application which has the 0x7e resources to find mapping.

This of course is highly dependent on having the old application which true app builders do not. I've been messing around with Instant apps like the discussion here - #2524 and trying to figure out how to make Android Studio be very verbose with all its commands to gradle/aapt2 so I can figure out the steps that it does to produce a binary from source.

With that knowledge, I could tweak Apktool to meet that behavior.

TLDR - Ongoing.

@Sunep-source
Copy link

Sunep-source commented Apr 4, 2021

I'm pretty sure the same issue occurs with the latest Snapchat apk:
https://www.file.io/download/l5kTneQcY7tV

Console output

java -jar apktool.jar d snapchat-11-21-1-37.apk -o snapps
I: Using Apktool 2.5.1-baf8bb-SNAPSHOT on snapchat-11-21-1-37.apk
I: Loading resource table...
I: Decoding Shared Library (com.snapchat.android.dummy_module_dynamic2), pkgId: 126
I: Decoding Shared Library (com.snapchat.android.native_specs_crypto_lib), pkgId: 123
I: Decoding Shared Library (com.snapchat.android.dummy_module_dynamic2), pkgId: 126
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: C:\Users\sunep\AppData\Local\apktool\framework\1.apk
W: Could not decode attr value, using undecoded value instead: ns=android, name=resource, value=0x7b010000
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Baksmaling classes.dex...
I: Baksmaling classes2.dex...
I: Baksmaling classes3.dex...
I: Baksmaling classes4.dex...
I: Baksmaling classes5.dex...
I: Baksmaling classes6.dex...
I: Baksmaling classes7.dex...
I: Baksmaling assets/secondary-dex/dummy_module_dynamic2.dex...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
I: Copying META-INF/services directory

java -jar apktool.jar b snapps
I: Using Apktool 2.5.1-baf8bb-SNAPSHOT
I: Checking whether sources has changed...
I: Smaling smali folder into classes.dex...
I: Checking whether sources has changed...
I: Smaling smali_assets folder into assets.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes2 folder into classes2.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes3 folder into classes3.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes4 folder into classes4.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes5 folder into classes5.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes6 folder into classes6.dex...
I: Checking whether sources has changed...
I: Smaling smali_classes7 folder into classes7.dex...
I: Checking whether resources has changed...
I: Building resources...
W: C:\Users\sunep\Downloads\snapps\AndroidManifest.xml:403: error: Error: No resource type specified (at 'resource' with value '@2063663104').
W:
brut.androlib.AndrolibException: brut.common.BrutException: could not exec (exit code = 1): [C:\Users\sunep\AppData\Local\Temp\brut_util_Jar_40083337941325849378736354795908562753.tmp, p, --forced-package-id, 127, --min-sdk-version, 19, --target-sdk-version, 29, --version-code, 2163, --version-name, 11.21.1.37, --no-version-vectors, -F, C:\Users\sunep\AppData\Local\Temp\APKTOOL2495946303200493142.tmp, -e, C:\Users\sunep\AppData\Local\Temp\APKTOOL2018924885346877021.tmp, -0, arsc, -I, C:\Users\sunep\AppData\Local\apktool\framework\1.apk, -S, C:\Users\sunep\Downloads\snapps\res, -M, C:\Users\sunep\Downloads\snapps\AndroidManifest.xml]

Apktool Version - 2.5.1-baf8bb-SNAPSHOT

Operating System - Windows

APK From? - Playstore

Following the patch you mentioned:
java -jar apktool.jar if snapchat-11-21-1-37.apk
*renaming 127.apk to 126.apk in framework folder
Gave the same output as above

@terrynguyen255
Copy link

Hi @iBotPeaches

I'm very excited about the patch and can not wait to try it, although it's a draft now.
But I'm getting some difficulty trying it. Could you please help with more details?

  1. Use this patch.

I cannot find any artifact file as apktool in the PR action. Could you show me ?

  1. Install the application itself as a package (rename to 126 - 0x7e)

How should I do this, and how will it help because it won't impact the APK file (which I will decode in the next step)?

  1. Decode application
  2. Rebuild application

I think I will be fine perform these steps. But if you have any tips that would be helpful

Thank you so much

@andykais
Copy link

@iBotPeaches whats the status of this bug? Is a fix for this possible on the 3.0.0 milestone horizon? Is there any additional research that can be done from people experiencing this bug? I am experiencing this as well, though unlike @Sunep-source, using the patch #2517 produced the same error as did v2.6.0.

error: '@<some_id>' is incompatible with attribute resource (attr) reference.
W: error: failed processing manifest.

I unfortunately cannot share the apk in question

@harshitshah4
Copy link

@iBotPeaches can we do something to help you on this issue ? Is it planned to be fixed in the next version or any upcoming versions ? Thanks.

@terrynguyen255
Copy link

Hi @iBotPeaches ,

Do you think it makes sense to prioritize this issue?
I think it's kinda critical because 2/5 APK files I tested encounter this issue.

@iBotPeaches
Copy link
Owner

@Nogias9x - I don't see this as something to prioritize, but you are welcome to continue from the research I have to come to a solution. I believe the Android 12/12L issues are more important at this time.

@ericsia
Copy link

ericsia commented Mar 21, 2022

@Nogias9x - I don't see this as something to prioritize, but you are welcome to continue from the research I have to come to a solution. I believe the Android 12/12L issues are more important at this time.

@iBotPeaches what do Support for only Android Manifest disassembling (no rebuild) means?
Is it possible apktool has option to decompile AndroidManifest.xml only and compile it again?
Maybe that reduce the chance of crashing?

@iBotPeaches
Copy link
Owner

@Nogias9x - I don't see this as something to prioritize, but you are welcome to continue from the research I have to come to a solution. I believe the Android 12/12L issues are more important at this time.

@iBotPeaches what do Support for only Android Manifest disassembling (no rebuild) means? Is it possible apktool has option to decompile AndroidManifest.xml and compile it again? Maybe that reduce the chance of crashing?

That is a feature I have planned for a future version to only disassemble the Manifest without needing to decode the entire resources.asrc file. It however, will not allow compilation. Unsure what that has to do with this ticket though.

@ericsia
Copy link

ericsia commented Mar 21, 2022

@Nogias9x - I don't see this as something to prioritize, but you are welcome to continue from the research I have to come to a solution. I believe the Android 12/12L issues are more important at this time.

@iBotPeaches what do Support for only Android Manifest disassembling (no rebuild) means? Is it possible apktool has option to decompile AndroidManifest.xml and compile it again? Maybe that reduce the chance of crashing?

That is a feature I have planned for a future version to only disassemble the Manifest without needing to decode the entire resources.asrc file. It however, will not allow compilation. Unsure what that has to do with this ticket though.

that would be so amazing when it's happening.
Because I was checking this apk and it asked to reinstall from playstore due to detected crashing.
Upon checking apktool log, it failed to decode smt, that is why being able to disassemble the Manifest only seem to be the solution

@sbg3k
Copy link

sbg3k commented May 4, 2022

@Nogias9x - I don't see this as something to prioritize, but you are welcome to continue from the research I have to come to a solution. I believe the Android 12/12L issues are more important at this time.

hi, any updates to this? I'm facing the same issues right now

@vellrya
Copy link

vellrya commented Jun 30, 2022

Same issue on popular app, @iBotPeaches, I would be very grateful if you could find a solution to this problem :)

@AlttiRi
Copy link

AlttiRi commented Jul 22, 2022

Just to pass the error replace

<meta-data android:name="com.android.vending.splits" android:resource="@12345"/>

with

<meta-data android:name="com.android.vending.splits" android:value="base"/>

It's possibly the similar this as here:
#2139 (comment)

@ij5

This comment was marked as off-topic.

@mfocuz
Copy link

mfocuz commented Nov 29, 2022

is it possible to move resources from splitted package into main one?

@mfocuz
Copy link

mfocuz commented Jan 4, 2023

in case you stuck with this - I fixed broken resources manually and it works. replaced null with required resource and moved appripriate file into base apk resources

@n0rb33r7
Copy link

in case you stuck with this - I fixed broken resources manually and it works. replaced null with required resource and moved appripriate file into base apk resources

Could you please give me an example what you've exactly done, please?

Currently when I try to recompile the untouched application, it fails with the following error:
apktooloutput.txt

I can see that it doesn't find the "accessibilityPaneTitle"

W: /asdf/base1/base/res/layout/material_timepicker_dialog.xml:2: error: No resource identifier found for attribute 'accessibilityPaneTitle' in package 'android'
W: 
brut.androlib.AndrolibException: brut.common.BrutException: could not exec (exit code = 1): [/tmp/brut_util_Jar_36301687030263586315688157136831404763.tmp, p, --forced-package-id, 127, --min-sdk-version, 24, --target-sdk-version, 32, --version-code, 1129, --version-name, 7.1.4, --no-version-vectors, -F, /tmp/APKTOOL625314186254050373.tmp, -e, /tmp/APKTOOL11616623642770033112.tmp, -0, arsc, -I, /root/.local/share/apktool/framework/1.apk, -S, /asdf/base1/base/res, -M, /asdf/base1/base/AndroidManifest.xml]

Attaching the AndroidManifests.xml and material_timepicker_dialog.xml file, base & split_en & split_xxhdpi files
packedapk.zip

Thank you for your help!

@kuri05
Copy link

kuri05 commented Sep 13, 2024

bump?
Is there any movement on this?

Where is this work-around-patch written about in the comments here? I have been unable to find it.

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

16 participants