New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Very slow performance of AndroidLib.decodeManifestWithResources #1489

Closed
prsekar opened this Issue Apr 28, 2017 · 7 comments

Comments

Projects
None yet
2 participants
@prsekar

prsekar commented Apr 28, 2017

Information

  1. Apktool Version (apktool -version) - 2.2.0 and above
  2. Operating System (Mac, Linux, Windows) - linux and mac
  3. APK From? (Playstore, ROM, Other) - any large apk 5mb or more its very evident

Stacktrace/Logcat

Include stacktrace here

Steps to Reproduce

  1. We call the java api AndroidLib.decodeManifestWithResources from our src code which is slowing down the unwrapping APK when compared with apktool 2.1.1.

Frameworks

If this APK is from an OEM ROM (Samsung, HTC, LG). Please attach framework files
(.apks that live in /system/framework or /system/priv-app)

APK

If this APK can be freely shared, please upload/attach a link to it.

Questions to ask before submission

  1. Have you tried apktool d, apktool b without changing anything? yes even cmdline is slow but its much more slower within jvm
  2. If you are trying to install a modified apk, did you resign it? no
  3. Are you using the latest apktool version? yes

@iBotPeaches iBotPeaches added the Feature label Apr 28, 2017

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Apr 28, 2017

Owner

This is because of the larger and larger framework files Apktool has to decode. Perhaps we can re-engineer the storage of these framework files into a format more suitable for apktool to process.

Owner

iBotPeaches commented Apr 28, 2017

This is because of the larger and larger framework files Apktool has to decode. Perhaps we can re-engineer the storage of these framework files into a format more suitable for apktool to process.

@prsekar

This comment has been minimized.

Show comment
Hide comment
@prsekar

prsekar Apr 28, 2017

We had been using 2.1.1 and it has been working like a charm for the same fat apk. Once we upgrade 2.2.0 or above we see the slow performance. Did something drastically change between those versions w.r.t the storage architecture?

prsekar commented Apr 28, 2017

We had been using 2.1.1 and it has been working like a charm for the same fat apk. Once we upgrade 2.2.0 or above we see the slow performance. Did something drastically change between those versions w.r.t the storage architecture?

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Apr 28, 2017

Owner

2.2.0 gained the Nougat framework which was huge in size in comparison to 2.1.1

Owner

iBotPeaches commented Apr 28, 2017

2.2.0 gained the Nougat framework which was huge in size in comparison to 2.1.1

@prsekar

This comment has been minimized.

Show comment
Hide comment
@prsekar

prsekar Apr 28, 2017

Back to the redesign of storage architecture, if you have plans, let me know if i could be of help.

Thanks for this oss library. Its very helpful.

prsekar commented Apr 28, 2017

Back to the redesign of storage architecture, if you have plans, let me know if i could be of help.

Thanks for this oss library. Its very helpful.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Apr 28, 2017

Owner

I've been thinking, we read the framework file (but store the framework file prior to decode) based on tags, etc. So decodes have to decode the framework file and then the internal framework of the apk in question. This is proving to take longer and longer. However, we are reading the file for one reason - the resources.

I might research the best flat file format for Java to parse and read, since at the end of the day we might be able to get away with a simple k/v store of resourceId -> resourceName (with some additional type data)

Owner

iBotPeaches commented Apr 28, 2017

I've been thinking, we read the framework file (but store the framework file prior to decode) based on tags, etc. So decodes have to decode the framework file and then the internal framework of the apk in question. This is proving to take longer and longer. However, we are reading the file for one reason - the resources.

I might research the best flat file format for Java to parse and read, since at the end of the day we might be able to get away with a simple k/v store of resourceId -> resourceName (with some additional type data)

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Jun 13, 2017

Owner

Moving all the speed/memory enhancement tickets together. The first report was #1396 so closing this one in favor of that one.

Owner

iBotPeaches commented Jun 13, 2017

Moving all the speed/memory enhancement tickets together. The first report was #1396 so closing this one in favor of that one.

@iBotPeaches iBotPeaches added Duplicate and removed Feature labels Jun 13, 2017

@iBotPeaches iBotPeaches added this to the 2.2.4 milestone Jun 28, 2017

@iBotPeaches iBotPeaches removed the Duplicate label Jun 28, 2017

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Jun 28, 2017

Owner

While closed. This change: b45054e has greatly helped in the speed execution.

apktool_224 d callrecorder.apk -f  14.76s user 2.68s system 276% cpu 6.307 total
apktool_223 d callrecorder.apk -f  25.46s user 3.64s system 195% cpu 14.853 total
Owner

iBotPeaches commented Jun 28, 2017

While closed. This change: b45054e has greatly helped in the speed execution.

apktool_224 d callrecorder.apk -f  14.76s user 2.68s system 276% cpu 6.307 total
apktool_223 d callrecorder.apk -f  25.46s user 3.64s system 195% cpu 14.853 total

alebcay added a commit to alebcay/opengapps that referenced this issue Jul 31, 2017

Update APKtool to 2.2.3
- Android O Final Dev Preview support (iBotPeaches/Apktool#1520)
- SnakeYAML 1.1.8 Android Support (iBotPeaches/Apktool#591)
- Fix issue with APKs taking longer than usual to parse resources (iBotPeaches/Apktool#1489)
- Fix issue with internal binaries not accessible in a Spring boot environment. (iBotPeaches/Apktool#1543)
- Fix issues with rebuilding applications originally built with aapt2. (iBotPeaches/Apktool#1520)
- Patch aapt to support the $ character in resource filenames. (iBotPeaches/Apktool#1532)
- Fix issue where apktool was holding locks onto files during execution. (iBotPeaches/Apktool#1561)
- Fix issue with APKs that last resource in pool is INVALID_TYPE_CONFIG. (iBotPeaches/Apktool#1534)
- Fix issue with APKs that are including malformed characters to break parser. (iBotPeaches/Apktool#1564)
- Only exit with 0 error code during version commands.
- Enforce license header on all source files.
- [Security] Prevent malicous directory traversal with unknown files.
- [Security] Prevent XXE vulnerability when given a malicious AndroidManifest.xml
- Upgrade to gradle 4.0.

alebcay added a commit to alebcay/opengapps that referenced this issue Jul 31, 2017

Update APKtool to 2.2.4
- Android O Final Dev Preview support (iBotPeaches/Apktool#1520)
- SnakeYAML 1.1.8 Android Support (iBotPeaches/Apktool#591)
- Fix issue with APKs taking longer than usual to parse resources (iBotPeaches/Apktool#1489)
- Fix issue with internal binaries not accessible in a Spring boot environment. (iBotPeaches/Apktool#1543)
- Fix issues with rebuilding applications originally built with aapt2. (iBotPeaches/Apktool#1520)
- Patch aapt to support the $ character in resource filenames. (iBotPeaches/Apktool#1532)
- Fix issue where apktool was holding locks onto files during execution. (iBotPeaches/Apktool#1561)
- Fix issue with APKs that last resource in pool is INVALID_TYPE_CONFIG. (iBotPeaches/Apktool#1534)
- Fix issue with APKs that are including malformed characters to break parser. (iBotPeaches/Apktool#1564)
- Only exit with 0 error code during version commands.
- Enforce license header on all source files.
- [Security] Prevent malicous directory traversal with unknown files.
- [Security] Prevent XXE vulnerability when given a malicious AndroidManifest.xml
- Upgrade to gradle 4.0.

mfonville added a commit to opengapps/opengapps that referenced this issue Jul 31, 2017

Update APKtool to 2.2.4
- Android O Final Dev Preview support (iBotPeaches/Apktool#1520)
- SnakeYAML 1.1.8 Android Support (iBotPeaches/Apktool#591)
- Fix issue with APKs taking longer than usual to parse resources (iBotPeaches/Apktool#1489)
- Fix issue with internal binaries not accessible in a Spring boot environment. (iBotPeaches/Apktool#1543)
- Fix issues with rebuilding applications originally built with aapt2. (iBotPeaches/Apktool#1520)
- Patch aapt to support the $ character in resource filenames. (iBotPeaches/Apktool#1532)
- Fix issue where apktool was holding locks onto files during execution. (iBotPeaches/Apktool#1561)
- Fix issue with APKs that last resource in pool is INVALID_TYPE_CONFIG. (iBotPeaches/Apktool#1534)
- Fix issue with APKs that are including malformed characters to break parser. (iBotPeaches/Apktool#1564)
- Only exit with 0 error code during version commands.
- Enforce license header on all source files.
- [Security] Prevent malicous directory traversal with unknown files.
- [Security] Prevent XXE vulnerability when given a malicious AndroidManifest.xml
- Upgrade to gradle 4.0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment