New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

custom attributes casted to android wrongly #1157

Closed
DaRkLiFe1234 opened this Issue Feb 1, 2016 · 24 comments

Comments

Projects
None yet
8 participants
@DaRkLiFe1234

DaRkLiFe1234 commented Feb 1, 2016

Hi I am trying to build the apk from RE'd apk after making some changes to the AndroidManifest.xml file but i keep getting this error:

`I: Using Apktool 2.0.3
I: Copying a classes.dex file...
I: Checking whether resources has changed...
I: Building resources...
APKTool\a\res\xml\application_settings.xm
l:11: error: No resource identifier found for attribute 'min_value' in package '
android'

APKTool\a\res\xml\application_settings.xm
l:11: error: No resource identifier found for attribute 'max_value' in package '
android'

APKTool\a\res\xml\application_settings.xm
l:11: error: No resource identifier found for attribute 'default_value' in packa
ge 'android'

APKTool\a\res\xml\application_settings.xm
l:15: error: No resource identifier found for attribute 'min_value' in package '
android'

APKTool\a\res\xml\application_settings.xm
l:15: error: No resource identifier found for attribute 'max_value' in package '
android'

APKTool\a\res\xml\application_settings.xm
l:15: error: No resource identifier found for attribute 'default_value' in packa
ge 'android'

Exception in thread "main" brut.androlib.AndrolibException: brut.androlib.Androl
ibException: brut.common.BrutException: could not exec command: [C:\Users\\AppData\Local\Temp\brut_util_Jar_7507297341147460085.tmp, p, --forced-package
-id, 127, --version-code, 50201, --version-name, 5.2.1, -F, C:\Users\\Ap
pData\Local\Temp\APKTOOL3067853975455239040.tmp, -0, arsc, -0, arsc, -I, C:\User
s\\apktool\framework\1.apk, -S, APKTool\a\res, -M, APKTool\a\AndroidManifest.xml]
        at brut.androlib.Androlib.buildResourcesFull(Androlib.java:472)
        at brut.androlib.Androlib.buildResources(Androlib.java:410)
        at brut.androlib.Androlib.build(Androlib.java:298)
        at brut.androlib.Androlib.build(Androlib.java:268)
        at brut.apktool.Main.cmdBuild(Main.java:225)
        at brut.apktool.Main.main(Main.java:84)
Caused by: brut.androlib.AndrolibException: brut.common.BrutException: could not
 exec command: [C:\Users\\AppData\Local\Temp\brut_util_Jar_7507297341147
460085.tmp, p, --forced-package-id, 127, --version-code, 50201, --version-name,
5.2.1, -F, C:\Users\\AppData\Local\Temp\APKTOOL3067853975455239040.tmp,
-0, arsc, -0, arsc, -I, C:\Users\\apktool\framework\1.apk, -S, APKTool\a\res, -M, APKTool\\AndroidManifest.xml]
        at brut.androlib.res.AndrolibResources.aaptPackage(AndrolibResources.jav
a:425)
        at brut.androlib.Androlib.buildResourcesFull(Androlib.java:458)
        ... 5 more
Caused by: brut.common.BrutException: could not exec command: [C:\Users\
\AppData\Local\Temp\brut_util_Jar_7507297341147460085.tmp, p, --forced-package-i
d, 127, --version-code, 50201, --version-name, 5.2.1, -F, C:\Users\\AppD
ata\Local\Temp\APKTOOL3067853975455239040.tmp, -0, arsc, -0, arsc, -I, C:\Users\
\apktool\framework\1.apk, -S, D:\Personal\software\android rev tools\APK
Tool\a\res, -M, APKTool\a\AndroidManifest
.xml]
        at brut.util.OS.exec(OS.java:89)
        at brut.androlib.res.AndrolibResources.aaptPackage(AndrolibResources.jav
a:419)
        ... 6 `more

Let me know if you require access to the file also, hope you can help resolve.

PS: Link to APK: http://www30.zippyshare.com/v/GwEWjfx8/file.html

Let me know if you need more information

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Feb 4, 2016

Owner

Take the bad resources for example res/xml/application_settings.xml. We get this line.

<com.xtralogic.android.rdpclient.LogarithmicSliderPreference android:max="100" android:title="@string/pref_pointer_sensitivity_title" android:key="pref_pointer_sensitivity" android:summary="@string/pref_pointer_sensitivity_summary" android:min_value="0.2" android:max_value="5.0" android:default_value="1.0" />

Then we see the 3 bad attributes

  • android:max_value="5.0"
  • android:min_value="0.2"
  • android:default_value="1.0"

If we look on AOSP resources. These values don't exist.

➜  values git:(c02a10d) ✗ grep -r -i 'android:min_value'
➜  values git:(c02a10d) ✗ grep -r -i 'min_value' 

So these actually aren't Android resources. So lets dump the resources using aapt d xmltree foo.apk res/values/application_settings.xml

and find the lines above

        E: com.xtralogic.android.rdpclient.LogarithmicSliderPreference (line=34)
          A: :(0x01010136)=(type 0x10)0x64
          A: :(0x010101e1)=@0x7f0a00be
          A: :(0x010101e8)="pref_pointer_sensitivity" (Raw: "pref_pointer_sensitivity")
          A: :(0x010101e9)=@0x7f0a00bf
          A: :(0x7f010000)=(type 0x4)0x3e4ccccd
          A: :(0x7f010001)=(type 0x4)0x40a00000
          A: :(0x7f010002)=(type 0x4)0x3f800000

So we can see the last 3 attributes that we are interested in have a 0x7f packageId. This is private or internal resources. So lets see if the local attribute file describes these.

➜  foo cat res/values/attrs.xml  
<?xml version="1.0" encoding="utf-8"?>
<resources>
    <attr name="min_value" format="float" />
    <attr name="max_value" format="float" />
    <attr name="default_value" format="float" />
</resources>

Wow. That seals the problem. Internal attributes are being decoded as framework attributes. So we peek at the namespaces in the file.

<?xml version="1.0" encoding="utf-8"?>
<PreferenceScreen
  xmlns:android="http://schemas.android.com/apk/res/android"
  xmlns:app="http://schemas.android.com/apk/res/com.xtralogic.android.rdpclient">

So instead of android we should use app. Then re-trigger the build.

➜  Bug1157  apktool b Remote-Desktop-v5-2-1
I: Using Apktool 2.1.0-8525be-SNAPSHOT
I: Checking whether sources has changed...
I: Checking whether resources has changed...
I: Building resources...
I: Copying libs... (/lib)
I: Building apk file...
I: Copying unknown files/dir...

So yes, confirmed bug. Will merge related bugs into this one, since the research on this is easier to follow.

Owner

iBotPeaches commented Feb 4, 2016

Take the bad resources for example res/xml/application_settings.xml. We get this line.

<com.xtralogic.android.rdpclient.LogarithmicSliderPreference android:max="100" android:title="@string/pref_pointer_sensitivity_title" android:key="pref_pointer_sensitivity" android:summary="@string/pref_pointer_sensitivity_summary" android:min_value="0.2" android:max_value="5.0" android:default_value="1.0" />

Then we see the 3 bad attributes

  • android:max_value="5.0"
  • android:min_value="0.2"
  • android:default_value="1.0"

If we look on AOSP resources. These values don't exist.

➜  values git:(c02a10d) ✗ grep -r -i 'android:min_value'
➜  values git:(c02a10d) ✗ grep -r -i 'min_value' 

So these actually aren't Android resources. So lets dump the resources using aapt d xmltree foo.apk res/values/application_settings.xml

and find the lines above

        E: com.xtralogic.android.rdpclient.LogarithmicSliderPreference (line=34)
          A: :(0x01010136)=(type 0x10)0x64
          A: :(0x010101e1)=@0x7f0a00be
          A: :(0x010101e8)="pref_pointer_sensitivity" (Raw: "pref_pointer_sensitivity")
          A: :(0x010101e9)=@0x7f0a00bf
          A: :(0x7f010000)=(type 0x4)0x3e4ccccd
          A: :(0x7f010001)=(type 0x4)0x40a00000
          A: :(0x7f010002)=(type 0x4)0x3f800000

So we can see the last 3 attributes that we are interested in have a 0x7f packageId. This is private or internal resources. So lets see if the local attribute file describes these.

➜  foo cat res/values/attrs.xml  
<?xml version="1.0" encoding="utf-8"?>
<resources>
    <attr name="min_value" format="float" />
    <attr name="max_value" format="float" />
    <attr name="default_value" format="float" />
</resources>

Wow. That seals the problem. Internal attributes are being decoded as framework attributes. So we peek at the namespaces in the file.

<?xml version="1.0" encoding="utf-8"?>
<PreferenceScreen
  xmlns:android="http://schemas.android.com/apk/res/android"
  xmlns:app="http://schemas.android.com/apk/res/com.xtralogic.android.rdpclient">

So instead of android we should use app. Then re-trigger the build.

➜  Bug1157  apktool b Remote-Desktop-v5-2-1
I: Using Apktool 2.1.0-8525be-SNAPSHOT
I: Checking whether sources has changed...
I: Checking whether resources has changed...
I: Building resources...
I: Copying libs... (/lib)
I: Building apk file...
I: Copying unknown files/dir...

So yes, confirmed bug. Will merge related bugs into this one, since the research on this is easier to follow.

@iBotPeaches iBotPeaches changed the title from No resource identifier found for attribute 'min_value' in package ' android' to custom attributes casted to android wrongly Feb 4, 2016

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Feb 4, 2016

Owner

The problem is in this function in AXmlResourceParser.xml, and the hacky fix I remember making.

    public String getAttributeNamespace(int index) {
        int offset = getAttributeOffset(index);
        int namespace = m_attributes[offset + ATTRIBUTE_IX_NAMESPACE_URI];
        if (namespace == -1) {
            return "";
        }

        System.out.println(namespace);
        System.out.println(m_strings.getString(namespace));

        // hacky: if the attribute names are proguarded, then so are the namespace
        // I don't know where these are located yet in the file, but it is always
        // this.android_ns in testing, so we will default to that for now.
        // @todo figure out where proguarded namespaces are stored.
        String value = m_strings.getString(namespace);

        if (value.length() == 0) {
            int offsetName = getAttributeOffset(index);
            int name = m_attributes[offsetName + ATTRIBUTE_IX_NAME];
            if (m_strings.getString(name).length() == 0) {
                value = android_ns;
            }
        }

        return value;
    }

Related commit here - e126a51

Basically the method for obtaining attribute namespace was failing and my early lack of knowledge about apktool and resources.arsc file, just defaulted to returning the stock namespace if one wasn't found. This method needs to be patched.

Merging related bugs into this one.

Owner

iBotPeaches commented Feb 4, 2016

The problem is in this function in AXmlResourceParser.xml, and the hacky fix I remember making.

    public String getAttributeNamespace(int index) {
        int offset = getAttributeOffset(index);
        int namespace = m_attributes[offset + ATTRIBUTE_IX_NAMESPACE_URI];
        if (namespace == -1) {
            return "";
        }

        System.out.println(namespace);
        System.out.println(m_strings.getString(namespace));

        // hacky: if the attribute names are proguarded, then so are the namespace
        // I don't know where these are located yet in the file, but it is always
        // this.android_ns in testing, so we will default to that for now.
        // @todo figure out where proguarded namespaces are stored.
        String value = m_strings.getString(namespace);

        if (value.length() == 0) {
            int offsetName = getAttributeOffset(index);
            int name = m_attributes[offsetName + ATTRIBUTE_IX_NAME];
            if (m_strings.getString(name).length() == 0) {
                value = android_ns;
            }
        }

        return value;
    }

Related commit here - e126a51

Basically the method for obtaining attribute namespace was failing and my early lack of knowledge about apktool and resources.arsc file, just defaulted to returning the stock namespace if one wasn't found. This method needs to be patched.

Merging related bugs into this one.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Feb 4, 2016

Owner

So I dumped the string table to check if the namespace I was looking for was in there. It was as shown below. I need to do some research and figure out the mapping because I see the data I want, but not sure how to reach it yet.

String decoded (index, offset, length): 38 1332 132 http://schemas.android.com/apk/res/com.xtralogic.android.rdpclient
Owner

iBotPeaches commented Feb 4, 2016

So I dumped the string table to check if the namespace I was looking for was in there. It was as shown below. I need to do some research and figure out the mapping because I see the data I want, but not sure how to reach it yet.

String decoded (index, offset, length): 38 1332 132 http://schemas.android.com/apk/res/com.xtralogic.android.rdpclient
@DaRkLiFe1234

This comment has been minimized.

Show comment
Hide comment
@DaRkLiFe1234

DaRkLiFe1234 Feb 4, 2016

Oh thanks, this is some help, i will try to use the workaround.

DaRkLiFe1234 commented Feb 4, 2016

Oh thanks, this is some help, i will try to use the workaround.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Feb 4, 2016

Owner

Notes to myself. I started dumping values to try and find the correct path to the namespace URI.

(index, offset, namespace, ns3, ns3, ns4) 0 0 17 -1 -1 -1 - CORRECT
(index, offset, namespace, ns3, ns3, ns4) 1 5 17 -1 -1 -1 - CORRECT
(index, offset, namespace, ns3, ns3, ns4) 2 10 17 -1 -1 -1 - CORRECT
(index, offset, namespace, ns3, ns3, ns4) 3 15 17 -1 -1 -1 - CORRECT
(index, offset, namespace, ns3, ns3, ns4) 4 20 17 -1 -1 -1  - WRONG
(index, offset, namespace, ns3, ns3, ns4) 5 25 17 -1 -1 -1 - WRONG
(index, offset, namespace, ns3, ns3, ns4) 6 30 17 -1 38 -1 - CORRECT
Writing out: max (android)
Writing out: title (android)
Writing out: key (android)
Writing out: summary (android)
Writing out: min_value (custom)
Writing out: max_value (custom)
Writing out: default_value (custom)

I got a few succesful hits (the id 38 shown for ns3), however this only worked for a few of them. I haven't yet made the connection between an incremental index and my NamespaceStack. The incremental nature of the index only allows the search to work when those numbers align.

This hit only occurs at the last attribute of an xml element. Since we get a 6 index * 5 = 30, then id 30 in the m_attributes equals 17. 17 * 2 + 2 = 38. The index of the secondary namespace we want. I can't find any pattern yet. Looking at the last 6 elements in the list above. The final 3 (id 4,5,6) should be the secondary namespace but only the last one hits it.

My only changing number is the index, which is just incremental from 0, offset * 5 to move into the correction position. This creates the pattern 0,0 | 1, 5 | 2, 10 | 3, 15 | 4, 20 | 5, 25 | 6, 30.

Owner

iBotPeaches commented Feb 4, 2016

Notes to myself. I started dumping values to try and find the correct path to the namespace URI.

(index, offset, namespace, ns3, ns3, ns4) 0 0 17 -1 -1 -1 - CORRECT
(index, offset, namespace, ns3, ns3, ns4) 1 5 17 -1 -1 -1 - CORRECT
(index, offset, namespace, ns3, ns3, ns4) 2 10 17 -1 -1 -1 - CORRECT
(index, offset, namespace, ns3, ns3, ns4) 3 15 17 -1 -1 -1 - CORRECT
(index, offset, namespace, ns3, ns3, ns4) 4 20 17 -1 -1 -1  - WRONG
(index, offset, namespace, ns3, ns3, ns4) 5 25 17 -1 -1 -1 - WRONG
(index, offset, namespace, ns3, ns3, ns4) 6 30 17 -1 38 -1 - CORRECT
Writing out: max (android)
Writing out: title (android)
Writing out: key (android)
Writing out: summary (android)
Writing out: min_value (custom)
Writing out: max_value (custom)
Writing out: default_value (custom)

I got a few succesful hits (the id 38 shown for ns3), however this only worked for a few of them. I haven't yet made the connection between an incremental index and my NamespaceStack. The incremental nature of the index only allows the search to work when those numbers align.

This hit only occurs at the last attribute of an xml element. Since we get a 6 index * 5 = 30, then id 30 in the m_attributes equals 17. 17 * 2 + 2 = 38. The index of the secondary namespace we want. I can't find any pattern yet. Looking at the last 6 elements in the list above. The final 3 (id 4,5,6) should be the secondary namespace but only the last one hits it.

My only changing number is the index, which is just incremental from 0, offset * 5 to move into the correction position. This creates the pattern 0,0 | 1, 5 | 2, 10 | 3, 15 | 4, 20 | 5, 25 | 6, 30.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Feb 8, 2016

Owner

Okay, weekend research update.

Take the offending apk dump of aapt d xmltree

        E: com.xtralogic.android.rdpclient.LogarithmicSliderPreference (line=34)
          A: :(0x01010136)=(type 0x10)0x64
          A: :(0x010101e1)=@0x7f0a00be
          A: :(0x010101e8)="pref_pointer_sensitivity" (Raw: "pref_pointer_sensitivity")
          A: :(0x010101e9)=@0x7f0a00bf
          A: :(0x7f010000)=(type 0x4)0x3e4ccccd
          A: :(0x7f010001)=(type 0x4)0x40a00000
          A: :(0x7f010002)=(type 0x4)0x3f800000

Then my custom application, which has the same attributes, but different values.

      E: com.ibotpeaches.issue1157.foo (line=3)
        A: android:max(0x01010136)=(type 0x10)0x64
        A: android:title(0x010101e1)="Issue" (Raw: "Issue")
        A: android:key(0x010101e8)="key" (Raw: "key")
        A: android:summary(0x010101e9)="Issue" (Raw: "Issue")
        A: app:bar(0x7f010000)=(type 0x4)0x4490a000
        A: app:bat(0x7f010001)=(type 0x4)0x4490accd
        A: app:baz(0x7f010002)=(type 0x4)0x4490a666

As you can see. Each XML attribute has a name and number associated with it. You can see android:max is 0x01010136. However, we have no name values to cross reference in the original application (Another Dexguard protection).

This was fixed for the value names here in #913, but not completely it seems. Since we find the resource ID in order to create bar, bat, baz. However, the reading process for apktool is namespace - name - value. So by the time we get the information we want in name, we have already read and set namespace, which is the default android namespace.

I understand the problem and how to fix it at least now. I just need to program it into apktool in a way that works.

Owner

iBotPeaches commented Feb 8, 2016

Okay, weekend research update.

Take the offending apk dump of aapt d xmltree

        E: com.xtralogic.android.rdpclient.LogarithmicSliderPreference (line=34)
          A: :(0x01010136)=(type 0x10)0x64
          A: :(0x010101e1)=@0x7f0a00be
          A: :(0x010101e8)="pref_pointer_sensitivity" (Raw: "pref_pointer_sensitivity")
          A: :(0x010101e9)=@0x7f0a00bf
          A: :(0x7f010000)=(type 0x4)0x3e4ccccd
          A: :(0x7f010001)=(type 0x4)0x40a00000
          A: :(0x7f010002)=(type 0x4)0x3f800000

Then my custom application, which has the same attributes, but different values.

      E: com.ibotpeaches.issue1157.foo (line=3)
        A: android:max(0x01010136)=(type 0x10)0x64
        A: android:title(0x010101e1)="Issue" (Raw: "Issue")
        A: android:key(0x010101e8)="key" (Raw: "key")
        A: android:summary(0x010101e9)="Issue" (Raw: "Issue")
        A: app:bar(0x7f010000)=(type 0x4)0x4490a000
        A: app:bat(0x7f010001)=(type 0x4)0x4490accd
        A: app:baz(0x7f010002)=(type 0x4)0x4490a666

As you can see. Each XML attribute has a name and number associated with it. You can see android:max is 0x01010136. However, we have no name values to cross reference in the original application (Another Dexguard protection).

This was fixed for the value names here in #913, but not completely it seems. Since we find the resource ID in order to create bar, bat, baz. However, the reading process for apktool is namespace - name - value. So by the time we get the information we want in name, we have already read and set namespace, which is the default android namespace.

I understand the problem and how to fix it at least now. I just need to program it into apktool in a way that works.

@DaRkLiFe1234

This comment has been minimized.

Show comment
Hide comment
@DaRkLiFe1234

DaRkLiFe1234 Feb 8, 2016

great progress iBotPeaches... hoping to see a new version out soon.
will keep testing it more to see if any other issues are there

DaRkLiFe1234 commented Feb 8, 2016

great progress iBotPeaches... hoping to see a new version out soon.
will keep testing it more to see if any other issues are there

@Jasi2169

This comment has been minimized.

Show comment
Hide comment
@Jasi2169

Jasi2169 Feb 21, 2016

Darklife buddy ;) i will fix it in my tool ,as previous apktool version doesnot show this manifest error you can compile with apktool previous version if only classes are decoded if resources are decoded then use this latest to compile it wont show error :)

Jasi2169 commented Feb 21, 2016

Darklife buddy ;) i will fix it in my tool ,as previous apktool version doesnot show this manifest error you can compile with apktool previous version if only classes are decoded if resources are decoded then use this latest to compile it wont show error :)

@DaRkLiFe1234

This comment has been minimized.

Show comment
Hide comment
@DaRkLiFe1234

DaRkLiFe1234 Feb 22, 2016

@JaspreetSinghXDA thanks bro, will check that

DaRkLiFe1234 commented Feb 22, 2016

@JaspreetSinghXDA thanks bro, will check that

@chetbox

This comment has been minimized.

Show comment
Hide comment
@chetbox

chetbox Mar 4, 2016

+1
I've just hit this problem too. Can send you the offending APK if it's any use.

chetbox commented Mar 4, 2016

+1
I've just hit this problem too. Can send you the offending APK if it's any use.

@chetbox

This comment has been minimized.

Show comment
Hide comment
@chetbox

chetbox Mar 4, 2016

Not sure if it's useful information, but I found if I manually fix all the namsepaces in the layout XML files then build a new APK, I can decompile and compile that new APK with no problems.

chetbox commented Mar 4, 2016

Not sure if it's useful information, but I found if I manually fix all the namsepaces in the layout XML files then build a new APK, I can decompile and compile that new APK with no problems.

@JiDW

This comment has been minimized.

Show comment
Hide comment
@JiDW

JiDW May 6, 2016

We have this bug too, didn't have it before (like 1 month ago it worked). We must have changed something in our compiler but I don't know what. Any luck finding the source of this bug ? This is critical for us :(

JiDW commented May 6, 2016

We have this bug too, didn't have it before (like 1 month ago it worked). We must have changed something in our compiler but I don't know what. Any luck finding the source of this bug ? This is critical for us :(

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches May 6, 2016

Owner

There are multiple posts in this thread that are paragraph worth of text that explain the problem and solutions in detail.

Owner

iBotPeaches commented May 6, 2016

There are multiple posts in this thread that are paragraph worth of text that explain the problem and solutions in detail.

@JiDW

This comment has been minimized.

Show comment
Hide comment
@JiDW

JiDW May 6, 2016

Well, maybe for people who knows things about APKs and how your tool works, but as an end user, I'm sorry to say that I don't understand enough to see what solutions are available right now :(

JiDW commented May 6, 2016

Well, maybe for people who knows things about APKs and how your tool works, but as an end user, I'm sorry to say that I don't understand enough to see what solutions are available right now :(

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches May 6, 2016

Owner

Then there is none, please wait for a fix.

Owner

iBotPeaches commented May 6, 2016

Then there is none, please wait for a fix.

@JiDW

This comment has been minimized.

Show comment
Hide comment
@JiDW

JiDW May 6, 2016

Sure ! Will do, thanks :)

JiDW commented May 6, 2016

Sure ! Will do, thanks :)

@solarkraft

This comment has been minimized.

Show comment
Hide comment
@solarkraft

solarkraft Apr 5, 2017

I'm guessing there is no fix so far? I hit this problem a lot and am also quite clueless. I'll try the workaround, but a real fix would be amazing.

solarkraft commented Apr 5, 2017

I'm guessing there is no fix so far? I hit this problem a lot and am also quite clueless. I'll try the workaround, but a real fix would be amazing.

@Nisthar

This comment has been minimized.

Show comment
Hide comment
@Nisthar

Nisthar Dec 6, 2017

@iBotPeaches Is the latest version(2.3.0) still have this issue?

Nisthar commented Dec 6, 2017

@iBotPeaches Is the latest version(2.3.0) still have this issue?

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Dec 6, 2017

Owner

Yes. As long as this bug is "Open", it is unfixed. Plenty of research above though for anyone to help out.

Owner

iBotPeaches commented Dec 6, 2017

Yes. As long as this bug is "Open", it is unfixed. Plenty of research above though for anyone to help out.

@Nisthar

This comment has been minimized.

Show comment
Hide comment
@Nisthar

Nisthar Dec 7, 2017

@iBotPeaches Anyway, many people using the old version does not face this issue i think. Can you tell me which version doesn't have this issue?

Is there a workaround for this issue?

Nisthar commented Dec 7, 2017

@iBotPeaches Anyway, many people using the old version does not face this issue i think. Can you tell me which version doesn't have this issue?

Is there a workaround for this issue?

@CunningLogic

This comment has been minimized.

Show comment
Hide comment
@CunningLogic

CunningLogic Dec 12, 2017

@Nisthar can you toss me an apk that does have this problem?

CunningLogic commented Dec 12, 2017

@Nisthar can you toss me an apk that does have this problem?

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Dec 13, 2017

Owner

Notes for myself and a small read for those interested. Dexguard has completely removed the values for both value names & namespace names. Android doesn't seem to care, perhaps because it only cares about resourceId. After an investigation scanning the string pool for the strings I am interested in.

m_strings.getString(index)
m_strings.getString(28) = "android"
m_strings.getString(30) = "app"
m_strings.getString(37) = "http://schemas.android.com/apk/res/android"
m_strings.getString(38) = "http://schemas.android.com/apk/res/com.xtralogic.android.rdpclient"

To understand these values, you need to understand how the reading of AXML goes. We will skip most and just get to the important part. To process namespace, name, value_string, value_data, and attribute length you need to do as follows.

For each attribute we start with the count. This is just incremental for each attribute, so it goes 1,2,3,4 etc. However, thats useless because each attribute contains 5 pieces of information. So we need to calculate the offset which is (count * 5) so our code works for every attribute in this tag.

So we will do a demo for this improperly decoded tag

<com.xtralogic.android.rdpclient.LogarithmicSliderPreference android:max="100" android:title="@string/pref_pointer_sensitivity_title" android:key="pref_pointer_sensitivity" android:summary="@string/pref_pointer_sensitivity_summary" android:min_value="0.2" android:max_value="5.0" android:default_value="1.0" />

This comment has the aapt representation if you are curious: #1157 (comment)

So for the first resource.

android:max="100" 
...
count = 0
offset = 5
namespace = 0

so (count * offset) + namespace = namespace_index
anything times 0 is 0. So we are looking for m_attributes[0]. Those are below.
m_attributes
 * 0 = 17
 1 = 5
 2 = -1
 3 = 16
 4 = 100
 * 5 = 17
 6 = 6
 7 = -1
 8 = 1
 9 = 2131361982
 * 10 = 17
 11 = 7
 12 = 55
 13 = 3
 14 = 55
 * 15 = 17
 16 = 8
 17 = -1
 18 = 1
 19 = 2131361983
 * 20 = 17
 21 = 14
 22 = -1
 23 = 4
 24 = 1045220557
 * 25 = 17
 26 = 15
 27 = -1
 28 = 4
 29 = 1084227584
 * 30 = 17
 31 = 16
 32 = -1
 33 = 4
 34 = 1065353216

As you can see, the namespace index is always 0, 5, 10 etc. So every single time we ask for the namespace at m_attributes[5/10/15/20/25/30] we get 17.

m_strings.getString(17) = null

Is that a bug in apktool? When we first discovered null values we realized we could no longer trust the value/namespace value in the code. Dexguard and other "minifiers" have removed those. So at this point we followed the resourceId in order to find the resource. For resources this was easy. Frameworks live at 0x01 and private resources are at 0x7f.

However, namespaces are a different story. So we failed and just hard coded it to return "android", hence this bug. If you have 3 different namespaces (Android, Library & Custom) all in a file Android actually compiles them down to 2 (Android and Custom) as Libraries become a part of the application on build time.

This creates problems if you have multiple libraries with same resource names, but thankfully thats not my problem to solve. Shout out to Adam L at AOSP for solving that.

So we can be sure that any non-android namespace at runtime belong to the app, but that means when you decode it we lose the ability to put "appcompat:var" and instead they would all resolve to like "app". Whatever the prefix of the custom application may be. So that does not match the original application, but will build and run (I think)

So looking back above at the resources. We can dump the index, offset and namespace index for each resource we hit.

0 , 0 , 17
1 , 5 , 17
2 , 10 , 17
3 , 15 , 17
4 , 20 , 17
5 , 25 , 17
6 , 30 , 17

So with those numbers, there is no way (in my brief investigation) to get to any value that ends up resolving to getString(28|30|37|38) for any of the prefixes/uri of the namespace.

The idea is we are doing

m_strings.getString(m_attributes[offset + valueType])
m_strings.getString(m_attributes[0 + 0])
m_strings.getString(17)
m_strings.getString(17) = null

So the offsets we are looking for to repeat again are 28|30|37|38 as those point to our namespace uri and prefix. However, take a look at the dump of m_attributes above. None of those values even point to 28|30|37|38

So hopefully you can see the problem. We will always get a null namespace with Dexguard. I am thinking to patch this, we look for a private resource id (0x7f or feature 0x80) and if so, use the 2nd namespace on stack if it exists. I'm not sure how prone to errors this is, but sounds like my only solution.

Enjoy the read.

Owner

iBotPeaches commented Dec 13, 2017

Notes for myself and a small read for those interested. Dexguard has completely removed the values for both value names & namespace names. Android doesn't seem to care, perhaps because it only cares about resourceId. After an investigation scanning the string pool for the strings I am interested in.

m_strings.getString(index)
m_strings.getString(28) = "android"
m_strings.getString(30) = "app"
m_strings.getString(37) = "http://schemas.android.com/apk/res/android"
m_strings.getString(38) = "http://schemas.android.com/apk/res/com.xtralogic.android.rdpclient"

To understand these values, you need to understand how the reading of AXML goes. We will skip most and just get to the important part. To process namespace, name, value_string, value_data, and attribute length you need to do as follows.

For each attribute we start with the count. This is just incremental for each attribute, so it goes 1,2,3,4 etc. However, thats useless because each attribute contains 5 pieces of information. So we need to calculate the offset which is (count * 5) so our code works for every attribute in this tag.

So we will do a demo for this improperly decoded tag

<com.xtralogic.android.rdpclient.LogarithmicSliderPreference android:max="100" android:title="@string/pref_pointer_sensitivity_title" android:key="pref_pointer_sensitivity" android:summary="@string/pref_pointer_sensitivity_summary" android:min_value="0.2" android:max_value="5.0" android:default_value="1.0" />

This comment has the aapt representation if you are curious: #1157 (comment)

So for the first resource.

android:max="100" 
...
count = 0
offset = 5
namespace = 0

so (count * offset) + namespace = namespace_index
anything times 0 is 0. So we are looking for m_attributes[0]. Those are below.
m_attributes
 * 0 = 17
 1 = 5
 2 = -1
 3 = 16
 4 = 100
 * 5 = 17
 6 = 6
 7 = -1
 8 = 1
 9 = 2131361982
 * 10 = 17
 11 = 7
 12 = 55
 13 = 3
 14 = 55
 * 15 = 17
 16 = 8
 17 = -1
 18 = 1
 19 = 2131361983
 * 20 = 17
 21 = 14
 22 = -1
 23 = 4
 24 = 1045220557
 * 25 = 17
 26 = 15
 27 = -1
 28 = 4
 29 = 1084227584
 * 30 = 17
 31 = 16
 32 = -1
 33 = 4
 34 = 1065353216

As you can see, the namespace index is always 0, 5, 10 etc. So every single time we ask for the namespace at m_attributes[5/10/15/20/25/30] we get 17.

m_strings.getString(17) = null

Is that a bug in apktool? When we first discovered null values we realized we could no longer trust the value/namespace value in the code. Dexguard and other "minifiers" have removed those. So at this point we followed the resourceId in order to find the resource. For resources this was easy. Frameworks live at 0x01 and private resources are at 0x7f.

However, namespaces are a different story. So we failed and just hard coded it to return "android", hence this bug. If you have 3 different namespaces (Android, Library & Custom) all in a file Android actually compiles them down to 2 (Android and Custom) as Libraries become a part of the application on build time.

This creates problems if you have multiple libraries with same resource names, but thankfully thats not my problem to solve. Shout out to Adam L at AOSP for solving that.

So we can be sure that any non-android namespace at runtime belong to the app, but that means when you decode it we lose the ability to put "appcompat:var" and instead they would all resolve to like "app". Whatever the prefix of the custom application may be. So that does not match the original application, but will build and run (I think)

So looking back above at the resources. We can dump the index, offset and namespace index for each resource we hit.

0 , 0 , 17
1 , 5 , 17
2 , 10 , 17
3 , 15 , 17
4 , 20 , 17
5 , 25 , 17
6 , 30 , 17

So with those numbers, there is no way (in my brief investigation) to get to any value that ends up resolving to getString(28|30|37|38) for any of the prefixes/uri of the namespace.

The idea is we are doing

m_strings.getString(m_attributes[offset + valueType])
m_strings.getString(m_attributes[0 + 0])
m_strings.getString(17)
m_strings.getString(17) = null

So the offsets we are looking for to repeat again are 28|30|37|38 as those point to our namespace uri and prefix. However, take a look at the dump of m_attributes above. None of those values even point to 28|30|37|38

So hopefully you can see the problem. We will always get a null namespace with Dexguard. I am thinking to patch this, we look for a private resource id (0x7f or feature 0x80) and if so, use the 2nd namespace on stack if it exists. I'm not sure how prone to errors this is, but sounds like my only solution.

Enjoy the read.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Dec 14, 2017

Owner

This should fix it - https://github.com/iBotPeaches/Apktool/pull/1693/files

You can give it a test and let me know, otherwise I'll test a few more applications and merge this in.

Owner

iBotPeaches commented Dec 14, 2017

This should fix it - https://github.com/iBotPeaches/Apktool/pull/1693/files

You can give it a test and let me know, otherwise I'll test a few more applications and merge this in.

@iBotPeaches

This comment has been minimized.

Show comment
Hide comment
@iBotPeaches

iBotPeaches Dec 15, 2017

Owner
➜  1157 apktool d Remote-Desktop-v5-2-1.apk -f
I: Using Apktool 2.3.1-ef9be8-SNAPSHOT on Remote-Desktop-v5-2-1.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: /home/ibotpeaches/.local/share/apktool/framework/1.apk
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Baksmaling classes.dex...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
➜  1157 apktool b Remote-Desktop-v5-2-1
I: Using Apktool 2.3.1-ef9be8-SNAPSHOT
I: Checking whether sources has changed...
I: Smaling smali folder into classes.dex...
I: Checking whether resources has changed...
I: Building resources...
I: Copying libs... (/lib)
I: Building apk file...
apktool I: Copying unknown files/dir...
➜  1157 apktool d Adobe\ Reader_10.4.2.apk 
I: Using Apktool 2.3.1-ef9be8-SNAPSHOT on Adobe Reader_10.4.2.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: /home/ibotpeaches/.local/share/apktool/framework/1.apk
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Baksmaling classes.dex...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
➜  1157 apktool b Adobe\ Reader_10.4.2
I: Using Apktool 2.3.1-ef9be8-SNAPSHOT
I: Checking whether sources has changed...
I: Smaling smali folder into classes.dex...
I: Checking whether resources has changed...
I: Building resources...
I: Copying libs... (/lib)
I: Building apk file...
I: Copying unknown files/dir...
➜  1157 
Owner

iBotPeaches commented Dec 15, 2017

➜  1157 apktool d Remote-Desktop-v5-2-1.apk -f
I: Using Apktool 2.3.1-ef9be8-SNAPSHOT on Remote-Desktop-v5-2-1.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: /home/ibotpeaches/.local/share/apktool/framework/1.apk
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Baksmaling classes.dex...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
➜  1157 apktool b Remote-Desktop-v5-2-1
I: Using Apktool 2.3.1-ef9be8-SNAPSHOT
I: Checking whether sources has changed...
I: Smaling smali folder into classes.dex...
I: Checking whether resources has changed...
I: Building resources...
I: Copying libs... (/lib)
I: Building apk file...
apktool I: Copying unknown files/dir...
➜  1157 apktool d Adobe\ Reader_10.4.2.apk 
I: Using Apktool 2.3.1-ef9be8-SNAPSHOT on Adobe Reader_10.4.2.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: /home/ibotpeaches/.local/share/apktool/framework/1.apk
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Baksmaling classes.dex...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
➜  1157 apktool b Adobe\ Reader_10.4.2
I: Using Apktool 2.3.1-ef9be8-SNAPSHOT
I: Checking whether sources has changed...
I: Smaling smali folder into classes.dex...
I: Checking whether resources has changed...
I: Building resources...
I: Copying libs... (/lib)
I: Building apk file...
I: Copying unknown files/dir...
➜  1157 
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment