Cannot find the HTTP_GET_DATA keyword

[OrenNeko]

Thanks for your work! But there is some problem when I use the tool...
When I search and open a mini-app, it shows some hook information but except the HTTP_GET_DATA format infomation.
Is it related to the version of wechat? I change it from 8.0.3 to 7.0.20 but the log is the same.
I notice the code uses the url of "https://mp.weixin.qq.com/wxa-cgi/innersearch/subsearch".
Hope for your response!

Here is the log:
D/WechatSearchToken( 9021): hooked:private static [I com.tencent.mm.plugin.appbrand.jsapi.l.Sn(java.lang.String)
D/WechatSearchToken( 9021): hooked:public com.tencent.mm.plugin.appbrand.jsapi.l(com.tencent.mm.plugin.appbrand.jsapi.d)
D/WechatSearchToken( 9021): hooked:public final java.lang.String com.tencent.mm.plugin.appbrand.jsapi.l.invokeHandler(java.lang.String,java.lang.String,int)
D/WechatSearchToken( 9021): hooked:public final void com.tencent.mm.plugin.appbrand.jsapi.l.publishHandler(java.lang.String,java.lang.String,java.lang.String)
D/WechatSearchToken( 8711): com.tencent.mm:appbrand0---->>>:invokeHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:com.tencent.mm.plugin.appbrand.jsapi.l
D/WechatSearchToken( 9134): hooked:private static [I com.tencent.mm.plugin.appbrand.jsapi.l.Sn(java.lang.String)
D/WechatSearchToken( 9134): hooked:public com.tencent.mm.plugin.appbrand.jsapi.l(com.tencent.mm.plugin.appbrand.jsapi.d)
D/WechatSearchToken( 9134): hooked:public final java.lang.String com.tencent.mm.plugin.appbrand.jsapi.l.invokeHandler(java.lang.String,java.lang.String,int)
D/WechatSearchToken( 9134): hooked:public final void com.tencent.mm.plugin.appbrand.jsapi.l.publishHandler(java.lang.String,java.lang.String,java.lang.String)
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:publishHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:Sn
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:invokeHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:invokeHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:publishHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:Sn
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:publishHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:Sn
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:invokeHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:invokeHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:publishHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:Sn
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:invokeHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:invokeHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:publishHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:Sn
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:invokeHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:invokeHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:invokeHandler
D/WechatSearchToken( 9021): com.tencent.mm:appbrand1---->>>:invokeHandler
...

Hi @MagicAZ, thank you for your interest. The Crawl Mini-app Metadata part needs to run with WeChat 7.0.19 due to that WeChat changes the class names for each version. Could you try 7.0.19?
Please attach the full log and the APK file you used for further investigation if it doesn't work.

[OrenNeko]

Thanks for your response! I only tried 7.0.20 but not 7.0.19 and I will try it next (for that 7.0.19 is not accessible as easy as 7.0.20). Besides, I'm very interested in your research. Could I apply for the dataset? I wonder what steps should I take to apply it. Does it need an application? Looking forward to your reply.

…

------------------ 原始邮件 ------------------ 发件人: "OSUSecLab/MiniCrawler" ***@***.***>; 发送时间: 2021年6月30日(星期三) 凌晨4:02 ***@***.***>; ***@***.******@***.***>; 主题: Re: [OSUSecLab/MiniCrawler] Cannot find the HTTP_GET_DATA keyword (#1) Hi @MagicAZ, thank you for you interest. The Crawl Mini-app Metadata part needs to run with WeChat 7.0.19 due to that WeChat changes the classe names for each version. Could you try 7.0.19? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[OrenNeko]

I changed the version to 7.0.19. But there is another mistake:
When I build the XposedPlugin and install it on phone, it throws exception as this:
Launching 'app' on motorola AOSP on Shama.
$ adb shell am start -n "com.example.vsa.xposedutility/com.example.vsa.xposedutility.xposedUtility" -a android.intent.action.MAIN -c android.intent.category.LAUNCHER
Connected to process 2679 on device 'motorola-aosp_on_shama-127.0.0.1:62001'.
Capturing and displaying logcat messages from application. This behavior can be disabled in the "Logcat output" section of the "Debugger" settings page.
D/AndroidRuntime: Shutting down VM

--------- beginning of crash

E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.vsa.xposedutility, PID: 2679
java.lang.RuntimeException: Unable to instantiate activity ComponentInfo{com.example.vsa.xposedutility/com.example.vsa.xposedutility.xposedUtility}: java.lang.ClassNotFoundException: Didn't find class "com.example.vsa.xposedutility.xposedUtility" on path: DexPathList[[zip file "/data/app/com.example.vsa.xposedutility-1/base.apk"],nativeLibraryDirectories=[/vendor/lib, /system/lib]]
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2238)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2392)
at android.app.ActivityThread.access$800(ActivityThread.java:153)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1305)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:135)
at android.app.ActivityThread.main(ActivityThread.java:5307)
at java.lang.reflect.Method.invoke(Native Method)
at java.lang.reflect.Method.invoke(Method.java:372)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:903)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:698)
at de.robv.android.xposed.XposedBridge.main(XposedBridge.java:107)
Caused by: java.lang.ClassNotFoundException: Didn't find class "com.example.vsa.xposedutility.xposedUtility" on path: DexPathList[[zip file "/data/app/com.example.vsa.xposedutility-1/base.apk"],nativeLibraryDirectories=[/vendor/lib, /system/lib]]
at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
at java.lang.ClassLoader.loadClass(ClassLoader.java:511)
at java.lang.ClassLoader.loadClass(ClassLoader.java:469)
at android.app.Instrumentation.newActivity(Instrumentation.java:1066)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2228)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2392) 
at android.app.ActivityThread.access$800(ActivityThread.java:153) 
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1305) 
at android.os.Handler.dispatchMessage(Handler.java:102) 
at android.os.Looper.loop(Looper.java:135) 
at android.app.ActivityThread.main(ActivityThread.java:5307) 
at java.lang.reflect.Method.invoke(Native Method) 
at java.lang.reflect.Method.invoke(Method.java:372) 
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:903) 
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:698) 
at de.robv.android.xposed.XposedBridge.main(XposedBridge.java:107) 
Suppressed: java.lang.ClassNotFoundException: Didn't find class "com.example.vsa.xposedutility.xposedUtility" on path: DexPathList[[dex file "/data/dalvik-cache/xposed_XResourcesSuperClass.dex", dex file "/data/dalvik-cache/xposed_XTypedArraySuperClass.dex"],nativeLibraryDirectories=[/vendor/lib, /system/lib]]
at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
at java.lang.ClassLoader.loadClass(ClassLoader.java:511)
at java.lang.ClassLoader.loadClass(ClassLoader.java:504)
... 14 more
Suppressed: java.lang.ClassNotFoundException: com.example.vsa.xposedutility.xposedUtility
at java.lang.Class.classForName(Native Method)
at java.lang.BootClassLoader.findClass(ClassLoader.java:781)
at java.lang.BootClassLoader.loadClass(ClassLoader.java:841)
at java.lang.ClassLoader.loadClass(ClassLoader.java:504)
... 15 more
Caused by: java.lang.NoClassDefFoundError: Class not found using the boot class loader; no stack available
Process 2679 terminated.

[zyueinfosec]

Hi @MagicAZ, we can share the dataset. However, given the entire dataset is over 1 million and it may take time for us to create an online repository. Instead, we can share with you privately through Google drive or similar approaches. Please let us know (1) your email; (2) how many mini-apps you want (the more mini-apps you want, the more time it will take since we have to upload them first). Thank you for your interest!

[OrenNeko]

Thanks for your response.
My frequently-used e-mail is wy952066338@163.com and it's also my google account e-mail.
And could you share about 1% of your dataset? I think about 10,000 mini-apps are enough to me.
I learned a little bit about the decompilation method before, so the wxapkg package is suitable.
Thanks a lot!

[zyueinfosec]

Hi @MagicAZ,
We have sent you the dataset. Please download these mini-apps in one week. Thanks.

[EEEEEEcho]

Thanks for your work!
I'm doing some research on mini-program,could you share about 1% of your dataset?
Thank you very much!
My google account e-mail is 814466057@qq.com.

[zyueinfosec]

Thanks for your work! I'm doing some research on mini-program,could you share about 1% of your dataset? Thank you very much! My google account e-mail is 814466057@qq.com.

@EEEEEEcho We have discussed this with Tencent, and our dataset can only be shared with organizations for research purposes. Therefore, please let us know your name and your university. My email is zhang.12047@osu.edu. Thanks.

[JessicaM9797]

Thanks for your excellent work, which gave me a lot of inspiration!
I'm doing program analysis research on mini apps recently, could you share about 10,000 mini-apps with me? Thanks a lot!
I've sent an email to you through my school email address, stating my name and university, and my google account e-mail is jesswood1766@gmail.com, thanks again!

[OrenNeko]

我已收到您的来信，谢谢！I have received your letter, thanks!