Android device/emulator hardly connects #636

faisalmohd83 · 2020-09-24T20:11:21Z

Proxyman version? 2.7.0

macOS Version? Catalina 10.15.6

Steps to reproduce

Connect the Android device/ Start an emulator
Follow the Proxyman config steps (as guided in the Proxyman Mac App)
Device/Emulator is able to connect sometime (if lucky)

It connects 'sometimes'. If connected, the device will complain 'No Connection' in some time.
More: Found co

Expected behavior

Connect the device all the time.

yauheniprakapenka · 2020-09-24T20:36:47Z

I have a proxy configured on my android device and it works through proxyman 2.7.0. I am having problems where requests from my test applications are not showing up in the proxy. The browser requests is displayed, but my applications requests are not displayed. I have no idea how to reproduce this. I just use it, and at some point they stop being displayed. Only complete removal of the proxymal name can help. And then requests disappear again. Unfortunately, I do not know how to suggest the steps of the playback. Therefore, I am not creating an issue.

NghiaTranUIT · 2020-09-24T23:19:42Z

I have received few reports about Android emulator connection too, but really hard to reproduce.

I suppose that it’s from the Android emulator itself, because I always work fine with iOS simulator.

Can you guys try to delete entire android and create again? And if it happens with Android real device?

Meanwhile I will spend today to try to reproduce it 👍

NghiaTranUIT · 2020-09-24T23:21:06Z

If you guys don’t mind, please try to use Android Emulator on other proxy tools like Charles, to see if it still persists or not 🧐

NghiaTranUIT · 2020-09-25T07:01:46Z

@faisalmohd83 @yauheniprakapenka just wondering: What is the Android Emulator you're using? I use Pixel 3 Android Emulator API 29 from Android Studio, but I'm not able to reproduce the bug 🤔

yauheniprakapenka · 2020-09-25T07:05:06Z

I am using only Android Device Xiaomi Redmi 9a android 10. This Saturday I will try to find how to reproduce the problem.

faisalmohd83 · 2020-09-25T08:00:49Z

@NghiaTranUIT I doubt it's an issue with device/emulator.

I and team use a combination of emulators and physical devices with different OS versions for development/testing. In Android, works well once connected but disconnects in a while and it'll start complaining as connection lost. The only way to reconnect would be forgetting the connected WiFi and redo all the configuration(though not guaranteed). Whereas in iOS, the Proxyman works super smooth and our dev/test team uses it extensively.

--
The combination of emulators(all are GooglePlay images) used are followed and tried with many of it. Regardless of the device/emulator configuration, the behaviour is the same.

Eagerly looking forward to solving it, Would be happy to provide more info.

NghiaTranUIT · 2020-09-25T08:20:06Z

Thanks for the detailed report @faisalmohd83. I would summarize that we have two different problems:

Lost connection after setting HTTP Proxy in Wifi Setting from your Android devices: It's common problems and it could be fixed by:

Your macOS's IP might change after a while => Please verify the IP in Wifi Setting (from Android Devices) that matches with the current macOS IP.
Firewall can block all incoming connections: Please turn off the Firewall or allow Proxyman in the whitelist.

Read more at https://docs.proxyman.io/troubleshooting/my-ios-devices-couldnt-connect-to-proxyman-via-proxy

You're able to connect to Proxyman, able to see HTTPS Traffic, but it disconnects after a period of time. => It's a problem that we're investigating since I'm unable to reproduce it and it works fine with iOS devices 🤔

To investigate, I might change some configuration when starting a Proxy Server (Powered by SwiftNIO)
Currently, I'm using:

ChannelOptions.backlog
ChannelOptions.socket(SOL_SOCKET, SO_REUSEADDR)
ChannelOptions.socket(IPPROTO_TCP, TCP_NODELAY)
ChannelOptions.maxMessagesPerRead
ChannelOptions.allowRemoteHalfClosure
ChannelOptions.recvAllocator
ChannelOptions.connectTimeout

NghiaTranUIT · 2020-09-25T09:14:48Z

Hey @faisalmohd83 , please help me to ON/OFF the option Use Android Studio HTTP Proxy Setting.

I assume that the Android Emulator has a bug. Sometime it follow the Android Studio HTTP Proxy or HTTP Proxy in Setting app, which might cause the connection lost.

I tried to set manual Proxy in Android Studio HTTP Proxy to Proxyman, but it always show the IP instead of the domain. It happens on Charles and Fiddler too

NghiaTranUIT · 2020-09-25T09:16:20Z

Meanwhile, I notices that from Android API 30, install Proxyman CA Certificate requires extra steps to install it properly.

For instance, I have manually install in Security Tab in Setting app

I will update the document 👍

faisalmohd83 · 2020-09-25T10:06:22Z

@NghiaTranUIT

Certificate downloading was an issue but found you answered it another thread and done it in Android 30 emulator.
Can you please also share the Mac DNS/Proxy/Firewall settings apart from emulator settings?

NghiaTranUIT · 2020-09-25T11:18:36Z

Sure, I will update 👍 if you don’t mind, please help me test out with Android Studio HTTP Proxy setting in the previous comment.

faisalmohd83 · 2020-09-26T11:10:24Z

@NghiaTranUIT

I tried with the above suggestions, had no luck!
In fact, all configuration was already in place except the

Emulator proxy >> None
Turning OFF the Mac firewall

Resulting behaviour:
The device connected but the internet connection lost after a while.

NghiaTranUIT · 2020-09-26T11:13:49Z

Hey @faisalmohd83 if you don’t mind, please help me test the Android issue with Charles Proxy app (it has a free version). If it doesn’t happen on Charles, it means something wrong with Proxyman and Android

NghiaTranUIT · 2020-09-27T08:32:02Z

@faisalmohd83 @yauheniprakapenka it seems that Android Emulator has a problem. I search for "Android Emulator lost connection after a while" and there are tons of StackOverflow about it.

They often suggest setting DNS as 8.8.8.8 as a solution.

Please help me to check it out 👍

yauheniprakapenka · 2020-09-27T08:43:02Z

I will add a new address 8.8.8.8 to macbook`s Wi-Fi settings and watching for correct works for a few days.

yauheniprakapenka · 2020-09-29T13:09:35Z

Good day! :)

At the moment, with such settings, it works fine for me on physical device. I continue to watch.

NghiaTranUIT · 2020-09-29T13:10:51Z

Hopefully, it works 🤣

NghiaTranUIT · 2020-09-29T14:17:59Z

I've updated the Troubleshooting Page for Android: https://docs.proxyman.io/troubleshooting/my-ios-devices-couldnt-connect-to-proxyman-via-proxy#android-devices-and-android-emulator 👍

ghowardqa · 2020-09-29T16:53:27Z

Evening guys

Hope you're well.

@faisalmohd83 has asked me to weigh in on the issue.

Massive fan of proxyman, I was a user of Charles before finding out about proxyman and haven't looked back!

I've always had issues with Android and any HTTP Proxy tool ever since Android has tightened their network security. So following the usual setup of adding the network_security_config.xml to the project and referencing it in the Android Manifest.xml works fine. Next, linking up the device or emulator to proxyman and installing the Root certificate and device certificate again goes smoothly.

Now, this where things start to happen, firstly attempt SSL handshake and then obviously making sure the endpoint is enabled for SSL HTTPS Response.

Next, attempted to forget the network connection and rejoin it network and attempt again. Now this sometimes works or as @faisalmohd83 pointed out a system alert is displayed of 'No Internet Connection' or within the device/emulator it'll say its connected but no internet connection.

So the physical device I'm using is a Samsung S10 running 10.0 (OneUI 2.)1 and the emulator is the standard AVD Google Pixel running OS 10.0

I'll also attempt the DNS server solution and report back. :)

NghiaTranUIT · 2020-09-30T01:53:38Z

Glad to know that Proxyman have many massive fans 😄

I just discover that we can automatically install the self-signed certificate to Android Emulator by using ADB 😲 https://ivrodriguez.com/installing-self-signed-certificates-on-android/

I will attempt to add this automatic flow to Proxyman to make it easier 🙌

faisalmohd83 · 2020-10-02T09:51:59Z

@yauheniprakapenka Curious! Is it solved on your side?

yauheniprakapenka · 2020-10-02T09:56:30Z

@yauheniprakapenka Curious! Is it solved on your side?

On my physical device, the problem no longer happens from Monday to this Friday.

On my MacBook in Wi-Fi DNS added 8.8.8.8
I did not add DNS on the device, only proxy address and port

faisalmohd83 · 2020-10-11T23:23:30Z

@NghiaTranUIT @yauheniprakapenka thanks for inputs.

Updates based on trials:

Certainly, Adding DNS to MacOS WiFi Setting helps.
It seems I found the issue, The Proxman Certificate should be installed as VPN and apps not W-Fi. Would be worth mentioning in the documentation.

NghiaTranUIT · 2020-10-12T01:15:07Z

It seems that adding DNS 8.8.8.8 to macOS could resolve this problem 👍

I will add this information to the Android Troubleshooting page. Thank all for testing it 😄

faisalmohd83 · 2020-10-12T06:35:50Z

It seems that adding DNS 8.8.8.8 to macOS could resolve this problem 👍

I will add this information to the Android Troubleshooting page. Thank all for testing it 😄

Only DNS didn't solve my issue, Proxyman Certificate should be installed as VPN and apps. I was always installed as WiFi

NghiaTranUIT · 2020-10-14T13:44:47Z

I'm happy to show you guys that I'm working on the Script to automatically do:

Override HTTP Proxy
Install and Trust Proxyman Certificate

One-click and work automatically with all Android Emulators (Include Android 11)

https://twitter.com/proxyman_app/status/1316370751612579841

I will send a Beta build here when it's out 🙌

faisalmohd83 · 2020-10-15T10:23:49Z

I'm happy to show you guys that I'm working on the Script to automatically do:

Override HTTP Proxy

Install and Trust Proxyman Certificate

One-click and work automatically with all Android Emulators (Include Android 11)

https://twitter.com/proxyman_app/status/1316370751612579841

I will send a Beta build here when it's out 🙌

Hope this will improve consistency. Though Android/Emulator connects well, the connection loses after some time.

faisalmohd83 · 2020-10-23T13:07:07Z

@NghiaTranUIT Awesome 👍🏼
The new auto-provisioning for emulator works our of the box.

NghiaTranUIT · 2020-10-23T13:07:48Z

Glad to hear that it works for you 😄

castrojr913 · 2021-03-02T06:18:04Z

@yauheniprakapenka Curious! Is it solved on your side?

On my physical device, the problem no longer happens from Monday to this Friday.

On my MacBook in Wi-Fi DNS added 8.8.8.8

I did not add DNS on the device, only proxy address and port

@yauheniprakapenka How did u resolve it? I keep trying through Android emulator and It doesn't work. I've followed steps, installed certificate and it's marked as Trusted one in User tab. The same thing that u faced, browser requests are caught by ProxyMan but It doesn't catch my mobile app requests. UPDATE: I've checked emulator version out and it's a Pixel one. When I try overriding its settings, it displays an ADB error on step 5:
adb: error: stat failed when trying to push to /data/misc/user/0/cacerts-added/e93dbc37.0: Permission denied

NghiaTranUIT · 2021-03-02T06:21:10Z

@castrojr913 Are you using Native Android or React Native for Android apps?

If it's a native Android, which Network Library you're using 🤔
If it's a React Native, I'm aware that it needs extra configs to make it works.

If you can see the traffic from the Browser on Android Emulator, but not your app. It means that your network library doesn't respect the HTTP/HTTPS Proxy

NghiaTranUIT · 2021-03-02T06:23:07Z

@castrojr913 here is troubleshooting that I collect for Android Emulator: https://docs.proxyman.io/troubleshooting/my-ios-devices-couldnt-connect-to-proxyman-via-proxy#1-after-setting-http-proxy-in-wifi-setting-i-could-not-open-http-proxy-man-ssl

Or you can use Automatic Script for Android Emulator: https://docs.proxyman.io/debug-devices/android-device/automatic-script-for-android-emulator

castrojr913 · 2021-03-02T06:25:18Z

@castrojr913 Are you using Native Android or React Native for Android apps?

If it's a native Android, which Network Library you're using 🤔

If it's a React Native, I'm aware that it needs extra configs to make it works.

If you can see the traffic from the Browser on Android Emulator, but not your app. It means that your network library doesn't respect the HTTP/HTTPS Proxy

@NghiaTranUIT Xamarin Forms and I'm using Rider IDE. Network library is Refit.

NghiaTranUIT · 2021-03-02T06:27:55Z

Yay @castrojr913 , it's a third-party network library. We have to manually set the Proxy Server for Refit.

Proxyman is listening at 127.0.0.1 port 9090

You should try this solution: reactiveui/refit#123 (comment)

castrojr913 · 2021-03-03T03:17:17Z

@NghiaTranUIT Good point sir. Thank u so much. Regarding that suggested implementation, I added ServerCertificateCustomValidationCallback property in order to ignore SSL errors.

new HttpClientHandler
{
    UseProxy = true,
    Proxy = new WebProxy(ProxyUrl, false),
    ServerCertificateCustomValidationCallback =  (sender, cert, chain, sslPolicyErrors) => true // Ignore SSL errors
}

ProxyUrl is http://proxyman_ip:proxyman_port

NghiaTranUIT · 2021-03-03T03:18:24Z

Glad to know it works 😄 🌮

NghiaTranUIT added the bug Something isn't working label Sep 25, 2020

NghiaTranUIT self-assigned this Sep 25, 2020

faisalmohd83 closed this as completed Oct 23, 2020

Android device/emulator hardly connects #636

Android device/emulator hardly connects #636

Comments

faisalmohd83 commented Sep 24, 2020

Proxyman version? 2.7.0

macOS Version? Catalina 10.15.6

Steps to reproduce

Expected behavior

yauheniprakapenka commented Sep 24, 2020 • edited Loading

NghiaTranUIT commented Sep 24, 2020

NghiaTranUIT commented Sep 24, 2020

NghiaTranUIT commented Sep 25, 2020 • edited Loading

yauheniprakapenka commented Sep 25, 2020

faisalmohd83 commented Sep 25, 2020

NghiaTranUIT commented Sep 25, 2020 • edited Loading

NghiaTranUIT commented Sep 25, 2020

NghiaTranUIT commented Sep 25, 2020

faisalmohd83 commented Sep 25, 2020

NghiaTranUIT commented Sep 25, 2020

faisalmohd83 commented Sep 26, 2020 • edited Loading

NghiaTranUIT commented Sep 26, 2020

NghiaTranUIT commented Sep 27, 2020 • edited Loading

yauheniprakapenka commented Sep 27, 2020

yauheniprakapenka commented Sep 29, 2020

NghiaTranUIT commented Sep 29, 2020

NghiaTranUIT commented Sep 29, 2020

ghowardqa commented Sep 29, 2020

NghiaTranUIT commented Sep 30, 2020

faisalmohd83 commented Oct 2, 2020

yauheniprakapenka commented Oct 2, 2020

faisalmohd83 commented Oct 11, 2020 • edited Loading

NghiaTranUIT commented Oct 12, 2020

faisalmohd83 commented Oct 12, 2020

NghiaTranUIT commented Oct 14, 2020

faisalmohd83 commented Oct 15, 2020

faisalmohd83 commented Oct 23, 2020

NghiaTranUIT commented Oct 23, 2020

castrojr913 commented Mar 2, 2021 • edited Loading

NghiaTranUIT commented Mar 2, 2021 • edited Loading

NghiaTranUIT commented Mar 2, 2021

castrojr913 commented Mar 2, 2021

NghiaTranUIT commented Mar 2, 2021

castrojr913 commented Mar 3, 2021 • edited Loading

NghiaTranUIT commented Mar 3, 2021

yauheniprakapenka commented Sep 24, 2020 •

edited

Loading

NghiaTranUIT commented Sep 25, 2020 •

edited

Loading

NghiaTranUIT commented Sep 25, 2020 •

edited

Loading

faisalmohd83 commented Sep 26, 2020 •

edited

Loading

NghiaTranUIT commented Sep 27, 2020 •

edited

Loading

faisalmohd83 commented Oct 11, 2020 •

edited

Loading

castrojr913 commented Mar 2, 2021 •

edited

Loading

NghiaTranUIT commented Mar 2, 2021 •

edited

Loading

castrojr913 commented Mar 3, 2021 •

edited

Loading