Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Network calls don't show in Charles Proxy Debugger #20376

Open
nashfive opened this issue Aug 9, 2018 · 30 comments
Open

Network calls don't show in Charles Proxy Debugger #20376

nashfive opened this issue Aug 9, 2018 · 30 comments

Comments

@nashfive
Copy link

@nashfive nashfive commented Aug 9, 2018

Steps to Reproduce

  1. Install Charles Proxy and configure it.
  2. Configure the simulators to use Charles Proxy (follow the tutorials for iOS and Android)
  3. Test any native apps (browser or your own native apps that makes network calls) and see that all calls are logged in the Charles window
  4. Run the Flutter app and make a network call (http and/or https). Not a single trace of network calls :(

My HTTPS calls are actually fired and I get HTTP responses in my code and I can see the calls in my server logs.
Flutter (or Dart ?) doesn't seem to use the system proxy so I can't see any of my network calls' payloads.

I tried to launch the app in debug mode with http_proxy and https_proxy in the command line but this doesn't change anything :

export https_proxy=http://<charles-hostname>:<charles-port>;
export http_proxy=http://<charles-hostname>:<charles-port>;
flutter run
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel beta, v0.5.1, on Mac OS X 10.13.6 17G65, locale en-CH)
[✓] Android toolchain - develop for Android devices (Android SDK 28.0.1)
[✓] iOS toolchain - develop for iOS devices (Xcode 9.4.1)
[✓] Android Studio (version 3.1)
[!] VS Code (version 1.25.1)
[✓] Connected devices (1 available)
@zoechi zoechi added the engine label Aug 23, 2018
@LcTwisk
Copy link

@LcTwisk LcTwisk commented Oct 8, 2018

It's also not working on a physical device... :(

@ErikDohmen
Copy link

@ErikDohmen ErikDohmen commented Oct 19, 2018

Same here, both on physical device nor simulator/ emulator no way to get them to show up in charlesproxy while badly needed

@xster
Copy link
Member

@xster xster commented Nov 15, 2018

We're still trying to figure out how to best default this but for now, could you guys try using the .findProxy method on HttpClients? https://api.dartlang.org/stable/2.1.0/dart-io/HttpClient/findProxy.html

If you want to override how HttpClients are created, use https://api.dartlang.org/stable/2.1.0/dart-io/HttpOverrides-class.html

@dnfield
Copy link
Member

@dnfield dnfield commented Nov 16, 2018

This should probably be addressed upstream in dart:io.

/cc @zanderso

From conversation with him, our best bet seems to be to target updating

https://github.com/dart-lang/sdk/blob/master/runtime/bin/platform_macos.cc#L203 with information from https://developer.apple.com/documentation/cfnetwork/1426754-cfnetworkcopysystemproxysettings?language=objc.

@dnfield
Copy link
Member

@dnfield dnfield commented Nov 16, 2018

Does this work as-is on Android? If not we'd probably want a similar strategy there.

@xster
Copy link
Member

@xster xster commented Nov 16, 2018

Nope. Stackoverflow says https://stackoverflow.com/a/13616054/. Can we access Java APIs from the dart:io code?

@zanderso
Copy link
Member

@zanderso zanderso commented Nov 19, 2018

There is currently not a good way to make JNI calls from the dart:io implementation. We typically deal with this problem by adding hooks where the engine can install a custom callback. I will investigate that approach.

@AbnerChen028
Copy link

@AbnerChen028 AbnerChen028 commented Dec 11, 2018

I'm using dio to set proxy.

@zoechi zoechi added this to the Goals milestone Dec 11, 2018
@akindone
Copy link
Contributor

@akindone akindone commented Dec 17, 2018

I'm using dio to set proxy.

I am using dio too. iOS physical device can get both http and https api payload, but android physical device can only get http api payload. And I have install certification for device and charles can get https api for native app. but can't work for flutter app. Did you encounter this problem?

@akindone
Copy link
Contributor

@akindone akindone commented Dec 17, 2018

I'm using dio to set proxy.

I am using dio too. iOS physical device can get both http and https api payload, but android physical device can only get http api payload. And I have install certification for device and charles can get https api for native app. but can't work for flutter app. Did you encounter this problem?

I find a way for android devices. But I am not sure any side effect

bool isProxyChecked = true // a variable for debug
String proxy = '192.168.2.2:8888' // ip:port
final dio = Dio()
    ..onHttpClientCreate = (client) {
      client.badCertificateCallback =
          (X509Certificate cert, String host, int port) {
        return isProxyChecked && Platform.isAndroid;
      };
      client.findProxy = (url) {
        return isProxyChecked ? 'PROXY $proxy' : 'DIRECT';
      };
    }

@wangyongf
Copy link

@wangyongf wangyongf commented Dec 17, 2018

I'm using dio to set proxy.

I am using dio too. iOS physical device can get both http and https api payload, but android physical device can only get http api payload. And I have install certification for device and charles can get https api for native app. but can't work for flutter app. Did you encounter this problem?

I find a way for android devices. But I am not sure any side effect

bool isProxyChecked = true // a variable for debug
String proxy = '192.168.2.2:8888' // ip:port
final dio = Dio()
    ..onHttpClientCreate = (client) {
      client.badCertificateCallback =
          (X509Certificate cert, String host, int port) {
        return isProxyChecked && Platform.isAndroid;
      };
      client.findProxy = (url) {
        return isProxyChecked ? 'PROXY $proxy' : 'DIRECT';
      };
    }

It works for me, thanks

@dazza5000
Copy link

@dazza5000 dazza5000 commented Jan 19, 2019

I was able to get this working using HttpClient with a snippet from the following SO question:

https://stackoverflow.com/questions/50949938/flutter-app-ios-simulator-set-transparent-proxy

localhost can be the ip address for your computer like 192.168.1.77

HttpClient client = HttpClient();
client.findProxy = (uri) {
  return "PROXY localhost:3128;";
};

A more full implementation:

  static HttpClient getHttpClient() {
    HttpClient client = new HttpClient()
      ..findProxy = (uri) {
        return "PROXY 192.168.1.199:8888;";
      }
      ..badCertificateCallback =
      ((X509Certificate cert, String host, int port) => true);

    return client;
  }

@zoechi
Copy link
Contributor

@zoechi zoechi commented Feb 12, 2019

@dazza5000 and you can use HttpOverrides to make Flutter use the same HttpClient. See https://stackoverflow.com/questions/54321077/best-way-to-set-default-header-for-all-request-in-flutter-http-request for an example

@zoechi zoechi mentioned this issue Feb 22, 2019
@lijy91
Copy link

@lijy91 lijy91 commented Feb 24, 2019

@nashfive
I developed a flippersdk for flutter, you can use flipper to debug your network request, maybe it's easier.

https://github.com/blankapp/flutter_flipperkit

@stuartsoft
Copy link

@stuartsoft stuartsoft commented May 18, 2019

@OPY-bbt
Copy link

@OPY-bbt OPY-bbt commented Jul 9, 2019

(dio.httpClientAdapter as DefaultHttpClientAdapter).onHttpClientCreate = (client) {
    // config the http client
    client.findProxy = (uri) {
        //proxy all request to localhost:8888
        return "PROXY localhost:8888";
    };
    // you can also create a new HttpClient to dio
    // return new HttpClient();
};

@bstolinski
Copy link

@bstolinski bstolinski commented Aug 6, 2019

class MyHttpOverrides extends HttpOverrides {
  MyHttpOverrides(this._ip);

  final String _ip;

  @override
  HttpClient createHttpClient(SecurityContext context) {
    return super.createHttpClient(context)
      ..findProxy = (uri) {
        return "PROXY $_ip:8888;";
      }
      ..badCertificateCallback =
          (X509Certificate cert, String host, int port) => true;
  }
}

Android Simulator

HttpOverrides.global = MyHttpOverrides("10.0.2.2");

iPhone Simulator

HttpOverrides.global = MyHttpOverrides("localhost");

physical device (Android/iPhone)
Also add proxy server settings to your device in system settings.

HttpOverrides.global = MyHttpOverrides("192.168.0.100"); //ip address of your computer

This can also be used for simulators (skip adding proxy server settings in system settings).

@jamesdixon
Copy link

@jamesdixon jamesdixon commented Sep 5, 2019

For anyone else who ends up here, I compiled all of the solutions above (for the most part) into a blog post along with some other options to debug network requests in Flutter:

https://flutterigniter.com/debugging-network-requests/

@jmagman jmagman added this to Awaiting triage in Mobile - iOS engine review Feb 25, 2020
@jmagman jmagman added this to Awaiting triage in Mobile - Android engine review Feb 25, 2020
@jmagman jmagman removed this from Awaiting triage in Mobile - Android engine review Feb 25, 2020
@gaaclarke
Copy link
Contributor

@gaaclarke gaaclarke commented Apr 8, 2020

I suspect dart-lang/sdk#41376 will solve this problem on iOS. Using posix sockets is probably not respecting the OS's proxy settings like it is for VPN settings.

@gaaclarke gaaclarke moved this from Awaiting triage to Engineer reviewed in Mobile - iOS engine review Apr 8, 2020
@kf6gpe kf6gpe added the P4 label May 29, 2020
@x-raay
Copy link

@x-raay x-raay commented Jun 21, 2020

Before I forget, dart devtools also supports network traffic monitoring

Capture

@fryette
Copy link

@fryette fryette commented Jun 29, 2020

@saurabhrayakwar Can you see a body of requests?

@ashbin
Copy link

@ashbin ashbin commented Aug 6, 2020

(dio.httpClientAdapter as DefaultHttpClientAdapter).onHttpClientCreate = (client) {
    // config the http client
    client.findProxy = (uri) {
        //proxy all request to localhost:8888
        return "PROXY localhost:8888";
    };
    // you can also create a new HttpClient to dio
    // return new HttpClient();
};

If you are using android studio emulator use 10.0.2.2 and for genymotion 10.0.3.2 if localhost is not working.

@Hixie Hixie removed this from the None. milestone Aug 17, 2020
@locskot
Copy link

@locskot locskot commented Aug 18, 2020

var httpClient = HttpClient();
httpClient.idleTimeout = const Duration(seconds: 5);
httpClient.findProxy = (uri) => 'PROXY localhost:8888';
httpClient.badCertificateCallback = (X509Certificate cert, String host, int port) => true;

Doesn't work for me. Can not proxy

@JordiGiros
Copy link

@JordiGiros JordiGiros commented Sep 8, 2020

Will this ever be solved? It's not only affecting Charles proxy. This problem affects the capability of Flutter created APPS to run using local capabilities (using proxies) on platforms like Browserstack or similar... What in the end will limit the usage of Flutter for production apps... As I see it, this is an understandable limitation.

@gaaclarke
Copy link
Contributor

@gaaclarke gaaclarke commented Sep 8, 2020

@JordiGiros Did you see the workaround posted here: dart-lang/sdk#41376 (comment)

At this point I think making a plugin is your best bet, doing what the linked workaround is doing or wrapping Apple's http APIs.

The Dart team at this point has demonstrated this is not a priority for them. The workaround is easy to implement, you are better off doing that. Maybe the Flutter team can make an official plugin. I'll shop the idea around.

@eikebartels
Copy link

@eikebartels eikebartels commented Sep 8, 2020

Does anyone know the reason behind. I would like to understand the technical perspective

@JordiGiros
Copy link

@JordiGiros JordiGiros commented Dec 17, 2020

The proposal from @gaaclarke was discarded after trying it some months ago due to the fact that the library being used returns "null" when asking for the proxy info using it in BrowserStack.
The only option remaining is solving this natively in Flutter I think.
I see that the issue "passed first triage" some time ago.
Any news on this @markusaksli-nc ?
Thanks.

@JordiGiros
Copy link

@JordiGiros JordiGiros commented Feb 15, 2021

Hello @markusaksli-nc
Just trying to push for this improvement. Any news?
Thank you!

@markusaksli-nc
Copy link
Member

@markusaksli-nc markusaksli-nc commented Feb 15, 2021

I don't have any extra insight or news, #20376 (comment) is still the best answer probably.

FYI passed first triage just means that the first triage team has looked over the issue and it is ready for some further investigation/work

Not sure what the status is on what was mentioned but I'm not the one to ask for that either. This is probably still your best bet #20376 (comment)

@renenucci
Copy link

@renenucci renenucci commented Dec 26, 2021

I'm facing this too.

Trying to debug my own App but with Charles Proxy.

I can see even Instagram requests, but not my app requests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Mobile - iOS engine review
  
Engineer reviewed
Linked pull requests

Successfully merging a pull request may close this issue.

None yet