Network calls don't show in Charles Proxy Debugger

[nashfive]

Steps to Reproduce

1. Install Charles Proxy and configure it.
2. Configure the simulators to use Charles Proxy (follow the tutorials for iOS and Android)
3. Test any native apps (browser or your own native apps that makes network calls) and see that all calls are logged in the Charles window
4. Run the Flutter app and make a network call (http and/or https). Not a single trace of network calls :(

My HTTPS calls are actually fired and I get HTTP responses in my code and I can see the calls in my server logs.
Flutter (or Dart ?) doesn't seem to use the system proxy so I can't see any of my network calls' payloads.

I tried to launch the app in debug mode with http_proxy and https_proxy in the command line but this doesn't change anything :

export https_proxy=http://<charles-hostname>:<charles-port>;
export http_proxy=http://<charles-hostname>:<charles-port>;
flutter run

Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel beta, v0.5.1, on Mac OS X 10.13.6 17G65, locale en-CH)
[✓] Android toolchain - develop for Android devices (Android SDK 28.0.1)
[✓] iOS toolchain - develop for iOS devices (Xcode 9.4.1)
[✓] Android Studio (version 3.1)
[!] VS Code (version 1.25.1)
[✓] Connected devices (1 available)

[LcTwisk]

It's also not working on a physical device... :(

[ErikDohmen]

Same here, both on physical device nor simulator/ emulator no way to get them to show up in charlesproxy while badly needed

[xster]

We're still trying to figure out how to best default this but for now, could you guys try using the .findProxy method on HttpClients? https://api.dartlang.org/stable/2.1.0/dart-io/HttpClient/findProxy.html

If you want to override how HttpClients are created, use https://api.dartlang.org/stable/2.1.0/dart-io/HttpOverrides-class.html

[dnfield]

This should probably be addressed upstream in dart:io.

/cc @zanderso

From conversation with him, our best bet seems to be to target updating

https://github.com/dart-lang/sdk/blob/master/runtime/bin/platform_macos.cc#L203 with information from https://developer.apple.com/documentation/cfnetwork/1426754-cfnetworkcopysystemproxysettings?language=objc.

[dnfield]

Does this work as-is on Android? If not we'd probably want a similar strategy there.

[xster]

Nope. Stackoverflow says https://stackoverflow.com/a/13616054/. Can we access Java APIs from the dart:io code?

[zanderso]

There is currently not a good way to make JNI calls from the dart:io implementation. We typically deal with this problem by adding hooks where the engine can install a custom callback. I will investigate that approach.

[AbnerChen028]

I'm using dio to set proxy.

[akindone]

I'm using dio to set proxy.

I am using dio too. iOS physical device can get both http and https api payload, but android physical device can only get http api payload. And I have install certification for device and charles can get https api for native app. but can't work for flutter app. Did you encounter this problem?

[akindone]

I'm using dio to set proxy.

I am using dio too. iOS physical device can get both http and https api payload, but android physical device can only get http api payload. And I have install certification for device and charles can get https api for native app. but can't work for flutter app. Did you encounter this problem?

I find a way for android devices. But I am not sure any side effect

bool isProxyChecked = true // a variable for debug
String proxy = '192.168.2.2:8888' // ip:port
final dio = Dio()
    ..onHttpClientCreate = (client) {
      client.badCertificateCallback =
          (X509Certificate cert, String host, int port) {
        return isProxyChecked && Platform.isAndroid;
      };
      client.findProxy = (url) {
        return isProxyChecked ? 'PROXY $proxy' : 'DIRECT';
      };
    }

[wangyongf]

I'm using dio to set proxy.

I am using dio too. iOS physical device can get both http and https api payload, but android physical device can only get http api payload. And I have install certification for device and charles can get https api for native app. but can't work for flutter app. Did you encounter this problem?

I find a way for android devices. But I am not sure any side effect

bool isProxyChecked = true // a variable for debug
String proxy = '192.168.2.2:8888' // ip:port
final dio = Dio()
    ..onHttpClientCreate = (client) {
      client.badCertificateCallback =
          (X509Certificate cert, String host, int port) {
        return isProxyChecked && Platform.isAndroid;
      };
      client.findProxy = (url) {
        return isProxyChecked ? 'PROXY $proxy' : 'DIRECT';
      };
    }

It works for me, thanks

[dazza5000]

I was able to get this working using HttpClient with a snippet from the following SO question:

https://stackoverflow.com/questions/50949938/flutter-app-ios-simulator-set-transparent-proxy

localhost can be the ip address for your computer like 192.168.1.77

HttpClient client = HttpClient();
client.findProxy = (uri) {
  return "PROXY localhost:3128;";
};

A more full implementation:

  static HttpClient getHttpClient() {
    HttpClient client = new HttpClient()
      ..findProxy = (uri) {
        return "PROXY 192.168.1.199:8888;";
      }
      ..badCertificateCallback =
      ((X509Certificate cert, String host, int port) => true);

    return client;
  }

[zoechi]

@dazza5000 and you can use HttpOverrides to make Flutter use the same HttpClient. See https://stackoverflow.com/questions/54321077/best-way-to-set-default-header-for-all-request-in-flutter-http-request for an example

[lijy91]

@nashfive
I developed a flippersdk for flutter, you can use flipper to debug your network request, maybe it's easier.

https://github.com/blankapp/flutter_flipperkit

[stuartsoft]

@dazza5000 TY!

[OPY-bbt]

(dio.httpClientAdapter as DefaultHttpClientAdapter).onHttpClientCreate = (client) {
    // config the http client
    client.findProxy = (uri) {
        //proxy all request to localhost:8888
        return "PROXY localhost:8888";
    };
    // you can also create a new HttpClient to dio
    // return new HttpClient();
};

[bstolinski]

class MyHttpOverrides extends HttpOverrides {
  MyHttpOverrides(this._ip);

  final String _ip;

  @override
  HttpClient createHttpClient(SecurityContext context) {
    return super.createHttpClient(context)
      ..findProxy = (uri) {
        return "PROXY $_ip:8888;";
      }
      ..badCertificateCallback =
          (X509Certificate cert, String host, int port) => true;
  }
}

Android Simulator

HttpOverrides.global = MyHttpOverrides("10.0.2.2");

iPhone Simulator

HttpOverrides.global = MyHttpOverrides("localhost");

physical device (Android/iPhone)
Also add proxy server settings to your device in system settings.

HttpOverrides.global = MyHttpOverrides("192.168.0.100"); //ip address of your computer

This can also be used for simulators (skip adding proxy server settings in system settings).

[jamesdixon]

For anyone else who ends up here, I compiled all of the solutions above (for the most part) into a blog post along with some other options to debug network requests in Flutter:

https://flutterigniter.com/debugging-network-requests/

[gaaclarke]

I suspect dart-lang/sdk#41376 will solve this problem on iOS. Using posix sockets is probably not respecting the OS's proxy settings like it is for VPN settings.

[x-raay]

Before I forget, dart devtools also supports network traffic monitoring

[fryette]

@saurabhrayakwar Can you see a body of requests?

[ashbin]

(dio.httpClientAdapter as DefaultHttpClientAdapter).onHttpClientCreate = (client) {
    // config the http client
    client.findProxy = (uri) {
        //proxy all request to localhost:8888
        return "PROXY localhost:8888";
    };
    // you can also create a new HttpClient to dio
    // return new HttpClient();
};

If you are using android studio emulator use 10.0.2.2 and for genymotion 10.0.3.2 if localhost is not working.

[locskot]

var httpClient = HttpClient();
httpClient.idleTimeout = const Duration(seconds: 5);
httpClient.findProxy = (uri) => 'PROXY localhost:8888';
httpClient.badCertificateCallback = (X509Certificate cert, String host, int port) => true;

Doesn't work for me. Can not proxy

[JordiGiros]

Will this ever be solved? It's not only affecting Charles proxy. This problem affects the capability of Flutter created APPS to run using local capabilities (using proxies) on platforms like Browserstack or similar... What in the end will limit the usage of Flutter for production apps... As I see it, this is an understandable limitation.

[gaaclarke]

@JordiGiros Did you see the workaround posted here: dart-lang/sdk#41376 (comment)

At this point I think making a plugin is your best bet, doing what the linked workaround is doing or wrapping Apple's http APIs.

The Dart team at this point has demonstrated this is not a priority for them. The workaround is easy to implement, you are better off doing that. Maybe the Flutter team can make an official plugin. I'll shop the idea around.

[eikebartels]

Does anyone know the reason behind. I would like to understand the technical perspective

[JordiGiros]

The proposal from @gaaclarke was discarded after trying it some months ago due to the fact that the library being used returns "null" when asking for the proxy info using it in BrowserStack.
The only option remaining is solving this natively in Flutter I think.
I see that the issue "passed first triage" some time ago.
Any news on this @markusaksli-nc ?
Thanks.

[JordiGiros]

Hello @markusaksli-nc
Just trying to push for this improvement. Any news?
Thank you!

[markusaksli-nc]

I don't have any extra insight or news, #20376 (comment) is still the best answer probably.

FYI passed first triage just means that the first triage team has looked over the issue and it is ready for some further investigation/work

Not sure what the status is on what was mentioned but I'm not the one to ask for that either. This is probably still your best bet #20376 (comment)

[renenucci]

I'm facing this too.

Trying to debug my own App but with Charles Proxy.

I can see even Instagram requests, but not my app requests.

[brianquinlan]

There are other solutions suggested for using Charles Proxy i.e. setting HttpClient.findProxy.

Another option is cupertino_http, which is a new experimental Flutter plugin that provides access to Apple's Foundation URL Loading System - which honors iOS proxy settings.

cupertino_http has the same interface as package:http Client so it is easy to use in a cross-platform way. For example:

late Client client;
if (Platform.isIOS) {
  final config = URLSessionConfiguration.ephemeralSessionConfiguration()
    # Do whatever configuration you want.
    ..allowsCellularAccess = false
    ..allowsConstrainedNetworkAccess = false
    ..allowsExpensiveNetworkAccess = false;
  client = CupertinoClient.fromSessionConfiguration(config);
} else {
  client = IOClient(); // Uses an HTTP client based on dart:io
}

final response = await client.get(Uri.https(
    'www.googleapis.com',
    '/books/v1/volumes',
    {'q': 'HTTP', 'maxResults': '40', 'printType': 'books'}));

I would really appreciate it if you can try cupertino_http out and see if it solves the VPN issues for you.

Comments or bugs in the cupertino_http issue tracker would be appreciated!

[SEGVeenstra]

I ran into the same issue but then one of my colleagues suggested this package: https://pub.dev/packages/http_proxy.
For me it allowed me to use Charles right away, so if that is your goal you can give it a try.

That said, It would be nice to work out-of-the-box.

[komaxx]

@SEGVeenstra
Glad that it works for you!
Please be aware, though, that http_proxy turns off all certificate checks, so your app is not protected by TLS, opening it up to all sorts of attacks.
So make sure that you only use it in dev code!

[brianquinlan]

cupertino_http has been available for over a year now and has transparent support for iOS proxies.

We are recommending that people use it on iOS.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.