dart:io HttpClient's request force-transforms the header names to lowercase

[isoos]

The toLowerCase() transformation may be useful when processing the response headers, but when sending, the server could be checking the cased version and fail if it doesn't find it that way. (Yeah, bad server.)

[zoechi]

Sounds like #25120

[isoos]

Yeah, very similar problem.

[maguilar2k]

If the RFC-2616 states that header names are case-insensitive... why force the header names transformation to lowercase? shouldn't that be a decision taken by the developer?

[isoos]

I think the reason for the lowercase transform is HTTP/2: it standardizes all header names to be lowercase, and with that it makes somewhat sense to do the same here.

[shaxxx]

I've spent half a day debugging to find out http client forces headers as lowercase.
Guess what, I'm stuck with 10 year old client devices with HTTP server's using case sensitive headers.
Why force anything? Let me decide what I want to send with MY request

[holly-hacker]

Same here, I ended up copying over a bunch of code from the SDK after debugging a broken WebSocket connection for a few days. I get the rationale but this isn't documented anywhere which is really annoying.

[shaxxx]

Since I can't say to thousands of users to update their receivers or fix their HTTP server implementation, and I don't expect Dart team to change this behavior, I had no other choice but to change default HTTP client.

I took current HttpClient from master branch (requires Dart 2.5, Flutter 1.8+), changed it to leave headers alone and packed in the package. Hopefully it will save someone someone from crying and pulling up their hair. Just use it as a drop in replacement for HttpClient. Tested and works with Dio.

You can find package HERE.

[a-siva]

There seems to be a suggestion that we should add an option to not change the case.

[zichangg]

@lrhn Any idea? This would be a breaking change if adding one more flag.

[zichangg]

To be more specific, A flag is probably needed to be passed from httpclient all the way down to httpheader.

sdk/sdk/lib/_http/http_impl.dart

Line 2071 in a1fdfb8

_HttpClient(this._context);

sdk/sdk/lib/_http/http_impl.dart

Line 770 in a1fdfb8

_HttpOutboundMessage(Uri uri, String protocolVersion, _HttpOutgoing outgoing,

sdk/sdk/lib/_http/http_impl.dart

Line 1078 in a1fdfb8

_HttpClientRequest(_HttpOutgoing outgoing, Uri uri, this.method, this._proxy,

Inside headers,

sdk/sdk/lib/_http/http_headers.dart

Line 41 in 4102e63

List<String> operator [](String name) => _headers[name.toLowerCase()];

sdk/sdk/lib/_http/http_headers.dart

Line 44 in 4102e63

name = name.toLowerCase();

sdk/sdk/lib/_http/http_headers.dart

Line 44 in 4102e63

name = name.toLowerCase();

[shaxxx]

Thats the one way to look at it.
Why would HttpClient assume anything?
You have bunch of Abstract classes in HTTP client that are used only as internal implementations. Take HttpHeaders for instance. It's abstract, and HttpClient uses one single concrete implementation, which is internal, and has no way to use some other implementation.
Why making abstract class in a first place if you're forcing it to be used in a single way?
Good and probably non breaking start would be to

• move internal implementations to separate public classes/files
• refactor existing calls to new classes
• add a way for user to provide specific implementation of abstract class (not just HttpHeaders)
• if no specific implementation is provided use default one

Last two can be done as optional class constructor with default value params (better) or as public properties (less transparent IMHO).
I'm speaking from top of my head now, but this shouldn't breake anything.
Dart has no reflection, and internal calls can be easily refactored.
And once you're there, in default HttpHeaders expose map of all headers as public property, leave current behaviour, as is, but let user modify the map directly however he/she wants. That's extra, I'd be fine if I could make HttpClient use my HttpHeaders instead default ones.

Just my 2 cents.

[songfei]

any update?

[zichangg]

Still in progress. I expect this cl to be landed within next few weeks.

[shaxxx]

Is there any ETA when this will land in a Dart release?

[zichangg]

The cl is pretty much done. But it has to wait for breaking change request getting approved.

[cdm2012]

Creating an API Client using HTTPClient with the following request headers:

I/flutter (32611): headers: {
I/flutter (32611):   "Content-type": "application/json",
I/flutter (32611):   "Accept": "application/json",
I/flutter (32611):   "Authorization": "Bearer: E7HlYmfiXwhLQT1nYApOAE4wqhvCZZXndrT7kiRv8tUBycj3npVnYwnQnH34"
I/flutter (32611): }
I/flutter (32611): 
I/flutter (32611): 
I/flutter (32611): 
I/flutter (32611): request.headers: 
I/flutter (32611): content-type: application/json
I/flutter (32611): accept: application/json
I/flutter (32611): authorization: Bearer: E7HlYmfiXwhLQT1nYApOAE4wqhvCZZXndrT7kiRv8tUBycj3npVnYwnQnH34

And the server (nginx/1.15.8) is returning a 400 error:

I/flutter (32611): <html>
I/flutter (32611): <head><title>400 Bad Request</title></head>
I/flutter (32611): <body>
I/flutter (32611): <center><h1>400 Bad Request</h1></center>
I/flutter (32611): <hr><center>nginx/1.15.8</center>
I/flutter (32611): </body>
I/flutter (32611): </html>

Using the same headers in android with the following request headers:

E: Called doInBackground w/ headers: [Accept: application/json, Content-type: application/json, Authorization: Bearer E7HlYmfiXwhLQT1nYApOAE4wqhvCZZXndrT7kiRv8tUBycj3npVnYwnQnH34]

And the same server is returning a 200 response, so I'm really looking forward to integrating the change by @zichangg : #39657 (comment)

[accek]

I/flutter (32611):   "Authorization": "Bearer: E7HlYmfiXwhLQT1nYApOAE4wqhvCZZXndrT7kiRv8tUBycj3npVnYwnQnH34"

It looks like the colon : after Bearer should not be there.

[cdm2012]

Thanks @accek, that was definitely an oversight. I just tried it again with these headers:

I/flutter ( 3473): headers: {
I/flutter ( 3473):   "Content-type": "application/json",
I/flutter ( 3473):   "Accept": "application/json",
I/flutter ( 3473):   "Authorization": "Bearer E7HlYmfiXwhLQT1nYApOAE4wqhvCZZXndrT7kiRv8tUBycj3npVnYwnQnH34"
I/flutter ( 3473): }
I/flutter ( 3473): 
I/flutter ( 3473): 
I/flutter ( 3473): 
I/flutter ( 3473): request.headers: 
I/flutter ( 3473): content-type: application/json
I/flutter ( 3473): accept: application/json
I/flutter ( 3473): authorization: Bearer E7HlYmfiXwhLQT1nYApOAE4wqhvCZZXndrT7kiRv8tUBycj3npVnYwnQnH34

But the result was still the same:

I/flutter ( 3473): responseString: <html>
I/flutter ( 3473): <head><title>400 Bad Request</title></head>
I/flutter ( 3473): <body>
I/flutter ( 3473): <center><h1>400 Bad Request</h1></center>
I/flutter ( 3473): <hr><center>nginx/1.15.8</center>
I/flutter ( 3473): </body>
I/flutter ( 3473): </html>

I didn't think to add the nginx logs before, but here are all the nginx logging from last night:

2019/12/29 03:15:05 [error] 3653#3653: *65 open() "/var/www/reely-api/testing/public/api/version" failed (2: No such file or directory), client: 10.0.0.5, server: reely.the-mac.local, request: "POST /api/version HTTP/1.1", host: "reely.the-mac.local"

Currently there are no logs showing, but I think that error was from using the wrong url protocol.

[sortie]

@accek We're working to make the breaking change, but it's a workaround for bad servers. The standard requires the protocol headers are case insensitive and you get the server fixed instead of debugging the issue here. I assume those bearer tokens are for debugging and have no actual access.

[cdm2012]

Yes the token is not usable @accek , it only worked for a private internal server. Could the solution be to use the HTTP/2 standard optionally @isoos , similar to @zichangg 's solution (but an opt in solution)?

@sortie I looked into the Server side, and according to the creator of Laravel, Laravel literally does not touch request headers in any form or fashion.

[MiriDevAndro]

The flutter channel dev dart sdk 2.8.0-dev.15.0 has to much bug and stable has ver dart sdk 2.7.0 max

[mit-mit]

Sorry, you will have to wait for the dev channel to stabilize then.

[MiriDevAndro]

ok thnx for explanation

[eirikb]

I'm also struggling with a server only accepting cased headers.
How can I use the above solution?

Tried:

final body = '{"hello":"world"}';
final client = new HttpClient();
final req = await client.put('localhost', 8000, '/');
req.headers.add('Content-Type', 'application/json', preserveHeaderCase: true);
req.headers.add('Content-Length', body.length, preserveHeaderCase: true);
req.write(body);
await req.close();

Versions:

> flutter --version
Flutter 1.15.21 • channel master • /home/eirikb/.cache/yay/flutter-git/flutter
Framework • revision e2b4edd286 (3 weeks ago) • 2020-03-13 02:46:02 -0400
Engine • revision d7a00b8b09
Tools • Dart 2.8.0 (build 2.8.0-dev.14.0 eb9c26bd37)

But headers are still lower-cased.
Tested by running Flutter in Android Emulator and dart in terminal:

> /opt/flutter/bin/cache/dart-sdk/bin/dart --version
Dart VM version: 2.8.0-dev.14.0.flutter-eb9c26bd37 (Fri Mar 13 01:41:41 2020 +0000) on "linux_x64"
> /opt/flutter/bin/cache/dart-sdk/bin/dart api_test.dart

Output from nc (also see same result in Wireshark):

> nc -l -p 8000
PUT / HTTP/1.1
user-agent: Dart/2.8 (dart:io)
content-type: application/json
accept-encoding: gzip
content-length: 17
host: localhost:8000

{"hello":"world"}

Printing headers show they were correctly set in HttpHeaders, but not sent correctly:

print(req.headers);

user-agent: Dart/2.8 (dart:io)
Content-Type: application/json
accept-encoding: gzip
Content-Length: 17
host: localhost:8000

[shaxxx]

@mit-mit I'm having exactly the same issue as @eirikb .
preserveHeaderCase as far as I can see is currently not working.
Using

Dart VM version: 2.8.2 (stable) (Mon May 11 15:06:42 2020 +0200) on "windows_x64"

Can someone please explain what's current status on this, is it working, if yes how, if not what's happening with it? Should this be re-opened?

[zichangg]

@shaxxx Sorry for the delayed reply. Code reviews for my fix took some time, I'll land the fix today.

[shaxxx]

@zichangg Thanks for the update, as far I can understand preserveHeaderCase should work in the next flutter stable?
It wasn't really clear from this thread if this is currently working or not since no one confirmed this is still an issue and this bug is closed. Is there another bug tracking progress on this?

[a-siva]

reopening issue so that we can track when the bug fix CL lands and update on when it gets rolled into Flutter. You will probably see it in the dev channel first.

[talregev]

reopening this issue because force lowercase http header make it breakable from legacy server that don't check for small letter. and http1 header is sensitive case.
Please make by default non force lowercase http header, and add an option to people to want to force it.
It break my communication with the server.
Also http in web, doing as http should. it non force lower case and adding automatically content-length in the header.
But, http in dart-io force you lowercase http header and not adding automatically content-length.
This is bad inconsistency. http request should be the same across all the platform. and doing the same as http in web.

[MiriDevAndro]

@talregev im okay with you , they should insert support for http 1.1

[cesuper]

Hey,

I also miss the case sensitive version of the headers. Any update in this topic?

[mraleph]

@cesuper @talregev this issue was closed because HttpHeaders add and set methods were expanded with explicit support for case-sensitive header via preserveHeaderCase: true named argument.

If this does not work for you (meaning: you set preserveHeaderCase: true but headers are sent in forced lowercase) then you should file a separate issue and provide a minimal reproduction of the issue.

There are no plans to switch to case-sensitive headers by default.

[MiriDevAndro]

Any update on this?
I think that this bug is on native compiled library, any idea ?

[mraleph]

@MiriDevAndro #33501 (comment)