Bug in HttpWebRequest when call an endpoint that returns 302

[Petermarcu]

@mrquake commented on Mon Apr 23 2018

Issue Title

When you call an endpoint that returns 302 the redirection url will have backslashes instead of forward slashes in "localpath".

General

For example when you call Azure API using Fiddler or any other web debugging tool: "https://management.azure.com/subscriptions/xxx-xxx-xxx-xxxx/providers/Microsoft.Commerce/RateCard" with property parameters, token, etc...

It will return 302 and redirect to: https://ratecard.blob.core.windows.net/ratecards/xxxx-xxxx-xxxxx-xxxxx/MS-AZR-0017P/USD/en-US/False/2015-06-01-preview/yyyy-yyyyy-yyyyy-yyyyy.json

No problem whatsoever...

But when you make the request using .NET Core 1.1/2.0 HttpWebRequest...
The redirection tries to call something like that:

https://ratecard.blob.core.windows.net/ratecards/xxxx-xxxx-xxxx-xxxxx%5CMS-AZR-0017P%5CUSD%5Cen-US%5CFalse/2015-06-01-preview/yyyy-yyyy-yyyy-yyyy-yyyy.json

With encoded backslashes instead of forward slashes....

[karelz]

@mrquake do you know if it is a problem with .NET Core 2.1?
Do you have a minimal repro demonstrating the problem which we could run in house?

[mrquake]

@karelz I don't know... but I know that using full framework it works perfectly...

You can use this example here (https://github.com/Azure-Samples/billing-dotnet-ratecard-api)... it uses the same logic that I used but you need to convert to .NET Core...

I investigated further and it seems that the problem is not related to the backslashes but the Authentication header that goes in the first request but when it calls the redirection it send again and it is not necessary to send that and because of that it returns 400 bad request...

[davidsh]

I investigated further and it seems that the problem is not related to the backslashes but the Authentication header that goes in the first request but when it calls the redirection it send again and it is not necessary to send that and because of that it returns 400 bad request...

Can you please attach a WireShark trace and/or a Fiddler trace? That would help diagnose this problem.

[mrquake]

yes.. here it is:

GET https://management.azure.com/subscriptions/xxxxx-xxxxx-xxxxx-xxxxxx-xxxxxx/providers/Microsoft.Commerce/RateCard?api-version=2016-08-31-preview&$filter=OfferDurableId%20eq%20'MS-AZR-0017P'%20and%20Currency%20eq%20'USD'%20and%20Locale%20eq%20'en-US'%20and%20RegionInfo%20eq%20'US' HTTP/1.1
Connection: Keep-Alive
Content-Type: application/json
Accept: application/json
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6ImlCakwxUmNxemhpeTRmcHhJeGRacW9oTTJZayIsImtpZCI6ImlCakwxUmNxemhpeTRmcHhJeGRacW9oTTJZayJ9.eyJhdWQiOiJodHRwczovL21hbmFnZW1lbnQuYXp1cmUuY29tLyIsImlzcyI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0LzU3MmYyMzI4LWJiODYtNDQ1NS1hZDg3LTY4MGE1ZTQ3NTA4Ni8iLCJpYXQiOjE1MjUyOTUxMjYsIm5iZiI6MTUyNTI5NTEyNiwiZXhwIjoxNTI1Mjk5MDI2LCJhaW8iOiJZMmRnWUxpWE05MDFOL25qNmlMUGY3OFkvM3BaQVFBPSIsImFwcGlkIjoiNmZmZWNkZjMtMWI3My00MzZjLWIyNzItZGI3NzFhNmI1NmU3IiwiYXBwaWRhY3IiOiIxIiwiZV9leHAiOjI2MjgwMCwiaWRwIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvNTcyZjIzMjgtYmI4Ni00NDU1LWFkODctNjgwYTVlNDc1MDg2LyIsIm9pZCI6IjI2MTUyNTdiLTc2NDYtNDk5Zi05ZWQyLWNkYThlYWUyOGJhZiIsInN1YiI6IjI2MTUyNTdiLTc2NDYtNDk5Zi05ZWQyLWNkYThlYWUyOGJhZiIsInRpZCI6IjU3MmYyMzI4LWJiODYtNDQ1NS1hZDg3LTY4MGE1ZTQ3NTA4NiIsInV0aSI6IkxoTEpiLXBVVDAyRTZkcXY3Y3NEQUEiLCJ2ZXIiOiIxLjAifQ.El1sQdtUDeGJ0ZqyIB5JSXbpICKKKMLS_u55SbYfo42q2AhX0fNqoLx5BgBlPJ5QsXKW3qyra4k2JbW2qb_rscda3tMCQtsyeOoWWQEKKJzj74grrSUxNmmToy9b1E-gvJ9Qw3iiCJH2p6zu3fQLE3JwAMFIVhW-We7Pk6ay6X4rmcTpBlHiaaC-0DG8_T32SdOaVxHG6xYcLFEjDCL9bvlA2I-RM8gfTSu08s4PTBa7h3V24BrvZRx8XfktUCRasaPCwCngTaIKRStQh62fyit1tyujez7TLi8xU08ZZnP2jbkr6GC-lY96J230jO-jVGeoGWELZ9_Qj-BIBNDNxA
Content-Length: 0
Host: management.azure.com

HTTP/1.1 302 Found
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
Location: https://ratecard.blob.core.windows.net/ratecards/27d96b24-db2f-4847-b3a0-d556072000f6%5CMS-AZR-0017P%5CUSD%5Cen-US%5CFalse/2016-08-31-preview/acf1c7b1-914f-404a-88ae-32ef12097824.json?sv=2016-05-31&sr=b&sig=qZbgf8PmYjfDzk9C%2FJHFXzHHjVW7fcPhaIsuvcTkPxo%3D&se=2018-05-02T21%3A26%3A59Z&sp=r&rsct=application%2Fjson
X-MS-Correlation-Id: 6b0c3b7b-e0e4-0002-727c-0c6be4e0d301
X-Content-Type-Options: nosniff
Server: Microsoft-IIS/10.0
x-ms-ratelimit-remaining-subscription-reads: 14961
x-ms-request-id: 04b5fccb-7101-46ae-ac0b-3f804456736c
x-ms-correlation-request-id: 04b5fccb-7101-46ae-ac0b-3f804456736c
x-ms-routing-request-id: BRAZILSOUTH:20180502T211200Z:04b5fccb-7101-46ae-ac0b-3f804456736c
Strict-Transport-Security: max-age=31536000; includeSubDomains
Date: Wed, 02 May 2018 21:11:59 GMT
Content-Length: 0

GET https://ratecard.blob.core.windows.net/ratecards/27d96b24-db2f-4847-b3a0-d556072000f6%5CMS-AZR-0017P%5CUSD%5Cen-US%5CFalse/2016-08-31-preview/acf1c7b1-914f-404a-88ae-32ef12097824.json?sv=2016-05-31&sr=b&sig=qZbgf8PmYjfDzk9C%2FJHFXzHHjVW7fcPhaIsuvcTkPxo%3D&se=2018-05-02T21%3A26%3A59Z&sp=r&rsct=application%2Fjson HTTP/1.1
Connection: Keep-Alive
Content-Type: application/json
Accept: application/json
Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6ImlCakwxUmNxemhpeTRmcHhJeGRacW9oTTJZayIsImtpZCI6ImlCakwxUmNxemhpeTRmcHhJeGRacW9oTTJZayJ9.eyJhdWQiOiJodHRwczovL21hbmFnZW1lbnQuYXp1cmUuY29tLyIsImlzcyI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0LzU3MmYyMzI4LWJiODYtNDQ1NS1hZDg3LTY4MGE1ZTQ3NTA4Ni8iLCJpYXQiOjE1MjUyOTUxMjYsIm5iZiI6MTUyNTI5NTEyNiwiZXhwIjoxNTI1Mjk5MDI2LCJhaW8iOiJZMmRnWUxpWE05MDFOL25qNmlMUGY3OFkvM3BaQVFBPSIsImFwcGlkIjoiNmZmZWNkZjMtMWI3My00MzZjLWIyNzItZGI3NzFhNmI1NmU3IiwiYXBwaWRhY3IiOiIxIiwiZV9leHAiOjI2MjgwMCwiaWRwIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvNTcyZjIzMjgtYmI4Ni00NDU1LWFkODctNjgwYTVlNDc1MDg2LyIsIm9pZCI6IjI2MTUyNTdiLTc2NDYtNDk5Zi05ZWQyLWNkYThlYWUyOGJhZiIsInN1YiI6IjI2MTUyNTdiLTc2NDYtNDk5Zi05ZWQyLWNkYThlYWUyOGJhZiIsInRpZCI6IjU3MmYyMzI4LWJiODYtNDQ1NS1hZDg3LTY4MGE1ZTQ3NTA4NiIsInV0aSI6IkxoTEpiLXBVVDAyRTZkcXY3Y3NEQUEiLCJ2ZXIiOiIxLjAifQ.El1sQdtUDeGJ0ZqyIB5JSXbpICKKKMLS_u55SbYfo42q2AhX0fNqoLx5BgBlPJ5QsXKW3qyra4k2JbW2qb_rscda3tMCQtsyeOoWWQEKKJzj74grrSUxNmmToy9b1E-gvJ9Qw3iiCJH2p6zu3fQLE3JwAMFIVhW-We7Pk6ay6X4rmcTpBlHiaaC-0DG8_T32SdOaVxHG6xYcLFEjDCL9bvlA2I-RM8gfTSu08s4PTBa7h3V24BrvZRx8XfktUCRasaPCwCngTaIKRStQh62fyit1tyujez7TLi8xU08ZZnP2jbkr6GC-lY96J230jO-jVGeoGWELZ9_Qj-BIBNDNxA
Content-Length: 0
Host: ratecard.blob.core.windows.net

HTTP/1.1 400 Authentication information is not given in the correct format. Check the value of Authorization header.
Content-Length: 297
Content-Type: application/xml
Server: Microsoft-HTTPAPI/2.0
x-ms-request-id: aa2d3a65-401e-00e7-035a-e2216e000000
Date: Wed, 02 May 2018 21:12:01 GMT

InvalidAuthenticationInfoAuthentication information is not given in the correct format. Check the value of Authorization header.

RequestId:aa2d3a65-401e-00e7-035a-e2216e000000
Time:2018-05-02T21:12:01.3890762Z

Without the Authorization header in the second request it works correctly...

[davidsh]

Without the Authorization header in the second request it works correctly...

The HTTP stack in .NET Core is not adding the "Authorization: Bearer" header directly. Perhaps something in the Azure SDK is adding that? In fact, with redirections (such as 302), the HTTP stack in .NET Core will remove Authorization request headers in most cases.

cc: @stephentoub

[mrquake]

I'm not using any Azure SDK... I do exactly the way is in this code below:

https://github.com/Azure-Samples/billing-dotnet-ratecard-api/blob/master/ConsoleApp-Billing-RateCard/ConsoleProj/Program.cs

[mrquake]

please take a look at the Sadiqh Ahmed Microsoft (MSFT CSG) answers in this link below:

https://social.msdn.microsoft.com/Forums/azure/en-US/72de1bac-9657-434f-8811-365c9fe09a69/authentication-information-is-not-given-in-the-correct-format-check-the-value-of-authorization?forum=windowsazurepurchasing

[davidsh]

We have confirmed the issue does not occur on .NET 2.1 RC and we are investigating additional supported product versions.

Please try out .NET Core 2.1 RC.
See: https://blogs.msdn.microsoft.com/dotnet/2018/05/07/announcing-net-core-2-1-rc-1/