Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The expected 'Content-Type' header that should be returned is application/json. #66458

Closed
taylorchasewhite opened this issue Nov 21, 2020 · 37 comments

Comments

@taylorchasewhite
Copy link

The expected 'Content-Type' header that should be returned is application/json. You may get an error as mentioned below if you use anything else like application/json; charset=utf-8

"Verification of publisher domain failed. Error getting JSON file from https:///.well-known/microsoft-identity-association. The server returned an unexpected content type header value. "

This is a problem. For any application that is deployed use static web files on a deployment service, the developer is unable to modify the response headers. As a result developers cannot verify their publisher domain without changing their deployment strategy.

This issue is loosely related to 39665.

Please fix this issue, for the following reasons:

  • there is no reason the charset designation should invalidate the hosted JSON file
  • it is a outsized cost to the developer to workaround this bug (by changing service providers), as compared to the seemingly small cost to removing the content-type check.
  • every developer using a standards compliant web server is going to hit this, and some may not be able to adjust the response headers.

Thank you,
Taylor

Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

@Snuff-Daddy
Copy link

Snuff-Daddy commented Nov 21, 2020

Couldn't agree more. Wasted a bunch of my time.

@SaurabhSharma-MSFT
Copy link
Contributor

@taylorchasewhite Thanks for your feedback! We will investigate and update as appropriate.

@YashYash
Copy link

Thank you @taylorchasewhite . We have been waiting on this for a while now. Was just about to create an issue, till I came across yours. Not sure why it was considered ok to close the other issues, when users have been dealing with this for almost a year.

@Wtrapp
Copy link

Wtrapp commented Nov 27, 2020

+1

@taylorchasewhite
Copy link
Author

Hi @souravmishra-msft & @rwike77, just checking in on this, have you guys decided the direction for this issue (I know addressing it may take some time), but I'd like to know roughly the path forward here.

Thanks!

@antiGithub
Copy link

+1

@CelesteDG
Copy link
Contributor

Hello @taylorchasewhite and @antiGithub - Thank you for bringing attention to this issue. We just confirmed with our engineering team that they rolled out a code fix a month ago for a fix that seems related to this. Their request is for you to retry to determine if that fix that was applied also fixes this issue and to please let us know here so we can further follow up. Thank you.

@taylorchasewhite
Copy link
Author

Sweet! Thanks @CeleseteDG. I can confirm this now works.

Thanks a lot, I'm closing with this comment, but please feel free to make whatever changes on your end.

@ShaneMcNamara
Copy link

ShaneMcNamara commented Feb 17, 2022

I'm still seeing this issue.

I'm returning Content-Type: application/json and Content-Length: 111

What is going on here?

@ajainvivek
Copy link

ajainvivek commented Feb 25, 2022

I have copy pasted the content on to static folder of the website. I still get the above error. What is the expected content type?
@CelesteDG

I have hosted the file in below location - https://compensa.io/.well-known/microsoft-identity-association.json

Headers:

Screen Shot 2022-02-26 at 12 17 50 am

@rmnegatives
Copy link

rmnegatives commented Apr 11, 2023

I'm still seeing this issue.

I'm returning Content-Type: application/json and Content-Length: 111

What is going on here?

accept-ranges: bytes age: 802 content-length: 111 content-type: application/json date: Tue, 11 Apr 2023 01:00:27 GMT etag: "309ad933671cbb2cfeac2834791f8295" last-modified: Tue, 11 Apr 2023 00:45:01 GMT server: AmazonS3

Same and it's now April 10,2023
hitting the url that microsoft wants works fine in the browser as well

@rmnegatives
Copy link

@ShaneMcNamara and @ajainvivek did you ever find a solution for this?

@taylorchasewhite
Copy link
Author

@ShaneMcNamara and @ajainvivek did you ever find a solution for this?

I stopped using Azure 😉

@CelesteDG
Copy link
Contributor

@ajainvivek and @ShaneMcNamara - My sincerest apologies. This original issue has been closed and so it's not showing up in our active GitHub issues query and we lose track of the ongoing thread or questions that you have here. If you don't mind, can either of you please open a new issue against the article you're following or finding incorrect info on and provide for us repo steps or other info to investigate against on our end? Thank you for your patience and my sincerest apologies again for missing all the chat after the issue has been closed.

@taylorchasewhite
Copy link
Author

taylorchasewhite commented Apr 11, 2023

@CelesteDG can you refer to, and reopen, this issue -- #39665? It has the technical information you are requesting. I can reopen this one if you prefer.

@CelesteDG
Copy link
Contributor

Hello @taylorchasewhite - If you wouldn't mind reopening this issue, that would be great. I can then reassign it to someone on my team to investigate and to look into the thread in 39665 as well. Thank you for your help and ongoing feedback.

@CelesteDG
Copy link
Contributor

@rwike77 Please see the ongoing thread here and in #39665 - Please create a work item to track any doc bug and let's get any needed doc fixes in a sprint (hopefully addressed no later than Build, if possible). Please bubble up product issues to our feature team(s). Thank you.

@rmnegatives
Copy link

Posting this here for folks who spent hours, searching different threads of the same issue.
I found this deep into one thread, instead of fighting this issue there is an entirely different way to verify domains.
https://learn.microsoft.com/en-us/azure/active-directory/fundamentals/add-custom-domain

Add a custom domain and just update the TXT or MX records in your DNS and Azure will verify the domain that way instantly.
This was simpler and has worked for 3 domains so far.

@rwike77
Copy link
Contributor

rwike77 commented Apr 12, 2023

Apologies, we totally lost track of the conversation here and in the other issue. I'm working to get a hold of someone in engineering to get some help.

@TrangHoangMicrosoft
Copy link

@rmnegatives, @ShaneMcNamara and @ajainvivek I apologize for the inconvenience caused. I am one of the engineers working on the Publisher Domain Verification feature. Based on the description provided in the GitHub issue, I do not have sufficient information to investigate the issue further or reproduce it.

To assist me in identifying the root cause of the issue, could you kindly provide me with the file you are hosting in the path /.well-known/microsoft-identity-association.json, the site that you host the file or the trace ID, if you have it? This information will help me to narrow down the issue and work towards finding a solution.

@rmnegatives
Copy link

rmnegatives commented Apr 22, 2023

@TrangHoangMicrosoft
This is what I still have hosted
https://app.getslatewell.com/.well-known/microsoft-identity-association.json
Verification did not work, so I mean to reproduce, you will need some sort of site to host this in the exact directory, with a new azure directory.. and pretty sure you would get this error.
If not, you can use this url to debug why this does not return true on the azure side.
Using curl this seems spot on to the requirement?

@pl4nty
Copy link
Contributor

pl4nty commented Apr 25, 2023

@TrangHoangMicrosoft there's a good description from 2021 here: https://stackoverflow.com/a/58843651

The verification service requires a Content-Length header in responses, but Content-Length is not included in HTTP 1.1 chunked responses per the HTTP spec.

In my case, this breaks verification when using Cloudflare: https://tplant.com.au/.well-known/microsoft-identity-association.json

@TrangHoangMicrosoft
Copy link

Hi @rmnegatives, I looked into the link you provided and noticed that you may be using Amazon S3 to host your server. When you added the JSON to the path .well-known/microsoft-identity-association.json, you did not specify the content type. As a result, Amazon S3 automatically assumed that the file was a text file and added the content type as "text/html" instead of "application/json".

Additionally, the response that you received with the following headers: "accept-ranges: bytes age: 802 content-length: 111 content-type: application/json date: Tue, 11 Apr 2023 01:00:27 GMT etag: "309ad933671cbb2cfeac2834791f8295" last-modified: Tue, 11 Apr 2023 00:45:01 GMT server: AmazonS3" may have been misleading because the call was made from the customer to the server, which may have added some content type that wasn't accurate. However, when we make a call from our server to another server, the content type is different. Let me know if you have any questions!

@TrangHoangMicrosoft
Copy link

Hi @pl4nty, we apologize for the inconvenience caused to you. We have identified the root cause of the problem you faced and it was due to our strict validation of the content length. However, we have taken measures to modify our system and ensure that you do not face this problem again in the future. Please note that the code change updates may take a few weeks to reflect on your end. We request your patience until the update is complete. If you still encounter any issues, please feel free to let us know.

@Snuff-Daddy
Copy link

TrangHoangMicrosoft, you are still not getting it. The Content-Type is irrelevant. It should not be considered or used in any way. There is absolutely no reason to enforce some kind of restriction on it, and it is driving people nuts.

@taylorchasewhite
Copy link
Author

Hi @pl4nty, we apologize for the inconvenience caused to you. We have identified the root cause of the problem you faced and it was due to our strict validation of the content length. However, we have taken measures to modify our system and ensure that you do not face this problem again in the future. Please note that the code change updates may take a few weeks to reflect on your end. We request your patience until the update is complete. If you still encounter any issues, please feel free to let us know.

What measures are you referring to / do you have a PR / change description here to share with us? Want to make sure you're fixing this in a way that works for everyone, as @Snuff-Daddy alludes to.

@TrangHoangMicrosoft
Copy link

Hi @Snuff-Daddy, Checking the content type of a website is important for several reasons. Firstly, it ensures that the data received by our end is appropriate and can be processed correctly. Different types of data require different processing techniques, so knowing the content type helps ensure that the data is handled correctly and efficiently.

Secondly, knowing the content type can help with security. If a website sends unexpected data with a different content type than what was expected, it could be a sign of a malicious attack or a mistake in the server configuration. For example, if an attacker sends a malicious executable file with a disguised content type, it could bypass security measures that are expecting only certain content types. Checking the content type can help identify these issues and prevent security vulnerabilities.

In summary, checking the content type of a website is important for ensuring proper data processing and for identifying and preventing security vulnerabilities.

@Snuff-Daddy

This comment was marked as abuse.

@CelesteDG
Copy link
Contributor

Hello, everyone. We really value and appreciate your feedback and while we're not able to address them right away, they are in our queue and we review them and work to prioritize them alongside our other work items. I would just like to request that we please keep the discussion civil. Thank you for your understanding.

@rmnegatives
Copy link

rmnegatives commented May 10, 2023

Hi @rmnegatives, I looked into the link you provided and noticed that you may be using Amazon S3 to host your server. When you added the JSON to the path .well-known/microsoft-identity-association.json, you did not specify the content type. As a result, Amazon S3 automatically assumed that the file was a text file and added the content type as "text/html" instead of "application/json".

Additionally, the response that you received with the following headers: "accept-ranges: bytes age: 802 content-length: 111 content-type: application/json date: Tue, 11 Apr 2023 01:00:27 GMT etag: "309ad933671cbb2cfeac2834791f8295" last-modified: Tue, 11 Apr 2023 00:45:01 GMT server: AmazonS3" may have been misleading because the call was made from the customer to the server, which may have added some content type that wasn't accurate. However, when we make a call from our server to another server, the content type is different. Let me know if you have any questions!

Hello @TrangHoangMicrosoft
Thats on me, I should have stated that was a curl command and not from chrome.
Here is the curl with verbose mode, it says that content type is indeed the correct one.

➜  ~ curl -v https://app.getslatewell.com/.well-known/microsoft-identity-association.json
*   Trying 18.161.34.79:443...
* Connected to app.getslatewell.com (18.161.34.79) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=app.getslatewell.com
*  start date: Feb 21 00:00:00 2023 GMT
*  expire date: Jul 10 23:59:59 2023 GMT
*  subjectAltName: host "app.getslatewell.com" matched cert's "app.getslatewell.com"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M01
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x120811600)
> GET /.well-known/microsoft-identity-association.json HTTP/2
> Host: app.getslatewell.com
> user-agent: curl/7.77.0
> accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200 
< content-type: application/json
< content-length: 111
< date: Wed, 10 May 2023 09:17:48 GMT
< last-modified: Wed, 03 May 2023 17:51:58 GMT
< etag: "309ad933671cbb2cfeac2834791f8295"
< x-amz-server-side-encryption: AES256
< x-amz-version-id: Wmxg6S_XDdq8uGv1IXZ4DHcYaHRpwadV
< accept-ranges: bytes
< server: AmazonS3
< x-cache: Miss from cloudfront
< via: 1.1 58cb3b9ee1488076083df365a7774696.cloudfront.net (CloudFront)
< x-amz-cf-pop: BOS50-P2
< x-amz-cf-id: Q0DKaeBTmT8K9DwizvvsWHrqeVixc_kZvhjjSA04blu7Kc4Nl23JkQ==
< x-frame-options: DENY
< content-security-policy: base-uri 'self'; upgrade-insecure-requests; frame-ancestors 'none'; form-action https://api.getslatewell.com;
< 
{
  "associatedApplications": [
    {
      "applicationId": "9b6f492f-96b9-461b-8555-82919609faee"
    }
  ]
* Connection #0 to host app.getslatewell.com left intact
}% ```

@pl4nty
Copy link
Contributor

pl4nty commented Jun 4, 2023

@TrangHoangMicrosoft is the rollout complete? I'm still seeing the same content-length error message a few minutes ago (trace is zKn0KUeyKOXxIhoL23rCtY). The app is registered to a personal account so I'm not sure which tenant it's in

@clemvdm
Copy link

clemvdm commented Jun 20, 2023

@TrangHoangMicrosoft any feedback on how the code change rollout is going? I have been monitoring this thread for weeks, in hope that there is an update on the matter, as we are experiencing the exact same issue as that of @pl4nty where the Content-Length is not included in HTTP 1.1 chunked responses.

@pl4nty
Copy link
Contributor

pl4nty commented Jun 21, 2023

@TrangHoangMicrosoft I tried again today with the same error. I've been blocked by this issue for almost two months.

If the resolution timeline has changed, please advise so I can start working on the DNS verification method. That method may take months for me, or not even be possible.

@rwike77
Copy link
Contributor

rwike77 commented Jun 21, 2023

@pl4nty @clemvdm I spoke with Trang briefly yesterday. She said that she pushed a fix out a couple weeks ago, but it takes a while (up to 3 months) to fully deploy across all regions. So it may be that the fix hasn't rolled out to the regions you're using. She said she'd try to check on progress of the rollout. @TrangHoangMicrosoft can you update? Thanks.

@pl4nty
Copy link
Contributor

pl4nty commented Jun 22, 2023

Thanks @rwike77. I'm in APAC and happy to keep waiting, just need an approx timeline I can take to stakeholders (or updates if it slips). Cheers

@rwike77
Copy link
Contributor

rwike77 commented Jul 21, 2023

@TrangHoangMicrosoft tells me that the fix has rolled out to all regions, now. I'll close the issue now. If people are still experiencing problems, feel free to re-open the issue and tag @rwike77 and @TrangHoangMicrosoft. #please-close

@Snuff-Daddy
Copy link

Gee, all it took was 3 years and lots of complaints to fix something that was obviously broken. 3 years of trying to convince Microsoft of the obvious, and them finally getting around to lifting a finger to fix it. And how does a fix take 3 months to deploy? If ever there was a company that doesn't deserve its place of prominence, it's Microsoft. Their employees are not competent, not intelligent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests