43.0.1 is not sampling correctly

[mythmon]

The recipe "Out of Date Test 2" is attempting to sample English users running Firefox version 43.0.1 at a rate of 20%. We have evidence from telemetry that there are about 5.8 million people running 43.0.1. We have only gotten 300 offers for survey, which is significantly lower than 20% of 5.8 million, even accounting for locale.

The recipe's filter expression is:

(
   [normandy.userId, normandy.recipe.id]|stableSample(0.2)
   && normandy.locale in ['en-US', 'en-AU', 'en-CA', 'en-GB', 'en-NZ', 'en-ZA']
   && normandy.version == '43.0.1'
)

[mythmon]

My current theory is that 43.0.1 is some how not compatible with Normandy. I'm going to get a copy of 43.0.1 and test it's behavior.

[mythmon]

I've found the issue.

In Firefox 43.0.1 in my local tests, the request to /api/v1/action/:id/ is made with the header Accept: text/html. In Firefox 52 (my local nightly), the request is made with the correct header Accept: application/json. It seems that the default content type for XHR changed between 43 and 52. When an Accept: text/html is sent, the server correctly responded with an HTML page containing the API response, instead of the JSON response the client expects.

This wasn't an issue until recently. Before we deployed v18 (on 9/26 at 16:06 PDT), we had Nginx forcing the Accept header to be application/json because of bug 1293469. We ignored what the client sent, so Firefox's default behavior didn't matter.

The fix for this is pretty easy. We need to fix the api call to load action data to send the proper headers. That happens in fetchAction, which does not set any extra headers on the request. For comparison, in fetchRecipes, the header is correctly added.

[mythmon]

The net effect of all of that is that we won't have any data after 9/26 from versions of Firefox less than some threshold which I don't know, but is no less than Firefox 43.0.1.

[mythmon]

v20, with this fix, was deployed on September 30th. Any data after that should not exhibit this problem.