Different scoring between observatory.mozilla.org and the git repository

[dalf]

The scoring seems different between observatory.mozilla.org and the git repository. May be I missed a setting ?

For example:

• The git repository:
  • Score: 45
  • Grade: C-
  • Tests Passed: 8/12
  • sum([ t["score_modifier"] for t in result['tests'].values()]) returns 55
  • Code:

from httpobs.scanner.local import scan 
result = scan('searx.laquadrature.net')

• On https://observatory.mozilla.org
  • Score: 40/100
  • Grade: D+
  • Tests Passed: 7/11 (no contribute test)
  • The sum of all the score is 55 (same results).

It seems there is -5 score modifier somewhere ?
This value is not constant. A test on a.searx.space gives the same results on observatory.mozilla.org and the git repository: 110 / A+.

See https://gist.github.com/dalf/5f12e52664eee4c6c610ace8c02044bb for the results return by httpobs.scanner.local.scan

[EDIT] I have called httpsobs/scripts/httpobs-regen-hsts-preload before the local test.

[floatingatoll]

It looks like there's an issue with the score report you're receiving with the local scanner. When the site has negative scores, the final grade does not consider positive scores. Your site has a +5 modifier for Referer Policy, which the website correctly sets aside when producing the score of 40 / D+. However, the local scanner report does not do so, and reports a score of 45, incorporating the +5 from the Referer Policy when it probably should not.

In this scenario, the website's grade is correct: you should only receive 40/100 for this site, not 45/100, as the Referer Policy +5 is discarded. Repairs may need to be made to the local scanner score report to bring that back into alignment with the scoring algorithm.

[dalf]

Thank you for your detail information. I've submitted a PR. I don't know if it fit the spirit of the project (and the status of the project).

[LeoMcA]

I've fixed this in the next branch by merging the local scanning logic into the api scanning logic: fa4e83f