Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support CSP reporting for a configurable proportion of page requests #14453

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

stevejalim
Copy link
Collaborator

@stevejalim stevejalim commented Apr 15, 2024

One-line summary

This changeset adds support for including a CSP report-uri in a certain percentage of pages, so that we can get an idea of whether pages have inappropriate CSP set, but without saturating our logging service, Sentry.

NB: the relevant config to enable this on Dev, Stage and Prod will be in a separate PR, and that work needs to be merged to make this changeset do anything.

Issue / Bugzilla link

Resolves #14451

Testing

I've added a script called seek_csp_report_uri.py which we can use to check the proportion of headers being set, either locally or on an actual server.

Here's some testing I've done with it:

With CSP_REPORT_PERCENTAGE not set as an env var

[steve] ~/Code/bedrock $ ./bin/manual_qa/seek_csp_report_uri.py http://localhost:8000/en-US/ 100
Checking http://localhost:8000/en-US/
100 requests made
defaultdict(<class 'int'>, {'report-uri not present': 100})

With CSP_REPORT_PERCENTAGE set to 0.005 (which is what we'd start with in production)

Checking http://localhost:8000/en-US/
1000 requests made
defaultdict(<class 'int'>, {'report-uri not present': 995, 'report-uri present': 5})

On mozorg-demo-8 with CSP_REPORT_PERCENTAGE set to 0.75:

[steve] ~/Code/bedrock $ ./bin/manual_qa/seek_csp_report_uri.py https://www-demo8.allizom.org/en-US/ 100
Checking https://www-demo8.allizom.org/en-US/
100 requests made
defaultdict(<class 'int'>, {'report-uri present': 74, 'report-uri not present': 26})

On pocket-demo-4 with CSP_REPORT_PERCENTAGE not set:

[steve] ~/Code/bedrock $ ./bin/manual_qa/seek_csp_report_uri.py https://www-demo4.tekcopteg.com/en-US/ 100                                       [14451-csp-fractional-reporting]
Checking https://www-demo4.tekcopteg.com/en-US/
100 requests made
defaultdict(<class 'int'>, {'report-uri not present': 100})

If you want to do the same

  • Pull this branch
  • Install kent-server (a fake Sentry from @willkg) with pipx install kent) and run it in it own shell: kent-server run -p 8011
  • in your .env file set:
CSP_REPORT_URI=http://public@127.0.0.1:8011/1/security
CSP_REPORT_PERCENTAGE="0.55"  # 55% of requests get the report-uri in the CSP header
  • Start your runserver in a separate shell
  • In a third shell, run the script to check your localhost's headers 200 times: ./bin/manual_qa/seek_csp_report_uri.py http://localhost:8000/en-US/ 200 and look at the results

@stevejalim
Copy link
Collaborator Author

@pmac @robhudson Do you think this'll have any significant downside (eg performance or CDN-related pain points)? I don't think so, but happy to be corrected

@stevejalim
Copy link
Collaborator Author

@stephaniehobson @alexgibson I'm not sure if you'll be able to see the PR for the extra env var set in the infra repo, so to check here instead:

  • Are you happy for us to initially set 0.5% of all requests to report on any CSP issues? If not, what percentage would you suggest, please?

Copy link

codecov bot commented Apr 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.72%. Comparing base (aff4707) to head (07d083a).
Report is 4 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #14453   +/-   ##
=======================================
  Coverage   75.72%   75.72%           
=======================================
  Files         144      144           
  Lines        7876     7877    +1     
=======================================
+ Hits         5964     5965    +1     
  Misses       1912     1912           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@robhudson robhudson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🕸️ 🚫

except IndexError:
sample_size = 100

check_for_report_uri(url, sample_size)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have click in our requirements already b/c of glean-parser. We could make it a main dependency instead and use that here to avoid some of this boilerplate for arguments.

"CSP_REPORT_PERCENTAGE",
default="0",
parser=float,
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe include the expected range of the value here? 0.0 to 1.0?

@stephaniehobson
Copy link
Contributor

I'm very happy with 0.5%. I think you could go even lower initially if you wanted.

@stevejalim stevejalim changed the title 14451 csp fractional reporting Support CSP reporting for a configurable proportion of page requests Apr 16, 2024
@stevejalim
Copy link
Collaborator Author

Note that to avoid this triggering loads of unnecessary false positives during integration tests (which hit the origin hostnames, not the CDNed hostname) we'll need to update the CSP_DEFAULT_SRC value in our infra config. Handily, #14466 means that extending that one item will now also cover font-src, which is great.

@janbrasna janbrasna mentioned this pull request Apr 18, 2024
1 task
@stevejalim
Copy link
Collaborator Author

@stephaniehobson I've taken it down to 0.2% for initial try-out on Prod

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable CSP violation reporting for a small proportion of visits
3 participants