Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] request blocking by http headers #2409

Conversation

NyaaaWhatsUpDoc
Copy link
Member

@NyaaaWhatsUpDoc NyaaaWhatsUpDoc commented Dec 3, 2023

Description

  • adds new HeaderFilter() middleware to filter requests by their header content
  • adds new database methods to get, store, delete and match against request header filters
  • adds new cache types for both allow and block type header filters
  • adds new headerfilter subpkg which handles the actual compiling of in-memory request header filters (header keys to value matching regular expressions) + stat collection
  • adds new admin API endpoints (with matching swagger docs) to manage allow / block header filters
  • adds new "advanced-header-filter-mode" configuration variable which matches the "instance-federation-mode" variable in how it functions re: allow vs. block modes
  • add new /docs/admin/header_filtering_modes.md documentation laying out the logic of the http request filtering modes

Checklist

  • I/we have read the GoToSocial contribution guidelines.
  • I/we have discussed the proposed changes already, either in an issue on the repository, or in the Matrix chat.
  • I/we have performed a self-review of added code.
  • I/we have written code that is legible and maintainable by others.
  • I/we have commented the added code, particularly in hard-to-understand areas.
  • I/we have made any necessary changes to documentation.
  • I/we have added tests that cover new code.
  • I/we have run tests and they pass locally with the changes.
  • I/we have run go fmt ./... and golangci-lint run.

@NyaaaWhatsUpDoc NyaaaWhatsUpDoc force-pushed the feature/request-header-blocking branch 2 times, most recently from aacdc7b to 25714c5 Compare December 12, 2023 13:59
@NyaaaWhatsUpDoc NyaaaWhatsUpDoc marked this pull request as ready for review December 14, 2023 12:32
@tsmethurst tsmethurst added this to the v0.14.0 milestone Dec 16, 2023
example/config.yaml Outdated Show resolved Hide resolved
*regexp.Regexp

// match count.
n atomic.Uint64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can probably be dropped, if we're going to use prometheus/otel metrics for this then we can use a counter for this instead when we get around to it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh I was just going to return this as a counter value, as I'm pretty sure that's what it uses under the hood is atomic ints etc. the plan is to make that stats function export Prometheus compatible data at some point. also this way I have some stat collection enabled required for one of the milestone bulletpoints

Copy link
Contributor

@tsmethurst tsmethurst Dec 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way counters work in OTel is I think to just call Add(i) on the counter when a value must be incremented, and let OTel handle the rest; you don't actually need to keep track in the application of the counter's current value afaik. That's one thing that makes OTel so nice :) I mean we could implement it ourself and serve that raw value, but it seems unnecessary unless there's some specific benefit / bug workaround in doing so, I think.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gotcha. maybe in the tracing / metric package i'll wrap it so it works with our enabling / disabling mechanism (otherwise this will make it a hard dependency in all cases)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so for now i've kept-in an initial statistics tracking implementation, but i have rejigged the header matching so the matched header key and regular expression are returned which should make it easier to instrument with otel in the future. see: eaa29f9

Copy link
Contributor

@tsmethurst tsmethurst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's looking great! Coming along nicely!

@NyaaaWhatsUpDoc NyaaaWhatsUpDoc merged commit 8ebb777 into superseriousbusiness:main Dec 18, 2023
2 checks passed
@NyaaaWhatsUpDoc NyaaaWhatsUpDoc deleted the feature/request-header-blocking branch December 18, 2023 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants