Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

automod consuming ozone events #698

Merged
merged 8 commits into from
Jul 12, 2024
Merged

automod consuming ozone events #698

merged 8 commits into from
Jul 12, 2024

Conversation

bnewbold
Copy link
Collaborator

@bnewbold bnewbold commented Jul 3, 2024

This has automod consume events from Ozone and feed them in to the rules engine, similar to firehose events. It uses the queryEvents endpoint, polling every 5 seconds. This is all optional; if an ozone admin client isn't configured, it doesn't happen.

  • some types of ozone events on an account subject result in account metadata getting flushed. this ensures that, eg, manually-added labels or takedowns end up in automod-visible cached account metadata
  • harassment protection rule is updated to look in private tag state from ozone, not just a deployed config JSON file. this was actually more complex than I would thought, and results in additional account metadata fetches, but those are all cached (with cache purging!) so should be fine
  • private/admin account metadata queries go to ozone, not PDS/entryway, by default. this could increase API load on ozone a bunch; we might need to optimize that code path
  • ignores events caused by mod service automation (eg, automod itself)
  • does not currently have any tests 😞 but I tested a fair bit in staging. would go slow and careful with any prod deploy

Some follow-ups for future work (recording these from my local dev notes):

  • automod "flags" and ozone "tags" are kind of duplicate. I originally thought these would have automod "flags" show up as "tags" in Ozone. but should think that through again (maybe flags are useful outside ozone?) and how to refactor things if they should be the same concept
  • entirely rip out PDS/entryway private metadata queries, once we are confident ozone is sufficient
  • get publicly-available account creation timestamp working consistently. this could include tweaks to appview+ozone endpoints, and possibly even going to PLC directory for more authoritative timestamps
  • should probably refactor account metadata struct in to more clearly distinct "bsky-specific" (from appview), "ozone-specific" (private metadata), and public account/identity info
  • should probably handle account metadata more consistently across ozone, automod, and other systems. related to the #identity and #account refactors, we now have a concept of account status at various pieces of infrastructure (active, takendown, deactivated, suspended, deleted), and should represent that cleanly
  • OzoneEventContext currently extends AccountContext, with the subject account as "the account". currently all ozone events have an associated subject account, while record is optional. we might end up with events that don't have a meaningful subject account? or want to have better ergonomics with a record-variant of ozone event context? those are more complex, feels like things are fine for now
  • would like to refactor the "consumption" code (firehose and ozone events) out of the cmd/hepa/ folder and in to automod/

Update: created a separate issue to track these refactors: #700

@bnewbold bnewbold force-pushed the bnewbold/automod-ozone branch from 6d02588 to 2da8cf8 Compare July 10, 2024 00:53
@bnewbold bnewbold requested review from foysalit and devinivy July 10, 2024 01:04
@bnewbold bnewbold marked this pull request as ready for review July 10, 2024 01:13
Copy link
Contributor

@foysalit foysalit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a lot of new syntax for me here and I'm not much familiar with overall architectural decisions made of course. Having said that, it was easy to follow the changeset and nothing major popped out!

@bnewbold bnewbold merged commit 7d12359 into main Jul 12, 2024
7 checks passed
@bnewbold bnewbold deleted the bnewbold/automod-ozone branch July 12, 2024 23:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants