Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"non-actioned" events #41

Closed
OmgImAlexis opened this issue Oct 21, 2021 · 3 comments
Closed

"non-actioned" events #41

OmgImAlexis opened this issue Oct 21, 2021 · 3 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@OmgImAlexis
Copy link

I'd like to see a page with the results for every check that's run as well as a button to rerun the check on that post/comment.

@FoxxMD
Copy link
Owner

FoxxMD commented Oct 21, 2021

It's a good idea but the blocking issue for this is scaling. For low-volume subreddits this would be a non-issue but CM runs on subreddits which do checks daily across 13k-30k activities. It's not feasible to hold all of those results in memory (the default cache/store configuration). Redis would work but still doesn't scale across multiple days...I've got one subreddit that's done 9.6 million activities in the last month.

Issues I see with this and some off-the-cuff remedies:

Scaling

How to store results in a way that works from 10 to 10k activities per day?

  • user-configurable tailing max limit IE similar to log limit, only retain last X number of activities per subreddit
  • results are already stored as plain objects/arrays/strings that are easily jsonified (works with redis) but could probably be compressed further to save space
  • user-configurable result store trigger IE only store if at least X rules triggered or rule X is triggered or at least X actions triggered...
    • this could get complicated...how to keep it simple?

Finding useful information

At lower volumes it should be easy to find the result you are looking for. But at higher volumes will need to be able to filter results or else it'll be as useless as fast logging for high-volume subreddits

  • Filter by
    • permalink
    • title/content (not sure this is good to store since content could be very long)
    • author
    • activity type (comment/submission)
    • check
    • rule
    • action

Additionally, may be prudent to fold all results to this change. So actioned/non-actioned alike. Then actioned can just be another filter (maybe still store separately so tailing limit doesn't apply?)

Retaining useful information

A user may want to retain a certain result but if the tailing limit is low or subreddit is high volume it could potentially be dropped quickly. Will need a way to store and retrieve user-defined results.

@FoxxMD FoxxMD added enhancement New feature or request help wanted Extra attention is needed labels Oct 21, 2021
@OmgImAlexis
Copy link
Author

I’ll expand more into this but redis. It’ll mean using redis. I’ve dealt with this kind of volume.

The other option would be to allow this to be enabled/disabled so smaller subs or subs needing to debug can use it.

@FoxxMD
Copy link
Owner

FoxxMD commented May 12, 2022

Implemented in 0.11.0 using Recording Options

@FoxxMD FoxxMD closed this as completed May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants