Log and/or post in chat for FindSpam Rules which take more than a defined amount of time

[makyen]

Log and/or post to chat high elapsed times for individual runs of each detection Rule

This is primarily for debugging and/or monitoring how long individual runs of a Rule (detection) takes. Output is to the console and/or into rooms with the "long-rule-times" role (new role).

The general definitions are:

# (<Log level>, <text prepended to chat message>, <minimum elapsed time>)
('debug', '', 1),  # > 1 s: Log a "debug" level message for the Rule; No chat message
('info', 'High ', 10),  # > 10 s: Log an "info" for the Rule and output to chat as "High "
('warning', '**Very High** ', 30),  # > 30 s: Log a "warning" and output to chat as bold "Very High"

Each Rule can define its own levels for these outputs, including turning them off. The main watchlist tends to be the rule that takes up the most time. It is currently using:

('debug', '', 10),
('info', 'High ', 20),
('warning', '**Very High** ', 45),

Output to chat will not, currently, be enabled.

In order for there to be chat messages posted, there needs to be a room defined in rooms.yml with the "long-rule-times" role. There currently isn't one. If we want this output in chat under normal operation, I recommend we set up a new room on chat.MSE for this output (e.g. "Charcoal Test MSE"). Putting it on chat.MSE will have the least impact on SmokeDetector chat rate limiting. The output provided isn't something which most users can do anything about, so there's not a lot of benefit for having it in Charcoal HQ.

A unique rule_id is now mandatory for each detection Rule

As part of this, each Rule is required to have a unique rule_id, in addition to the reason. The reason, such as "potentially bad keyword in {}" may be used on any number of detection Rule entries, but the rule_id must be unique. This is so any characteristics of the detection can be uniquely matched to the Rule when someone is investigating the output. If the first, and only the first, Rule registered with a particular reason does not have a rule_id, then the reason is used as the rule_id for that Rule.

Example chat messages

You can see the following example output in chat here.

High Rule elapsed time: 20.67 s: potentially bad keyword in {}: main watchlist for stackoverflow.com/q/72441471
High Rule elapsed time: 10.81 s: potentially bad IP for hostname in {}: potentially bad IP for hostname in {} for stackoverflow.com/q/72441597
High Rule elapsed time: 10.81 s: bad IP for hostname in {}: bad IP for hostname in {} for stackoverflow.com/q/72441597
Very High Rule elapsed time: 49.21 s: potentially bad keyword in {}: main watchlist for pt.stackoverflow.com/q/554267

Testing for this was done in the Makyen Test 02 chat room around here.