MatchedRules design can be changed to store matches per rule.

[piyushroshan]

Summary

MatchedRules design can be changed to store matches per rule.
Currently every match, especially multiMatch may add many matches. However getting unique rule matches would be an unnecessary extra task.
Storing the matches per rule and adding all matches mapped together would provide a way to do Anomaly Scoring like Modsec and better visualization of matches. It would also reduce memory use by not deduplicating other fields of MatchedRule.

Basic example

If the MatchedRules structure is changed as follows, this will give interesting observation.

type MatchedRule struct {
	// Full request uri unparsed
	URI string
	// Transaction id
	ID string
	// Is disruptive
	Disruptive bool
	// Server IP address
	ServerIPAddress string
	// Client IP address
	ClientIPAddress string
	// A slice of matched variables
	MatchedDatas []MatchData <--- List of MatchedData
	// A reference to the triggered rule
	Rule *Rule
}

type MatchData struct {
	// variable name stored for cache
	VariableName string
	// Variable
	Variable variables.RuleVariable
	// Key of the variable, blank if no key is required
	Key string
	// Value of the current VARIABLE:KEY
	Value string
	// Macro expanded message
	Message string  <-- Moved here from MatchedRule 
	// Macro expanded logdata
	Data string  <-- Moved here from MatchedRule 
}

Motivation

Better results, handling multi matches on a rules. Anomaly observation.

[jptosso]

Matched Rule is sent to the log callback to generate audit logs, if we do this the user would have to read the matched rule and iterate over the matched data. I think it is possible but we are already far from ModSecurity behavior, they use the log action to instantly write the log, while we mark the rule as "loggable" and in future versions this could change

[piyushroshan]

We have already seen issues of logs being flooded in modsecurity from multimatch. There is no point in sending many log events for one rule match. Its the rule and any of the matched data that is important. The handling could be left to the users if they want to print each one.

[syinwu]

I think this is a good idea, it can reduce the memory usage of MatchedRules. We can keep the format of the log as it is.