Add scoring system

[toniblyx]

I think it would be good idea to add a scoring system like what Lynis does, here an example:

================================================================================

  Lynis security scan details:

  Hardening index : 68 [#############       ]
  Tests performed : 205

Something like Hardening index based on the test results would be helpful to have an order of magnitude on our status in just one number (not very exhaustive but helpful). Not sure how to implement it yet though.

[MrSecure]

Would be good if there's a way to model it the same way the CIS-CAT tools scores for other benchmarks.

[toniblyx]

Yes, I have to look at it but my initial thought was to add a % per check because it will depends on the items to review on each check, then each check will give a value from 0 to 100 then the total scoring index would be the average of all checks.

[toniblyx]

Looking at their documentation and a sample report I have just created in my laptop it looks like it works in the same way or pretty similar to how I said.

[pchaganti]

👍

[sidewinder12s]

We've also talked about doing some kind of easy to digest score for accounts, though we ended up classifying each check based on high, medium, low risk.

[toniblyx]

Cool @sidewinder12s! Can you share that classification? I have tried to think the best way to score the checks but I don't know how to implemented and also make it work with any extra/future check, since more checks are coming as usual.

For instance, if you have properly set the password length you get 1 point but if you have 3 out of 10 security groups with SSH open to the internet, is that a -3? I don't know... With the current amount of checks and the different between them, some check just one single configuration point, others search for specific security related misconfiguration in multiple resources, this may be challenging to implement, any idea is welcomed!

[sidewinder12s]

I think the main problem with any kind of classification is that it really depends on the organization, what their doing, what level of risk they are willing to tolerate. That is sort of why we went with 'level' of risk rather than any kind of specific point or grade level, at least right now.

We figured it'd properly note that something might be at high risk of causing a problem if there was a misconfiguration, but you could mitigate many of the challenges if proper attention was paid.

Things like any resources that are public are high risk because its very easy to not properly apply security groups/other controls. Un-rotated access keys were high risk for us because it implied that the user hadn't been used in a while or might have broad permissions. Public S3 buckets are high risk because there is almost no reason to do that in most configurations.

Many of the CloudWatch Logging group items are lower risk for example because they don't lead to exposure of anything. SGs not in use is lower risk because they aren't actually being used

[sidewinder12s]

With that in mind it might be useful to define whatever grading (point, grade, risk) into a file for mapping against the checks that makes it easy for people to edit each level.

[toniblyx]

So far, the new groups implementation makes easier to create you own report or collection of checks that you need for your organization. Scoring is a nice to have but I want to prioritize now GDPR checks for example, or other compliance related checks that may help us and others improve their security.

[toniblyx]

This is now available as a POC in devel branch, option -s

[toniblyx]

added to master branch, since 2.0.1.

[toniblyx]