Initial readthedocs release (#84)

* Write base docs in mkdocs

* Add docs requirements

* Add readthedocs config file

* Fix docs url

* Automatic Rule documentation

* Remove unneded files

* Improve rule formatting

* Linter fix

* Add noqa to * imports handled by __all__

* Linter fixes

* More linter fixes

* Update README.md

* Update README.md

.readthedocs.yml

@@ -0,0 +1,19 @@
+# .readthedocs.yml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Build documentation with MkDocs
+mkdocs:
+  configuration: mkdocs.yml
+
+# Optionally build your docs in additional formats such as PDF and ePub
+formats: all
+
+# Optionally set the version of Python and requirements required to build your docs
+python:
+  version: 3.7
+  install:
+    - requirements: docs/requirements.txt

README.md

@@ -1,57 +1,10 @@
-<img src="docs/images/logo.png" width="200">
+<img src="docs/img/logo.png" width="200">
 
 # CFripper
 
 [![Build Status](https://travis-ci.org/Skyscanner/cfripper.svg?branch=master)](https://travis-ci.org/Skyscanner/cfripper)
 [![PyPI version](https://badge.fury.io/py/cfripper.svg)](https://badge.fury.io/py/cfripper)
 
-Lambda function to "rip apart" a CloudFormation template and check it for security compliance.
+Library designed to be used as part of a Lambda function to "rip apart" a CloudFormation template and check it for security compliance.
 
-## Sample pipeline with CFripper
-
-CFripper is a Python tool that aims to prevent vulnerabilities from getting to production infrastructure through vulnerable CloudFormation scripts. As with the other security tools that we use at Skyscanner, CFripper is part of the CI/CD pipeline. It runs just before a CloudFormation stack is deployed or updated and if the CloudFormation script fails to pass the security check it fails the deployment and notifies the team that owns the stack. This is an example of how you might set up CFripper as an AWS Lambda:
-![CFripperPipeline](docs/images/cfripper.png)
-
-Another approach that we use at Skyscanner is the Infrastructure as Code pipeline. Code is built and tested using drone and then our internal CD tool deals with calling CFripper to validate the script and then trigger the deployment of the infrastructure provided that the CloudFormation script is valid:
-![CFripperPipeline](docs/images/cfripper2.png)
-
-## Lambda Installation
-
-To install the lambda first generate the zip package using `make lambda.zip`.
-The runtime of the environment should be `Python 3.7` and the handler `cfripper.main.handler`.
-
-## Developing
-
-The project comes with a set of commands you can use to run common operations:
-
-- `make install`: Installs run time dependencies.
-- `make install-dev`: Installs dev dependencies together with run time dependencies.
-- `make freeze`: Freezes dependencies from `setup.py` to `requirements.txt` (including transitive ones).
-- `make lint`: Runs static analysis.
-- `make coverage`: Runs all tests collecting coverage.
-- `make test`: Runs `lint` and `component`.
-
-
-## Running the simulator
-
-To run the simulator make sure you have the dependencies installed using `make install-dev` and run `python simulator/simulator.py`
-You can add more scripts to the test set in `simulator/test_cf_scripts`.
-Be sure to also add them in the `scripts` dictionary with their name, service name and project so that the simulator can pick them up.
-
-## Custom Rules
-
-To add custom rules first extend the [Rule](cfripper/rule_processor.py) class. Then implement the `invoke` method by adding your logic.
-
-CFripper uses [pycfmodel](https://github.com/Skyscanner/pycfmodel) to create a Python model of the CloudFormation script. This model is passed to the `invoke` function as the `resources` parameter. You can use the model's iterate through the resources and other objects of the model and use the helper functions to perform various checks. Look at the [current rules](cfripper/rules) for examples.
-
-![CFripperRule](docs/images/rule.png)
-
-## Monitor Mode
-By default, each rule has `MONITOR_MODE` set to false. Monitor model will return the failed rules in another field in the response, instead in the main "failed rules". This way new rules can be tested before they are removed from monitor mode and start triggering alarms.
-
-## Contributing
-
-See [CONTRIBUTING.md](CONTRIBUTING.md) file to add a contribution.
-
-## Attribution
-Some of our rules were inspired by [cfn-nag](https://github.com/stelligent/cfn_nag). We also use their example scripts in our test cases.
+Docs available in https://cfripper.readthedocs.io/

cfripper/rules/__init__.py

@@ -12,44 +12,41 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
-from cfripper.rules.cross_account_trust import S3CrossAccountTrustRule
-from cfripper.rules.s3_public_access import S3BucketPublicReadWriteAclRule
-from cfripper.rules.security_group import SecurityGroupOpenToWorldRule
-from cfripper.rules.sqs_queue_policy import SQSQueuePolicyPublicRule
-
-from .cross_account_trust import CrossAccountTrustRule
-from .ebs_volume_has_sse import EBSVolumeHasSSERule
-from .hardcoded_RDS_password import HardcodedRDSPasswordRule
-from .iam_roles import IAMRolesOverprivilegedRule
-from .kms_key_wildcard_principal import KMSKeyWildcardPrincipal
-from .managed_policy_on_user import ManagedPolicyOnUserRule
-from .policy_on_user import PolicyOnUserRule
-from .privilege_escalation import PrivilegeEscalationRule
-from .s3_bucked_policy import S3BucketPolicyPrincipalRule
-from .s3_public_access import S3BucketPublicReadAclAndListStatementRule
-from .security_group import SecurityGroupIngressOpenToWorld
-from .sns_topic_policy_not_principal import SNSTopicPolicyNotPrincipalRule
-from .sqs_queue_policy import SQSQueuePolicyNotPrincipalRule
-from .wildcard_principals import FullWildcardPrincipalRule, PartialWildcardPrincipalRule
+from cfripper.rules.cloudformation_authentication import *  # noqa: F403
+from cfripper.rules.cross_account_trust import *  # noqa: F403
+from cfripper.rules.ebs_volume_has_sse import *  # noqa: F403
+from cfripper.rules.hardcoded_RDS_password import *  # noqa: F403
+from cfripper.rules.iam_managed_policy_wildcard_action import *  # noqa: F403
+from cfripper.rules.iam_roles import *  # noqa: F403
+from cfripper.rules.kms_key_wildcard_principal import *  # noqa: F403
+from cfripper.rules.managed_policy_on_user import *  # noqa: F403
+from cfripper.rules.policy_on_user import *  # noqa: F403
+from cfripper.rules.privilege_escalation import *  # noqa: F403
+from cfripper.rules.s3_bucked_policy import *  # noqa: F403
+from cfripper.rules.s3_public_access import *  # noqa: F403
+from cfripper.rules.security_group import *  # noqa: F403
+from cfripper.rules.sns_topic_policy_not_principal import *  # noqa: F403
+from cfripper.rules.sqs_queue_policy import *  # noqa: F403
+from cfripper.rules.wildcard_principals import *  # noqa: F403
 
 DEFAULT_RULES = {
-    "IAMRolesOverprivilegedRule": IAMRolesOverprivilegedRule,
-    "SecurityGroupOpenToWorldRule": SecurityGroupOpenToWorldRule,
-    "S3BucketPublicReadWriteAclRule": S3BucketPublicReadWriteAclRule,
-    "SecurityGroupIngressOpenToWorld": SecurityGroupIngressOpenToWorld,
-    "ManagedPolicyOnUserRule": ManagedPolicyOnUserRule,
-    "PolicyOnUserRule": PolicyOnUserRule,
-    "SNSTopicPolicyNotPrincipalRule": SNSTopicPolicyNotPrincipalRule,
-    "SQSQueuePolicyNotPrincipalRule": SQSQueuePolicyNotPrincipalRule,
-    "S3BucketPolicyPrincipalRule": S3BucketPolicyPrincipalRule,
-    "EBSVolumeHasSSERule": EBSVolumeHasSSERule,
-    "PrivilegeEscalationRule": PrivilegeEscalationRule,
-    "CrossAccountTrustRule": CrossAccountTrustRule,
-    "S3BucketPublicReadAclAndListStatementRule": S3BucketPublicReadAclAndListStatementRule,
-    "SQSQueuePolicyPublicRule": SQSQueuePolicyPublicRule,
-    "S3CrossAccountTrustRule": S3CrossAccountTrustRule,
-    "HardcodedRDSPasswordRule": HardcodedRDSPasswordRule,
-    "KMSKeyWildcardPrincipal": KMSKeyWildcardPrincipal,
-    "FullWildcardPrincipal": FullWildcardPrincipalRule,
-    "PartialWildcardPrincipal": PartialWildcardPrincipalRule,
+    "IAMRolesOverprivilegedRule": IAMRolesOverprivilegedRule,  # noqa: F405
+    "SecurityGroupOpenToWorldRule": SecurityGroupOpenToWorldRule,  # noqa: F405
+    "S3BucketPublicReadWriteAclRule": S3BucketPublicReadWriteAclRule,  # noqa: F405
+    "SecurityGroupIngressOpenToWorld": SecurityGroupIngressOpenToWorld,  # noqa: F405
+    "ManagedPolicyOnUserRule": ManagedPolicyOnUserRule,  # noqa: F405
+    "PolicyOnUserRule": PolicyOnUserRule,  # noqa: F405
+    "SNSTopicPolicyNotPrincipalRule": SNSTopicPolicyNotPrincipalRule,  # noqa: F405
+    "SQSQueuePolicyNotPrincipalRule": SQSQueuePolicyNotPrincipalRule,  # noqa: F405
+    "S3BucketPolicyPrincipalRule": S3BucketPolicyPrincipalRule,  # noqa: F405
+    "EBSVolumeHasSSERule": EBSVolumeHasSSERule,  # noqa: F405
+    "PrivilegeEscalationRule": PrivilegeEscalationRule,  # noqa: F405
+    "CrossAccountTrustRule": CrossAccountTrustRule,  # noqa: F405
+    "S3BucketPublicReadAclAndListStatementRule": S3BucketPublicReadAclAndListStatementRule,  # noqa: F405
+    "SQSQueuePolicyPublicRule": SQSQueuePolicyPublicRule,  # noqa: F405
+    "S3CrossAccountTrustRule": S3CrossAccountTrustRule,  # noqa: F405
+    "HardcodedRDSPasswordRule": HardcodedRDSPasswordRule,  # noqa: F405
+    "KMSKeyWildcardPrincipal": KMSKeyWildcardPrincipal,  # noqa: F405
+    "FullWildcardPrincipal": FullWildcardPrincipalRule,  # noqa: F405
+    "PartialWildcardPrincipal": PartialWildcardPrincipalRule,  # noqa: F405
 }

cfripper/rules/cloudformation_authentication.py

@@ -12,10 +12,12 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["CloudFormationAuthenticationRule"]
 from ..model.rule import Rule
 
 
 class CloudFormationAuthenticationRule(Rule):
+    """This rule checks for hardcoded credentials"""
 
     REASON = "Hardcoded credentials in {}"
 

cfripper/rules/cross_account_trust.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["CrossAccountCheckingRule", "CrossAccountTrustRule", "S3CrossAccountTrustRule"]
 import logging
 import re
 from typing import Set

cfripper/rules/ebs_volume_has_sse.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["EBSVolumeHasSSERule"]
 from ..model.enums import RuleMode
 from ..model.rule import Rule
 

cfripper/rules/hardcoded_RDS_password.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["HardcodedRDSPasswordRule"]
 from pycfmodel.model.parameter import Parameter
 
 from ..model.rule import Rule

cfripper/rules/iam_managed_policy_wildcard_action.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["IAMManagedPolicyWildcardActionRule"]
 from pycfmodel.model.resources.iam_managed_policy import IAMManagedPolicy
 
 from ..config.regex import REGEX_WILDCARD_POLICY_ACTION

cfripper/rules/iam_roles.py

@@ -12,6 +12,11 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = [
+    "IAMRolesOverprivilegedRule",
+    "IAMRoleWildcardActionOnPermissionsPolicyRule",
+    "IAMRoleWildcardActionOnTrustPolicyRule",
+]
 from pycfmodel.model.resources.iam_role import IAMRole
 
 from ..config.regex import REGEX_IS_STAR, REGEX_WILDCARD_POLICY_ACTION

cfripper/rules/kms_key_wildcard_principal.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["KMSKeyWildcardPrincipal"]
 import logging
 import re
 

cfripper/rules/managed_policy_on_user.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["ManagedPolicyOnUserRule"]
 from pycfmodel.model.cf_model import CFModel
 from pycfmodel.model.resources.iam_managed_policy import IAMManagedPolicy
 

cfripper/rules/policy_on_user.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["PolicyOnUserRule"]
 from pycfmodel.model.cf_model import CFModel
 from pycfmodel.model.resources.iam_policy import IAMPolicy
 

cfripper/rules/privilege_escalation.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["PrivilegeEscalationRule"]
 from pycfmodel.model.resources.iam_policy import IAMPolicy
 
 from ..model.rule import Rule

cfripper/rules/s3_bucked_policy.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["S3BucketPolicyPrincipalRule", "S3BucketPolicyWildcardActionRule"]
 import logging
 import re
 

cfripper/rules/s3_public_access.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["S3BucketPublicReadAclAndListStatementRule", "S3BucketPublicReadWriteAclRule"]
 import logging
 import re
 

cfripper/rules/security_group.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["SecurityGroupOpenToWorldRule", "SecurityGroupIngressOpenToWorld", "SecurityGroupMissingEgressRule"]
 from pycfmodel.model.resources.security_group import SecurityGroup
 from pycfmodel.model.resources.security_group_ingress import SecurityGroupIngress
 

cfripper/rules/sns_topic_policy_not_principal.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["SNSTopicPolicyNotPrincipalRule"]
 from pycfmodel.model.resources.sns_topic_policy import SNSTopicPolicy
 
 from ..model.enums import RuleMode

cfripper/rules/sqs_queue_policy.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["SQSQueuePolicyNotPrincipalRule", "SQSQueuePolicyPublicRule", "SQSQueuePolicyWildcardActionRule"]
 import logging
 import re
 

cfripper/rules/wildcard_principals.py

@@ -12,6 +12,7 @@
 CONDITIONS OF ANY KIND, either express or implied. See the License for the
 specific language governing permissions and limitations under the License.
 """
+__all__ = ["GenericWildcardPrincipalRule", "PartialWildcardPrincipalRule", "FullWildcardPrincipalRule"]
 import logging
 import re
 

docs/code_of_conduct.md

@@ -0,0 +1 @@
+../CODE_OF_CONDUCT.md

docs/contributing.md

@@ -0,0 +1 @@
+../CONTRIBUTING.md

docs/index.md

@@ -0,0 +1,47 @@
+<img src="img/logo.png" width="200">
+
+# CFripper
+
+[![Build Status](https://travis-ci.org/Skyscanner/cfripper.svg?branch=master)](https://travis-ci.org/Skyscanner/cfripper)
+[![PyPI version](https://badge.fury.io/py/cfripper.svg)](https://badge.fury.io/py/cfripper)
+
+Lambda function to "rip apart" a CloudFormation template and check it for security compliance.
+
+## Sample pipeline with CFripper
+
+CFripper is a Python tool that aims to prevent vulnerabilities from getting to production infrastructure through vulnerable CloudFormation scripts. As with the other security tools that we use at Skyscanner, CFripper is part of the CI/CD pipeline. It runs just before a CloudFormation stack is deployed or updated and if the CloudFormation script fails to pass the security check it fails the deployment and notifies the team that owns the stack. This is an example of how you might set up CFripper as an AWS Lambda:
+![CFripperPipeline](img/cfripper.png)
+
+Another approach that we use at Skyscanner is the Infrastructure as Code pipeline. Code is built and tested using drone and then our internal CD tool deals with calling CFripper to validate the script and then trigger the deployment of the infrastructure provided that the CloudFormation script is valid:
+![CFripperPipeline](img/cfripper2.png)
+
+## Lambda Installation
+
+To install the lambda first generate the zip package using `make lambda.zip`.
+The runtime of the environment should be `Python 3.7` and the handler `cfripper.main.handler`.
+
+## Developing
+
+The project comes with a set of commands you can use to run common operations:
+
+- `make install`: Installs run time dependencies.
+- `make install-dev`: Installs dev dependencies together with run time dependencies.
+- `make freeze`: Freezes dependencies from `setup.py` to `requirements.txt` (including transitive ones).
+- `make lint`: Runs static analysis.
+- `make coverage`: Runs all tests collecting coverage.
+- `make test`: Runs `lint` and `component`.
+
+
+## Running the simulator
+
+To run the simulator make sure you have the dependencies installed using `make install-dev` and run `python simulator/simulator.py`
+You can add more scripts to the test set in `simulator/test_cf_scripts`.
+Be sure to also add them in the `scripts` dictionary with their name, service name and project so that the simulator can pick them up.
+
+
+## Contributing
+
+See [CONTRIBUTING.md](https://github.com/Skyscanner/cfripper/blob/master/CONTRIBUTING.md) file to add a contribution.
+
+## Attribution
+Some of our rules were inspired by [cfn-nag](https://github.com/stelligent/cfn_nag). We also use their example scripts in our test cases.

docs/macros.py

@@ -0,0 +1,18 @@
+import inspect
+
+from cfripper import rules
+
+
+def define_env(env):
+    @env.macro
+    def cfripper_rules():
+        rules_inspection = inspect.getmembers(rules, inspect.isclass)
+        results = []
+        for _, klass in rules_inspection:
+            doc = inspect.getdoc(klass)
+            # Remove ABCMeta default docstring
+            if not doc.startswith("Helper class that"):
+                results.append((klass.__name__, doc))
+            else:
+                results.append((klass.__name__, None))
+        return sorted(results)

docs/requirements.txt

@@ -0,0 +1,24 @@
+ansi2html==1.5.2
+click==7.0                # via mkdocs
+htmlmin==0.1.12           # via mkdocs-minify-plugin
+jinja2==2.10.3            # via mkdocs, mkdocs-macros-plugin
+jsmin==2.2.2              # via mkdocs-minify-plugin
+livereload==2.6.1         # via mkdocs
+markdown-include==0.5.1
+markdown==3.1.1           # via markdown-include, mkdocs, mkdocs-material, pymdown-extensions
+markupsafe==1.1.1         # via jinja2
+mkdocs-exclude==1.0.2
+mkdocs-macros-plugin==0.3.2
+mkdocs-material==4.5.1
+mkdocs-minify-plugin==0.2.1  # via mkdocs-material
+mkdocs==1.0.4
+pep562==1.0               # via pymdown-extensions
+pygments==2.5.2
+pymdown-extensions==6.2   # via mkdocs-material
+pyyaml==5.2               # via mkdocs, mkdocs-macros-plugin
+repackage==0.7.3          # via mkdocs-macros-plugin
+six==1.13.0               # via ansi2html, livereload
+termcolor==1.1.0          # via mkdocs-macros-plugin
+tornado==6.0.3            # via livereload, mkdocs
+
+.                         # CFRipper in the current version