Concise custom rules (#1702)

* Allow for users to create custom rules easier using plain text

Co-authored-by: TaeHun Kang <31365193+Birds@users.noreply.github.com>
Co-authored-by: Maria Ines Parnisari <mparnisa@amazon.com>
Co-authored-by: Pat Myron <PatMyron@users.noreply.github.com>

README.md

@@ -140,6 +140,8 @@ ignore_templates:
   - codebuild.yaml
 include_checks:
   - I
+custom_rules:
+  - custom_rules.txt
 ```
 
 ### Parameters
@@ -149,6 +151,7 @@ Optional parameters:
 | Command Line  | Metadata | Options | Description |
 | ------------- | ------------- | ------------- | ------------- |
 | -h, --help  |   | | Get description of cfn-lint |
+| -z, --custom-rules | | filename | Text file containing user-defined custom rules. See [here](#Custom-Rules) for more information |
 | -t, --template  |   | filename | Alternative way to specify Template file path to the file that needs to be tested by cfn-lint |
 | -f, --format    | format | quiet, parseable, json, junit, pretty | Output format |
 | -l, --list-rules | | | List all the rules |
@@ -243,6 +246,46 @@ This collection of rules can be extended with custom rules using the `--append-r
 
 More information describing how rules are set up and an overview of all the Rules that are applied by this linter are documented [here](docs/rules.md).
 
+## Custom Rules
+
+The linter supports the creation of custom one-line rules which compare any resource with a property using pre-defined operators. These custom rules take the following format:
+```
+<Resource Type> <Property[*]> <Operator> <Value> [Error Level] [Custom Error Message]
+```
+
+### Example
+A seperate custom rule text file must be created.
+
+The example below validates `example_template.yml` does not use any EC2 instances of size `m4.16xlarge`
+
+_custom_rule.txt_
+```
+AWS::EC2::Instance InstanceSize NOT_EQUALS "m4.16xlarge" WARN "This is an expensive instance type, don't use it"
+```
+
+_example_template.yml_
+```
+AWSTemplateFormatVersion: "2010-09-09"
+Resources:
+        myInstance:
+                Type: AWS::EC2::Instance
+                Properties:
+                        InstanceType: m4.16xlarge
+                        ImageId: ami-asdfef
+```
+
+The custom rule can be added to the [configuration file](#Config-File) or ran as a [command line argument](#Parameters)
+
+The linter will produce the following output, running `cfn-lint example_template.yml -z custom_rules.txt`:
+
+```
+W9001  This is an expensive instance type, don't use it
+mqtemplate.yml:6:17
+```
+
+
+More information describing how custom rules are setup and an overview of all operators available is documented [here](docs/custom_rules.md).
+
 ## Customize specifications
 
 The linter follows the [AWS CloudFormation Resource Specifications](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-resource-specification.html) by default. However, for your use case specific requirements might exist. For example, within your organisation it might be mandatory to use [Tagging](https://aws.amazon.com/answers/account-management/aws-tagging-strategies/).

custom_rules.txt

@@ -0,0 +1,3 @@
+#Comments can be made with a '#' symbol at the start of the line
+#Syntax: <Resource Type> <Property[*]> <Operator> <Value>
+#Example: AWS::EC2::Instance InstanceType EQUALS "p3.2xlarge"

docs/custom_rules.md

@@ -0,0 +1,76 @@
+# Custom Rules
+The Custom Rules feature allows any customer to create simple rules using a pre-defined set of operators. These rules support any resource property comparisons.
+
+# Syntax
+Each rule has been designed to be easy to write, and thus the syntax is very flexible in regards to grammar. Each rule is always one line long, and follows a set structure in regards to syntax.
+
+The template used for each custom rules have been included below. Angle brackets specify required values, while square brackets indicate optional values.
+
+`<Resource Type> <Property[*]> <Operator> <Value> [Error Level] [Custom Error Message]`
+
+
+### General Guidelines
+
+* The ruleID is auto-generated based on the line number. The E9XXX and W9XXX blocks are allocated towards custom rules.
+    * As an example, custom rule on line 4 of the rules file which has error level “ERROR” would become E9004
+* Comments are supported through the use of the # symbol at the beginning of a line (e.g `#This is a comment`)
+* The syntax is "quote-flexible" and will support all permutations shown below
+```
+AWS::EC2::Instance Property EQUALS "Cloud Formation"
+AWS::EC2::Instance Property EQUALS Cloud Formation
+AWS::EC2::Instance Property EQUALS Cloud Formation WARN "Custom Error"
+AWS::EC2::Instance Property EQUALS Cloud Formation WARN Custom Error
+```
+
+#### Resource Type
+
+Name of the resource type as specified within the template. (e.g `AWS::EC2::Instance`)
+
+
+#### Property
+
+Any property of a resource. Dot notation may be used to specify lower-level properties. (e.g `AssumeRolePolicyDocument.Version`)
+
+#### Operator
+
+The specified operator to be used for this rule. The supported values are defined below.
+
+| Operator          | Function |
+| --------------------- | ------------- |
+| EQUALS | Checks the specified property is equal to the value given |
+| == | Identical to `EQUALS` |
+| NOT_EQUALS | Checks the specified property is not equal to the value given |
+| != | Identical to `NOT_EQUALS` |
+| IN | Checks the specified property is equal to or contained by the array value |
+| NOT_IN | Checks the specified property is not equal to or not contained by the array value |
+| \>= | Checks the specified property is greater than or equal to the value given |
+| <= | Checks the specified property is less than or equal to the value given |
+
+#### Value
+
+The value which the operator is comparing against (e.g `CompareMe`).
+
+Multi-word inputs are accepted  (e.g `Compare Me`). Array inputs are also accepted for set operations (e.g `[Apples, Oranges, Pears]`).
+
+#### Error Level (Optional)
+
+To specify the error level any breach of this rule is categorized. The supported values include all existing error levels (e.g `ERROR` or `WARN`)
+
+#### Custom Error Message (Optional, Pre-requisites)
+
+Pre-Requisites: The Custom Error Message requires an error level to be specified.
+
+A custom error message can be used to override the existing fallback messages. (e.g `Show me this custom message`)
+
+## Example
+This following example shows how a you can create a custom rule.
+
+This rule validates all EC2 instances in a template aren’t using the instance type “p3.2xlarge”.
+
+```
+AWS::EC2::Instance InstanceType != "p3.2xlarge"
+```
+
+To include this rules, include your custom rules text file using the `-z custom_rules.txt` argument when running cfn-lint.
+
+

src/cfnlint/config.py

@@ -19,6 +19,7 @@
 except ImportError:  # pragma: no cover
     from pathlib2 import Path
 
+# pylint: disable=too-many-public-methods
 LOGGER = logging.getLogger('cfnlint')
 
 
@@ -395,6 +396,10 @@ def __call__(self, parser, namespace, values, option_string=None):
         )
         standard.add_argument('--config-file', dest='config_file',
                               help='Specify the cfnlintrc file to use')
+        standard.add_argument(
+            '-z', '--custom-rules', dest='custom_rules',
+            help='Allows specification of a custom rule file.'
+        )
         advanced.add_argument(
             '-o', '--override-spec', dest='override_spec',
             help='A CloudFormation Spec override file that allows customization'
@@ -461,6 +466,9 @@ def set_template_args(self, template):
                     if config_name == 'override_spec':
                         if isinstance(config_value, (six.string_types)):
                             defaults['override_spec'] = config_value
+                    if config_name == 'custom_rules':
+                        if isinstance(config_value, (six.string_types)):
+                            defaults['custom_rules'] = config_value
                     if config_name == 'ignore_bad_template':
                         if isinstance(config_value, bool):
                             defaults['ignore_bad_template'] = config_value
@@ -628,6 +636,11 @@ def append_rules(self):
     def override_spec(self):
         return self._get_argument_value('override_spec', False, True)
 
+    @property
+    def custom_rules(self):
+        """ custom_rules_spec """
+        return self._get_argument_value('custom_rules', False, True)
+
     @property
     def update_specs(self):
         return self._get_argument_value('update_specs', False, False)

src/cfnlint/core.py

@@ -43,6 +43,7 @@ class UnexpectedRuleException(CfnLintExitException):
 def run_cli(filename, template, rules, regions, override_spec, build_graph, registry_schemas, mandatory_rules=None):
     """Process args and run"""
 
+
     if override_spec:
         cfnlint.helpers.override_specs(override_spec)
 
@@ -94,7 +95,8 @@ def get_formatter(fmt):
     return formatter
 
 
-def get_rules(append_rules, ignore_rules, include_rules, configure_rules=None, include_experimental=False, mandatory_rules=None):
+def get_rules(append_rules, ignore_rules, include_rules, configure_rules=None, include_experimental=False,
+              mandatory_rules=None, custom_rules=None):
     rules = RulesCollection(ignore_rules, include_rules, configure_rules,
                             include_experimental, mandatory_rules)
     rules_paths = [DEFAULT_RULESDIR] + append_rules
@@ -104,6 +106,8 @@ def get_rules(append_rules, ignore_rules, include_rules, configure_rules=None, i
                 rules.create_from_directory(rules_path)
             else:
                 rules.create_from_module(rules_path)
+
+        rules.create_from_custom_rules_file(custom_rules)
     except (OSError, ImportError) as e:
         raise UnexpectedRuleException('Tried to append rules but got an error: %s' % str(e), 1)
     return rules
@@ -205,13 +209,14 @@ def get_template_rules(filename, args):
             args.configure_rules,
             args.include_experimental,
             args.mandatory_checks,
+            args.custom_rules,
         )
 
     return(template, __CACHED_RULES, [])
 
 
 def run_checks(filename, template, rules, regions, mandatory_rules=None):
-    """Run Checks against the template"""
+    """Run Checks and Custom Rules against the template"""
     if regions:
         if not set(regions).issubset(set(REGIONS)):
             unsupported_regions = list(set(regions).difference(set(REGIONS)))

src/cfnlint/custom_rules.txt

@@ -0,0 +1,4 @@
+#Comments can be made with a '#' symbol at the start of the line
+#Syntax: <Resource Type> <Property[*]> <Operator> <Value>
+#Example: AWS::EC2::Instance InstanceType EQUALS "p3.2xlarge"
+AWS::EC2::Instance ImageId EQUALS "SomethingItIsNotEqualTo"

src/cfnlint/data/CfnLintCli/config/schema.json

@@ -68,6 +68,13 @@
       "description": "Merges lists between configuration layers",
       "type": "boolean"
     },
+    "custom_rules": {
+      "description": "custom rule file to use",
+      "items": {
+        "type": "string"
+      },
+      "type": "array"
+    },
     "output_file": {
       "description": "Path to the file to write the main output to",
       "type": "string"

src/cfnlint/helpers.py

@@ -422,6 +422,8 @@ def create_rules(mod):
     from the given module."""
     result = []
     for _, clazz in inspect.getmembers(mod, inspect.isclass):
+        if clazz.__name__ == 'CustomRule' and clazz.__module__ == 'cfnlint.rules.custom':
+            continue
         method_resolution = inspect.getmro(clazz)
         if [clz for clz in method_resolution[1:] if clz.__module__ in ('cfnlint', 'cfnlint.rules') and clz.__name__ == 'CloudFormationLintRule']:
             # create and instance of subclasses of CloudFormationLintRule

src/cfnlint/rules/__init__.py

@@ -9,8 +9,10 @@
 import traceback
 import six
 import cfnlint.helpers
+import cfnlint.rules.custom
 from cfnlint.decode.node import TemplateAttributeError
 
+
 LOGGER = logging.getLogger(__name__)
 
 
@@ -420,6 +422,21 @@ def create_from_directory(self, rulesdir):
             result = cfnlint.helpers.load_plugins(os.path.expanduser(rulesdir))
         self.extend(result)
 
+    def create_from_custom_rules_file(self, custom_rules_file):
+        """Create rules from custom rules file """
+        custom_rules = []
+        if custom_rules_file:
+            with open(custom_rules_file) as customRules:
+                line_number = 1
+                for line in customRules:
+                    LOGGER.debug('Processing Custom Rule Line %d', line_number)
+                    custom_rule = cfnlint.rules.custom.make_rule(line, line_number)
+                    if custom_rule:
+                        custom_rules.append(custom_rule)
+                    line_number += 1
+
+        self.extend(custom_rules)
+
 
 class RuleMatch(object):
     """Rules Error"""