Failed to parse Terraform plan JSON file with YAML error

[jwicks]

I have a strange error when using this action to scan a Terraform plan output file in JSON format:

Run fugue/regula-action@v1.6.0
  with:
    input_path: infrastructure/terraform.plan.json
    input_type: tf-plan
    rego_paths: example_custom_rule
    user_only: false
    severity: unknown
...
level=fatal msg="Failed to parse JSON file infrastructure/terraform.plan.json: yaml: line 4: mapping values are not allowed in this context"

Here is my workflow step config:

      # Run PaC scan against terraform plan json so that interpolated values are utilized
      - uses: fugue/regula-action@v1.6.0
        with:
          input_path: infrastructure/terraform.plan.json
          input_type: tf-plan
          rego_paths: example_custom_rule

I've debugged and made sure the infrastructure/terraform.plan.json file is there. It starts with {"format_version":"0.2","terraform_version":"1.0.5",...

I've run regula via docker from my workstation and confirmed it works:

docker run --rm -t -v $(pwd):/workspace \
		fugue/regula:v1.6.0 run \
		-f json \
		-i example_custom_rule \
		-t tf-plan \
		infrastructure/terraform.plan.json
{
 ...
  "summary": {
    "filepaths": [
      "infrastructure/terraform.plan.json"
    ],
    "rule_results": {
      "FAIL": 0,
      "PASS": 5,
      "WAIVED": 0
    },
    ...
  }
}

Also strangely, when I run the above docker run... approach from a GitHub Action shell command, I get the same error as using the github-action:

level=fatal msg="Failed to parse JSON file infrastructure/terraform.plan.json: yaml: line 4: mapping values are not allowed in this context"

Is this an issue with the regula-action or regula itself? Why is regula attempting to parse my TF plan JSON file using YAML? I'm assuming it's executing this line: https://github.com/fugue/regula/blob/9d6ddbaf2fa6e1bfe8b4afb50fbe0548ce5f8013/pkg/loader/tfplan.go#L35

[jason-fugue]

Hi, @jwicks! I was able to reproduce this issue using the Terraform github action. The problem is that the Terraform github action wraps the Terraform binary with a script that does two things:

1. It echoes out the command that it's running
2. After running the command, it echoes out two Github Action workflow commands:

::set-output name=std-err
::set-output name=exit-code

So assuming you're doing something like this:

terraform show -json plan.tfplan > terraform.plan.json

Your JSON file ends up with a few extra lines. It's definitely not ideal, but two potential ways that you can work around this is to either use the Terraform binary directly or strip out those lines, like:

terraform show -json plan.tfplan | tail -n +2 | head -n 1 > terraform.plan.json

Why is regula attempting to parse my TF plan JSON file using YAML?

The YAML specification is actually a superset of JSON, so all valid JSON is also valid YAML. The YAML parser that we're using gives us some nice features beyond what's available in JSON libraries that we looked at and it lets us reuse code in a few places.

[jason-fugue]

If you're using the setup-terraform action, there's an option to use the Terraform binary without the wrapper and it seems to fix the issue:

    - uses: hashicorp/setup-terraform@v1
      with:
        terraform_wrapper: false
        terraform_version: 1.0.5

If this resolves your issue, I'll go ahead and add this to the README.

[jwicks]

Nice! Yep using terraform_wrapper: false fixed it for me!! 👍 Thanks!