Autofix overwrites wrong line with multiple multiline matches (second example)

[finnigja]

Describe the bug
When using autofix to manipulate Terraform configuration files, I found that fixes for multiple issues within the same file would not be written to the correct line locations.

This appears to be similar to an already reported issue - #3388 - but I'm reporting it here because having a separate test case may help with development of a fix.

To Reproduce
https://semgrep.dev/s/chair6:terraform-multiline-autofix-bug

Input file:

resource "aws_instance" "example1" {
  ami           = "ami-005e54dee72cc1d01"
  instance_type = "t2.micro"
}

resource "aws_instance" "example2" {
  ami           = "ami-005e54dee72cc1d02"
  instance_type = "t2.micro"
}

Rule file:

rules:
- id: ec2-instance-metadata-options
  languages:
  - terraform
  message: EC2 instance does not set metadata options
  severity: WARNING
  patterns:
  - pattern-inside: |
      resource "aws_instance" "$RESNAME" {
      ...
      }
  - pattern-not-inside: |
      resource "aws_instance" "..." {
        ...
        metadata_options {
          ...
        }
        ...
      }
  fix-regex:
    regex: (.*)\}
    replacement: |
      \1
        metadata_options {
          http_tokens = "required"
        }
      }

Output without --autofix:

$ semgrep -c ec2-instance-metadata-options.yml main.tf
Running 1 rules...
main.tf
rule:ec2-instance-metadata-options: EC2 instance does not set metadata options
autofix: s/(.*)\}/\1
  metadata_options {
    http_tokens = "required"
  }
}
/g
1:resource "aws_instance" "example1" {
2:  ami           = "ami-005e54dee72cc1d01"
3:  instance_type = "t2.micro"
4:}
--------------------------------------------------------------------------------
autofix: s/(.*)\}/\1
  metadata_options {
    http_tokens = "required"
  }
}
/g
6:resource "aws_instance" "example2" {
7:  ami           = "ami-005e54dee72cc1d02"
8:  instance_type = "t2.micro"
9:}
ran 1 rules on 1 files: 2 findings

Output with --autofix:

$ semgrep --autofix -c ec2-instance-metadata-options.yml main.tf
Running 1 rules...
successfully modified 1 file.
main.tf
rule:ec2-instance-metadata-options: EC2 instance does not set metadata options
autofix: s/(.*)\}/\1
  metadata_options {
    http_tokens = "required"
  }
}
/g
1:resource "aws_instance" "example1" {
2:  ami           = "ami-005e54dee72cc1d01"
3:  instance_type = "t2.micro"
4:}
--------------------------------------------------------------------------------
autofix: s/(.*)\}/\1
  metadata_options {
    http_tokens = "required"
  }
}
/g
6:resource "aws_instance" "example2" {
7:  ami           = "ami-005e54dee72cc1d02"
8:  instance_type = "t2.micro"
9:}
ran 1 rules on 1 files: 2 findings

Input file after modification by --autofix:

resource "aws_instance" "example1" {
  ami           = "ami-005e54dee72cc1d01"
  instance_type = "t2.micro"

  metadata_options {
    http_tokens = "required"

  metadata_options {
    http_tokens = "required"
  }
}


  metadata_options {
    http_tokens = "required"
  }
}

resource "aws_instance" "example2" {
  ami           = "ami-005e54dee72cc1d02"
  instance_type = "t2.micro"
}

I get similar-but-different confused results in the playground linked above.

Expected behavior
Semgrep should have modified the file as notated in the dryrun, but it appears to have gotten confused by the multiple issues and multiple lines.

What is the priority of the bug to you?

• P0: blocking your adoption of Semgrep or workflow
• P1: important to fix or quite annoying
• P2: regular bug that should get fixed

Environment

~ $ semgrep --version
0.76.2

[ievans]

Fixed in 0.77 with tests in #4429, thanks again @chair6!