Associate .phtml extension to PHP

[p4p3r]

Sometimes PHP application use the .phtml extension, especially for files which contain a mix of html and php code instead of just php.

In the PHP specs, everything that is not within the <?php and ?> tags is considered unstructured text:

script:
   script-section
   script script-section

script-section:
   text start-tag statement-list end-tag text

start-tag:
   <?php
   <?=

end-tag:
   ?>

text:
   arbitrary text not containing any of start-tag sequences

Our grammar seems to follow this rather closely.

Associating .phtml with PHP basically means:

• we will be able to apply PHP rules to .phtml files, without parsing errors (thanks to the arbitrary text syntactic element)
• we will not parse the arbitrary text as html (which it usually is), we'd need something like extract mode for that, but it's probably a minor issue with smaller impact than having to use generic mode for every .phtml files like today

I did some cursory testing on .phtml files to confirm that they parse fine (using pointer from ) and it seems to be the case.
Here's an example of rule for PHP running on a phtml file: https://semgrep.dev/playground/s/d8EkP

• I ran make setup && make to update the generated code after editing a .atd file (TODO: have a CI check)
• I made sure we're still backward compatible with old versions of the CLI.
  For example, the Semgrep backend need to still be able to consume data generated
  by Semgrep 1.17.0.
  See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades

[github-actions]

Backwards compatibility summary:

Checking backward compatibility of semgrep_output_v1.atd against past version v1.29.0
Skipping v1.30.0 because commit 78720c795cd5a186f5102c87125ef876c6435a0c has already been checked
Skipping v1.31.0 because commit 78720c795cd5a186f5102c87125ef876c6435a0c has already been checked
Skipping v1.31.1 because commit 78720c795cd5a186f5102c87125ef876c6435a0c has already been checked
Skipping v1.31.2 because commit 78720c795cd5a186f5102c87125ef876c6435a0c has already been checked
Skipping v1.32.0 because commit 78720c795cd5a186f5102c87125ef876c6435a0c has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.33.0
Skipping v1.33.1 because commit 8849e56ddb0977e38a120a6cfbd1c396eb6fa15e has already been checked
Skipping v1.33.2 because commit 8849e56ddb0977e38a120a6cfbd1c396eb6fa15e has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.34.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.37.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.38.0
Skipping v1.38.1 because commit fd294683e7369cabf63738febeaba8a22c925187 has already been checked
Skipping v1.38.2 because commit fd294683e7369cabf63738febeaba8a22c925187 has already been checked
Skipping v1.38.3 because commit fd294683e7369cabf63738febeaba8a22c925187 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.39.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.40.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.41.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.42.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.43.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.44.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.45.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.46.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.47.0
Skipping v1.48.0 because commit 278ed753e0c66b8bfc3f2d805fde53be022dd4b6 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.49.0
Skipping v1.50.0 because commit 857682f41eb09e0b330a247ff1adf3bfeaf9d9ca has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.52.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.53.0
Skipping v1.54.0 because commit 3b72d494260258497e796d094b1a4916501a6df1 has already been checked
Skipping v1.54.1 because commit 3b72d494260258497e796d094b1a4916501a6df1 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.54.2
Skipping v1.54.3 because commit 9f1c50383a9a9969e2fe7a5f9bff9ca0a7c837bb has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.55.0
Skipping v1.55.1 because commit 6dffeaa692153fd33b4f154fddaefde1f2f1ae27 has already been checked
Skipping v1.55.2 because commit 6dffeaa692153fd33b4f154fddaefde1f2f1ae27 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.56.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.57.0
Skipping v1.58.0 because commit 4cc11b00d411c02fc611aa8c78a336520438fb48 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.59.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.59.1
Checking backward compatibility of semgrep_output_v1.atd against past version v1.60.0
Skipping v1.60.1 because commit eed58a091fd7d19e402a6d4cf2d287e137215d03 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.61.0
Skipping v1.61.1 because commit bbfd1c5b91bd411bceffc3de73f5f0b37f04433d has already been checked
Skipping v1.62.0 because commit bbfd1c5b91bd411bceffc3de73f5f0b37f04433d has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.63.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.64.0
Checking backward compatibility of semgrep_output_v1.atd against past version v1.65.0
Skipping v1.66.0 because commit 3e7bbafa2b7e722d893303a7fb90a83dab6737a7 has already been checked
Checking backward compatibility of semgrep_output_v1.atd against past version v1.66.1
Skipping v1.66.2 because commit 215a54782174de84f97188632b4a37e35ba0f827 has already been checked

[p4p3r]

@aryx @mjambon, do I need to update the Semgrep repo too?

[aryx]

Yes you need a PR in semgrep that update the cli/src/semgrep/semgrep_interface submodule to point to the latest version,
otherwise we will compile semgrep with the older version of semgrep_interface (which contains the old version of the generated Language.ml, which is linked in semgrep).