Release v1.70.0

1.70.0 - 2024-04-24

Added

• Added guidance for resolving API token issues in CI environments. (gh-10133)

• The osemgrep show command supports 2 new options: dump-ast dump-pattern.
  See osemgrep show --help for more information. (osemgrep_show)

• Added additional output flags which allow you to write output to multiple files in multiple formats.

  For example, the comand semgrep ci --text --json-output=result.json --sarif-output=result.sarif.json
  Displays text output on stdout, writes the output that would be generated by passing the --json flag
  to result.json, and writes the output that would be generated by passing the --sarif to result.sarif.json. (saf-341)

• Added an experimental feature for users to use osemgrep to format
  SARIF output.

  When both the flags --sarif and --use-osemgrep-sarif are specified,
  semgrep will use the ocaml implementation to format SARIF.

  This flag is experimental and can be removed any time. Users must not
  rely on it being available. (saf-978)

Changed

• The main regex engine is now PCRE2 (was PCRE). While the syntax is mostly
  compatible, there are some minor instances where updates to rules may be
  needed, since PCRE2 is slightly more strict in some cases. For example, while
  we previously accepted [\w-.], such a pattern would now need to be written
  [\w.-] or [\w\-.] since PCRE2 rejects the first as having an invalid range. (scrt-467)

Fixed

• Semgrep LS now waits longer for users to login (gh-10109)

• When semgrep ci finishes scanning and uploads findings, it tells the
  app to mark the scan as completed.

  For large findings, this may take a while and marking the scan as
  completed may timeout. When a scan is not marked as completed, the app
  may show that the repo is still processing, and confuses the user.

  This change increases the timeout (previously 20 minutes) to 30
  minutes. (saf-980)

• Fix semgrep ci --oss-only when secrets product is enabled. (scrt-223)

Release v1.69.0

1.69.0 - 2024-04-16

Added

• Tracing: remove support for SEMGREP_OTEL_ENDPOINT and replace with --trace-endpoint <url>.
  This change is for an internal feature for debugging performance. (saf-885)

Changed

• Passing --debug to Semgrep will not print much, unless a set of tags is specified
  via LOG_TAGS. You can get all debug logs with LOG_TAGS=everything. We do not
  want --debug's output to be enourmous, as it tends not to be useful and yet cause
  some problems. Note that --debug is mainly intended for Semgrep developers, please
  ask for help if needed. (gh-10044)
• • The environment variables used to select the debug-level log messages
    are now prefixed with SEMGREP_ (or PYTEST_SEMGREP_) to avoid namespace
    pollution and undesired cross-application side effects.
    The supported environment variables are now SEMGREP_LOG_TAGS
    and PYTEST_SEMGREP_LOG_TAGS. (gh-10087)
• The implicit tag to show all debug-level log messages changes from
  everything to all. All debug-level messages shown by default are
  now tagged and selectable with a default tag. (gh-10089)

Fixed

• In generic mode (default, spacegrep engine), matching a pattern that
  ends with an ellipsis now favors the longest match rather than the shortest
  match when multiple matches are possible. For example, for a given target
  program a a b, the pattern a ... b will match a b as before but
  the pattern a ... will now match the longer a a b rather than a b. (gh-10039)
• Fixed the inter-file diff scan issue where the removal of pre-existing findings
  didn't work properly when adding a new file or renaming an existing file. (saf-897)

Release v1.68.0

1.68.0 - 2024-04-08

Added

• Scan un-changed lockfiles in diff-aware scans (gh-9899)
• Languages: Added the QL language (used by CodeQL) to Semgrep (saf-947)
• SwiftPM parser will now report package url and reference. (sc-1218)
• Add support for Elixir (Mix) SCA parsing for pro engine users. (sc-1303)

Fixed

• Output for sarif format includes dataflow traces. (gh-10004)
• The environment variable LOG_LEVEL (as well as PYTEST_LOG_LEVEL) is
  no longer consulted by Semgrep to determine the log level. Only
  SEMGREP_LOG_LEVEL is consulted. PYTEST_SEMGREP_LOG_LEVEL is also
  consulted in the current implementation but should not be used outside of
  Semgrep's Pytest tests. This is to avoid accidentally affecting Semgrep
  when inheriting the LOG_LEVEL destined to another application. (gh-10044)
• Fixed swiftpm parser to no longer limit the amount of found packages in manifest file. (sc-1364)
• Fixed incorrect ecosystem being used for Elixir. Hex should be used instead of Mix. (sc-elixir)
• Fixed the match_based_ids of lockfile-only findings to differentiate between findings in cases where one rule produces multiple findings in one lockfile (sca-mid)
• Secrets historical scans: fixed a bug where historical scans could run on differential scans. (scrt-545)

Release v1.67.0

1.67.0 - 2024-03-28

Added

• --historical-secrets flag for running Semgrep Secrets regex rules on git
  history (requires Semgrep Secrets). This flag is not yet implemented for
  --experimental. (scrt-531)

Changed

• Files with the .phtml extension are now treated as PHP files. (gh-10009)

• [IMPORTANT] Logged in users running semgrep ci will now run the pro engine by default! All semgrep ci scans will run with our proprietary languages (Apex and Elixir), as well as cross-function taint within a single file, and other single file pro optimizations we have developed. This is equivalent to semgrep ci --pro-intrafile. Users will likely see improved results if they are running semgrep ci and did not already have additional configuration to enable pro analysis.

  The current default engine does not include cross-file analysis. To scan with cross-file analysis, turn on the app toggle or pass in the flag --pro. We recommend this unless you have very large repos (talk to our support to get help enabling cross-file analysis on monorepos!)

  To revert back to our OSS analysis, pass the flag --oss-only (or use --pro-languages to continue to receive our proprietary languages).

  Reminder: because we release first to our canary image, this change will only immediately affect you if you are using semgrep/semgrep:canary. If you are using semgrep/semgrep:latest, it will affect you when we bump canary to latest. (saf-845)

Fixed

• Fixed a parsing error in Kotlin when there's a newline between the class name and the primary constructor.

  This could not parse before

  class C
  constructor(arg:Int){}
  

  because of the newline between the class name and the constructor.

  Now it's fixed. (saf-899)

Release v1.66.2

1.66.2 - 2024-03-26

Added

• osemgrep now respects HTTP_PROXY and HTTPS_PROXY when making network requests (cdx-253)

Changed

• [IMPORTANT] The public rollout of inter-file differential scanning has been
  temporarily reverted for further polishing of the feature. We will reintroduce
  it in a later version. (saf-268)

Fixed

• Autofix on variable definitions should now handle the semicolon
  in Java, C++, and C#. (saf-928)

Release v1.66.1

1.66.1 - 2024-03-25

Fixed

• Autofix on variable definitions should now handle the semicolon
  in Rust, Cairo, Solidity, Dart. (autofix_vardef)
• [IMPORTANT] we restored bash, jq, and curl in our semgrep docker image as some
  users were relying on it. We might remove them in the futur but in the
  mean time we restored the packages and if we remove them we will announce
  it more loudly. We also created a new page giving more information
  about our policy for our docker images:
  https://semgrep.dev/docs/semgrep-ci/packages-in-semgrep-docker/ (docker_bash)
• Fixed autofix application on lines containing multi-byte characters. (multibyte)

Release v1.66.0

1.66.0 - 2024-03-19

Added

• Added information about interfile pre-processing to --max-memory help. (gh-9932)
• We've implemented basic support for the yield keyword in Python. The Pro
  engine now detects taint findings from taint sources returned by the yield
  keyword. (saf-281)

Changed

• osemgrep --remote will no longer clone into a tmp folder, but instead the CWD (cdx-remote)

• [IMPORTANT] Inter-file differential scanning is now enabled for all Pro users.

  Inter-file differential scanning is now enabled for all Pro users. While it may
  take longer than intra-file differential scanning, which is the current default
  for pro users, it offers deeper analysis of dataflow paths compared to
  intra-file differential scanning. Additionally, it is significantly faster
  than non-differential inter-file scanning, with scan times reduced to
  approximately 1/10 of the non-differential inter-file scan. Users who
  enable the pro engine and engage in differential PR scans on GitHub or
  GitLab may experience the impact of this update. If needed, users can
  revert to the previous intra-file differential scan behavior by configuring
  the --no-interfile-diff-scan command-line option. (saf-268)

Fixed

• The official semgrep docker image does not contain anymore the
  bash, jq, and curl utilities, to reduce its attack surface. (saf-861)

Release v1.65.0

1.65.0 - 2024-03-11

Changed

• Removed the extract-mode rules experimental feature. (extract_mode)

Release v1.64.0

1.64.0 - 2024-03-07

Changed

• Removed the AST caching experimental feature (--experimental --ast-caching
  in osemgrep and -parsing_cache_dir in semgrep-core). (ast_caching)
• Removed the Registry caching experimental feature (--experimental --registry-caching)
  in osemgrep. (registry_caching)

Fixed

• Clean any credentials from project URL before using it, to prevent leakage. (saf-876)
• ci: Updated logic for informational message printed when no rules are sent to
  correctly display when secrets is enabled (in additional to
  when code is). (scrt-455)

Release v1.63.0

1.63.0 - 2024-02-27

Added

• Dataflow: Added support for nested record patterns such as { body: { param } }
  in the LHS of an assignment. Now given { body: { param } } = tainted Semgrep
  will correctly mark param as tainted. (flow-68)
• Matching: metavariable-regex can now match on metavariables of interpolated
  strings which use variables that have known values. (saf-865)
• Add support for parsing Swift Package Manager manifest and lockfiles (sc-1217)

Fixed

• fix: taint signatures do not capture changes to parameters' fields (flow-70)
• Scan summary links printed after semgrep ci scans now reflect a custom SEMGREP_APP_URL, if one is set. (saf-353)