Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elixir mix lockfiles fail to parse certain git definitions #10170

Open
tophersmith opened this issue Apr 26, 2024 · 2 comments
Open

Elixir mix lockfiles fail to parse certain git definitions #10170

tophersmith opened this issue Apr 26, 2024 · 2 comments
Labels
bug Something isn't working lang:elixir parsing Requires a fix in a parser, typically a tree-sitter or menhir grammar. priority:medium

Comments

@tophersmith
Copy link

tophersmith commented Apr 26, 2024

Describe the bug
semgrep's semdep elixir/mix parser has a bug where :git definitions fail when they do not use the :tag checkout option.

See: https://hexdocs.pm/mix/1.14.5/Mix.Tasks.Deps.html#module-git-options-git

This causes a None in the parsed_lockfile list in mix.py here:

parsed_lockfile, parsed_manifest, errors = safe_parse_lockfile_and_manifest(

and causes an unpacking error in the following for loop on

for line_number, (package, version) in lockfile_deps:

I believe that the problem/solution is in the mix parser definition here:

>> string("tag")
where you'll need to support ref and branch

To Reproduce
scan a mix.lock file with a definition like:
"mylib": {:git, "https://github.com/my_org/mylib.git", "1962a9d9a06fe559814724101e05e0a14e55beeb", [ref: "1962a9d9a06fe559814724101e05e0a14e55beeb"]},

Expected behavior
parse the lockfile completely and try to catch errors like this for mis-parsed files as well

Screenshots
N/A

What is the priority of the bug to you?
P1: important to fix or quite annoying

Environment
This is in all builds

Use case
I can parse elixir dependencies

@tophersmith
Copy link
Author

A few related parser issues I'm finding as well:

lockfile can have these definitions for dependencies that aren't supported by this parser:
{:hex, :testing, "1.0.1", "1234bb4db5b32fc0f8aa5c4a2040348b4aa36687100fb8837b850e90cf60e06", [:mix], [], "hexpm"}
(i.e. there's no final comma, quoted_str)

Also, git tag/refs can be empty:
{:git, "https://github.com/okeuday/erlang_term.git", "9eb28c6eb54acab3d491fa69655f9eaf663378da", []},

>> string("tag")

FWIW, I am also getting frequent issues related to trailing newlines in .exs files and I don't understand why.
Failed to parse [bold]discord_voice_curator/deps/observer_cli/mix.exs[/bold] at [bold]17:1[/bold] - expected 'did not
find other parser'
17 |
for this file: https://github.com/zhongwencool/observer_cli/blob/master/mix.exs

@ievans ievans added bug Something isn't working priority:medium parsing Requires a fix in a parser, typically a tree-sitter or menhir grammar. lang:elixir labels Apr 30, 2024
@jarrydlee
Copy link
Contributor

Hey @tophersmith, I'm looking at these issues now. I'll link a PR once it's up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working lang:elixir parsing Requires a fix in a parser, typically a tree-sitter or menhir grammar. priority:medium
Development

No branches or pull requests

3 participants