semgrep-agent from the command-line connects to incorrect dashboard project

[msorens]

Describe the bug

When I attempt to run semgrep-agent on the command-line in the same fashion that I run it in CI, it is not connecting to the right project on the web UI dashboard (and therefore not using the correct policy).
In the figure, my two real projects are highlighted: chef/automate (which exists at https://github.com/chef/automate) and chef/chef-cloud (https://github.com/chef/chef-cloud).

In CI, I use the block of code below (for Buildkite). The relevant portions are highlighted.

The equivalent from the command-line, as I understand it, is this:

$ cd ~/code/go/src/github.com/chef/automate
$ docker run  --rm  \
    --volume $(realpath .):/chef/automate --workdir /chef/automate \
    returntocorp/semgrep-action:v1 \
    python -m semgrep_agent --publish-token $SEMGREP_TOKEN --publish-deployment $SEMGREP_ID --baseline-ref master

Notable:

1. When executed, that generates a new project in the dashboard at the top -- item (1) -- "automate" compared to the real "/chef/automate".
2. If I change the docker command above to use just "automate" (without the /chef parent) so the partial line is this: --volume $(realpath .):/automate --workdir /automate, it still results in updating "automate" in the dashboard -- item (1) again.
3. If I change to --volume $(realpath .):/foo --workdir /foo or --volume $(realpath .):/src --workdir /src, then it creates (or updates) items (2) and (3) respectively.
4. In all cases, running from the command-line is using the default policy, "Getting Started"; note that the real project chef/automate is using policy "Chef-01".

To Reproduce
As above.

Expected behavior
Should be able to connect to the "chef/automate" project and use "Chef-01" policy.

Screenshots
As above.

What is the priority of the bug to you?
Is this a P0 (blocking your adoption of Semgrep or workflow), P1 (important to fix or quite annoying), P2 (regular bug that should get fixed)?
P2 (a bit frustrating, but I can get by without it for a time)

Environment
docker

[mschwager]

Hi @msorens,

It looks like this functionality went in in https://github.com/returntocorp/semgrep-action/pull/120.

I suspect a few things could be going on here, but I'll need a bit more information to confirm:

• Are you running the latest version of returntocorp/semgrep-action? It looks like the v1 tag is updated on every commit: https://github.com/returntocorp/semgrep-action/tags, but you may need a docker pull returntocorp/semgrep-action:v1.
• Can you copy your environment information here? E.g. it should look something like this during a semgrep-action run:
  • It should mention SEMGREP_REPO_NAME if the env variable is correctly set.

=== detecting environment
| versions          - semgrep 0.31.1 on Python 3.7.0
| environment       - running in environment git, triggering event is 'unknown'
| manage            - logged in as deployment returntocorp/semgrep#1
| policy            - using default
| SEMGREP_REPO_NAME - test_name

• Are you correctly passing the SEMGREP_REPO_NAME env variable to the Docker environment?
  • I.e. I wouldn't expect your docker run command to work as expected without something like the following: -e 'SEMGREP_REPO_NAME=test_name'
  • I'm not familiar with BuildKite, but I see you're specifying propogate-environment: true. Is SEMGREP_REPO_NAME included in the environment key of your included image?

I suspect if we get the repo_name right it will use the correct policy.

[msorens]

Your suspicion was 100% correct -- I had neglected to pass environment variables on the command-line. Once I did that, it connects to the right project and uses the correct policy. Thanks!!

[dlukeomalley]

Thanks all! Transferring this issue to semgrep-action for discoverability in the future.

[msorens]

Afterthought: probably should fail the run immediately if SEMGREP_REPO_NAME is not provided to give a clear indication of this easy-to-fall-into trap.

[dlukeomalley]

@chmccreery @underyx Any thoughts on Michael's above suggestion?

[chmccreery]

We intentionally do not require this environment variable, to make it possible to run semgrep-action in a very basic setup, without having to configure a bunch of environment variables. I would like to preserve that simplicity for users who only care about the 80% solution, and just want to run their default policy or get their rules from a local yaml file.

We could certainly print out the environment variables that are not set, as well as those that are, but this might get a bit verbose.