semgrep not getting detected

[R34LUS3R]

Thanks for keeping it open source.

On macOS, semgrep is installed using brew

But not getting detected by the extension:

there is no error in the import and I have tried by removing and adding it again but still the same.

uname -a output:
Darwin MacBook-Pro 22.3.0 Darwin Kernel Version 22.3.0: Sun Dec 4 18:16:43 PST 2022; root:xnu-8792.80.21.0.2~9/RELEASE_ARM64_T8103 arm64

[gbiagomba]

@gand3lf I dont know if you saw the above comment but i am having the same issue

[gand3lf]

Hi @R34LUS3R and @gbiagomba, thank you for letting me know about this issue. Unfortunatelly, I don't have a macbook available, for this reason this fix requires a little bit more time. In any case, I will try to fix it very soon.

[gbiagomba]

@gand3lf do you need help with fixing the code? If you'd like we can schedule some time and look it over together. On another note, I looked at the code and i find the section of code where the err occurs semgrepper/src/burp/BurpExtender.java:L22-55. I am not a Java dev but my gut tells me the issue might be here. I will keep looking to see if I can find the bug and fix it in the interim.

        ProcessBuilder processBuilder = new ProcessBuilder();
        List<String> cmdParam = new ArrayList<>();
        cmdParam.add("semgrep");
        cmdParam.add("--version");
        processBuilder.command(cmdParam);
        try {
            Process process = processBuilder.start();
            int exitVal = process.waitFor();
            if(exitVal != 0)
                semgrepInstalled = false;
        }catch(Exception e){
            semgrepInstalled = false;
        }

        if(semgrepInstalled) {
            File theDir = new File(System.getProperty("java.io.tmpdir") + "/" + BurpExtender.SEMDIR);
            if (!theDir.exists()) {
                theDir.mkdirs();
            }

            Tab mainTab = new Tab(new Gui(callbacks).rootPanel);
            callbacks.addSuiteTab(mainTab);
        }else{
            JPanel errPanel = new JPanel();
            String msg = "\nIt seems that you don't have Semgrep installed!\n\nPlease, follow these instructions to install it:\n";
            msg += " - Ubuntu, Windows through Windows Subsystem for Linux (WSL), Linux, macOS:\n     python3 -m pip install semgrep\n";
            msg += " - macOS:\n     brew install semgrep";
            JTextArea textArea = new JTextArea(msg);
            textArea.setBorder(null);
            textArea.setEditable(false);
            textArea.setForeground(Color.darkGray);
            errPanel.add(textArea);
            Tab mainTab = new Tab(errPanel);
            callbacks.addSuiteTab(mainTab);
        }

[gbiagomba]

For line 30, would it make more sense to use int exitVal = process.exitValue();? Reading on waitFor, and the issues that might come of it, I am thinking grabbing the exit code would make more sense especially since the process will exit right away upon execution. I tried using gradle to build the code but my set up is not right and i keep getting build err

 *  Executing task: gradle: build 


> Task :compileJava FAILED

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.6/userguide/command_line_interface.html#sec:command_line_warnings
1 actionable task: 1 executed
<-------------> 0% WAITING
> IDLE
warning: $HOME/.gradle/caches/modules-2/files-2.1/net.portswigger.burp.extender/burp-extender-api/2.3/805f2177b7a20961faf4f49cf9ad8c7f0c33242f/burp-extender-api-2.3.jar(burp/IScannerCheck.class): major version 53 is newer than 52, the highest major version supported by this compiler.
  It is recommended that the compiler be upgraded.
warning: $HOME/.gradle/caches/modules-2/files-2.1/net.portswigger.burp.extender/burp-extender-api/2.3/805f2177b7a20961faf4f49cf9ad8c7f0c33242f/burp-extender-api-2.3.jar(burp/IExtensionHelpers.class): major version 53 is newer than 52, the highest major version supported by this compiler.
  It is recommended that the compiler be upgraded.
warning: $HOME/.gradle/caches/modules-2/files-2.1/net.portswigger.burp.extender/burp-extender-api/2.3/805f2177b7a20961faf4f49cf9ad8c7f0c33242f/burp-extender-api-2.3.jar(burp/IBurpExtenderCallbacks.class): major version 53 is newer than 52, the highest major version supported by this compiler.
  It is recommended that the compiler be upgraded.
warning: $HOME/.gradle/caches/modules-2/files-2.1/net.portswigger.burp.extender/burp-extender-api/2.3/805f2177b7a20961faf4f49cf9ad8c7f0c33242f/burp-extender-api-2.3.jar(burp/IHttpRequestResponse.class): major version 53 is newer than 52, the highest major version supported by this compiler.
  It is recommended that the compiler be upgraded.
warning: $HOME/.gradle/caches/modules-2/files-2.1/net.portswigger.burp.extender/burp-extender-api/2.3/805f2177b7a20961faf4f49cf9ad8c7f0c33242f/burp-extender-api-2.3.jar(burp/IScanIssue.class): major version 53 is newer than 52, the highest major version supported by this compiler.
  It is recommended that the compiler be upgraded.
warning: $HOME/.gradle/caches/modules-2/files-2.1/net.portswigger.burp.extender/burp-extender-api/2.3/805f2177b7a20961faf4f49cf9ad8c7f0c33242f/burp-extender-api-2.3.jar(burp/IScannerInsertionPoint.class): major version 53 is newer than 52, the highest major version supported by this compiler.
  It is recommended that the compiler be upgraded.
warning: $HOME/.gradle/caches/modules-2/files-2.1/net.portswigger.burp.extender/burp-extender-api/2.3/805f2177b7a20961faf4f49cf9ad8c7f0c33242f/burp-extender-api-2.3.jar(burp/IHttpService.class): major version 53 is newer than 52, the highest major version supported by this compiler.
  It is recommended that the compiler be upgraded.
warning: $HOME/.gradle/caches/modules-2/files-2.1/net.portswigger.burp.extender/burp-extender-api/2.3/805f2177b7a20961faf4f49cf9ad8c7f0c33242f/burp-extender-api-2.3.jar(burp/IBurpExtender.class): major version 53 is newer than 52, the highest major version supported by this compiler.
  It is recommended that the compiler be upgraded.
warning: $HOME/.gradle/caches/modules-2/files-2.1/net.portswigger.burp.extender/burp-extender-api/2.3/805f2177b7a20961faf4f49cf9ad8c7f0c33242f/burp-extender-api-2.3.jar(burp/IExtensionStateListener.class): major version 53 is newer than 52, the highest major version supported by this compiler.
  It is recommended that the compiler be upgraded.
warning: $HOME/.gradle/caches/modules-2/files-2.1/net.portswigger.burp.extender/burp-extender-api/2.3/805f2177b7a20961faf4f49cf9ad8c7f0c33242f/burp-extender-api-2.3.jar(burp/ITab.class): major version 53 is newer than 52, the highest major version supported by this compiler.
  It is recommended that the compiler be upgraded.
warning: $HOME/.gradle/caches/modules-2/files-2.1/net.portswigger.burp.extender/burp-extender-api/2.3/805f2177b7a20961faf4f49cf9ad8c7f0c33242f/burp-extender-api-2.3.jar(burp/IResponseInfo.class): major version 53 is newer than 52, the highest major version supported by this compiler.
  It is recommended that the compiler be upgraded.
warning: $HOME/.gradle/caches/modules-2/files-2.1/net.portswigger.burp.extender/burp-extender-api/2.3/805f2177b7a20961faf4f49cf9ad8c7f0c33242f/burp-extender-api-2.3.jar(burp/IRequestInfo.class): major version 53 is newer than 52, the highest major version supported by this compiler.
  It is recommended that the compiler be upgraded.
$HOME/Documents/Git/semgrepper/src/burp/Gui.java:194: error: cannot find symbol
                Path filePath = Path.of(selectedPath);
                                    ^
  symbol:   method of(java.lang.String)
  location: interface java.nio.file.Path
$HOME/Documents/Git/semgrepper/src/burp/Gui.java:197: error: cannot find symbol
                    fileContent = Files.readString(filePath);
                                       ^
  symbol:   method readString(java.nio.file.Path)
  location: class java.nio.file.Files
Note: $HOME/Documents/Git/semgrepper/src/burp/Gui.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
2 errors
12 warnings

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org
Could not execute build using connection to Gradle distribution 'https://services.gradle.org/distributions/gradle-7.6-bin.zip'.
 *  The terminal process terminated with exit code: 1. 

[gand3lf]

Semgrepper.zip
Thank you so much @gbiagomba, I have build the project for you.
I've changed the mentioned line from:
int exitVal = process.waitFor();
to:
int exitVal = process.exitValue();

Please let me know if your solution works. In that case, I will accept your pull request.

[gbiagomba]

@gand3lf thank you! & fingers crossed! In the meantime, if you have the jar, could you send it to me so I can test it? I know github actions had a successful biuld, I just dont know how to retrieve that build file haha

[gand3lf]

@gbiagomba You can download the build here: https://github.com/gand3lf/semgrepper/files/10900441/Semgrepper.zip
Unfortunately, it does not work on my configuration.

I have prepared also a version for debugging the issue, can you try this one on your Mac and post the exception message?
Semgrepper-debug.zip

[R34LUS3R]

@gand3lf while import there is still no error with your debug build though the extension tab shows this on the same mac

java.io.IOException: Cannot run program "semgrep": error=2, No such file or directory

though I can confirm its still installed:

[gand3lf]

@R34LUS3R with this debug version you can print the environment variables in use by the target code:
Semgrepper.zip

Could you provide me your PATH variable value? Thx

[R34LUS3R]

USER=waba-laba-dab-dab
COMMAND_MODE=unix2003
__CFBundleIdentifier=com.install4j.7318-9294-3757-1226.70
PATH=/usr/bin:/bin:/usr/sbin:/sbin
LOGNAME=waba-laba-dab-dab
DISPLAY=/private/tmp/com.apple.launchd.JvAKfFeXgD/org.xquartz:0
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.mX5mSD0Wh4/Listeners
HOME=/Users/waba-laba-dab-dab
SHELL=/bin/zsh
TMPDIR=/var/folders/c8/ffvmqkxd0h5dy99w9zz__jmm0000gn/T/
__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
XPC_SERVICE_NAME=application.com.install4j.7318-9294-3757-1226.70.53846856.53846862
XPC_FLAGS=0x0
LANG=en_PK.UTF-8
JAVA_MAIN_CLASS_96491=com.install4j.runtime.launcher.MacLauncher

@gand3lf

[gand3lf]

@R34LUS3R ok, I think that executing this command you will solve the problem:
ln -s ////opt/homebrew/bin/semgrep /usr/bin/

Can you launch it and try the original Semgrepper extension please?

[R34LUS3R]

nope it didn't work, neither the stock version nor any of the other debug one shared (I've checked the paths are linked properly)

I guess this is the issue:

You can't link a file to this path even with sudo!
So probably, if the build can get the path via a system call "which semgrep" or "whereis semgrep" and setting it up as default path might solve the issue

not a dev so might be wrong

[gbiagomba]

@gand3lf is the version you shared of semgrepper have my "fix" or suggestion in it? If yes, this is the err I got

[gbiagomba]

So doing research on it, it seems it doesnt know where the file or program is, I am going to do more digging to see if there is a way to work around that.

• https://stackoverflow.com/questions/54742540/java-caused-by-java-io-ioexception-error-2-no-such-file-or-directory
• https://katalonsupport.force.com/katalonhelpcenter/s/article/How-to-solve-the-error-java-io-IOException-Cannot-run-program-error-2-No-such-file-or-directory-on-MacOS

[gbiagomba]

@R34LUS3R Try doing sudo -EH ln -s /opt/homebrew/bin/semgrep /usr/local/bin/semgrep. Be advised my semgrep install is installed there too and that is still causing the issue lol. But try that, and see if the plugin works as is. If not, try the version @gand3lf shared here #1 (comment) and see if that works too. Let us know either way!

[R34LUS3R]

@gbiagomba in the SS shared in my last comment, when it didn't work with path /usr/bin/ I tried to link it with /usr/local/bin/, which worked fine though semgrepper still didn't fetch it.

cc: @gand3lf

[gbiagomba]

Ohhh Okay, thank you @R34LUS3R , I skimmed over your comment, i should have read it more thoroughly. At least this helps me with my hutch, it seems java doesnt know where to find the executable. There are a couple options we can explore here @gand3lf

A. We can add a conditional which checks what OS the user is running and set the variable to call semgrep full path based on the OS. (e.g., NIX == /usr/local/bin/semgrep || Windows == "Program Files\Semgrep\Semgrep") or something like that.
B. Find another way to execute the program without having to do all that, so that might mean ditching ProcessBuilder all together and using something else.

I have a work project I need to finish today, tomorrow I should (fingers crossed) have some free cycles to do R&D, I will look into other fixes and see what works. Obviously if either one of you and anyone else reading this finds a fix let us know. Lastly, I am open to having a live cross collab, if you are all free tomorrow for an hour or two. I am on the east cost BTW and my calendar is here.

[gbiagomba]

@R34LUS3R if memory serves me rgiht, MacOSX does NOT like user controlled binaries placed in /usr/bin/ but it does not mind /usr/local/bin. I believe the same goes with /usr/share and /usr.local/share. Now on other NIX based OS, I believe using sudo you can force binaries almost anywhere you want lol.

[gand3lf]

Thanks for your help @gbiagomba and @R34LUS3R!
This version should work (I hope):
Semgrepper.zip

It contains the following changes:

• if the system is MacOS then add /opt/homebrew/bin to the current PATH env
• if the current PATH env does not contain /usr/local/bin then add it

Please let me know

[R34LUS3R]

@gand3lf still didn't work
/usr/local/bin/semgrep is linked with /opt/homebrew/bin
both of these path are added as default binary path variables

The fixed version you shared still not able to detect semgrepper on either of these paths :-/

[gand3lf]

@R34LUS3R ok T_T
With this debug version you can verify the environment variables:
Semgrepper.zip

And with this debug version you can print the exception message:
Semgrepper.zip

Could you give me the result of these two versions?

[R34LUS3R]

env variables:

PATH=/usr/bin:/bin:/usr/sbin:/sbin:/opt/homebrew/bin:/usr/local/bin
__CFBundleIdentifier=com.install4j.7318-9294-3757-1226.70
SHELL=/bin/zsh
JAVA_MAIN_CLASS_8025=com.install4j.runtime.launcher.MacLauncher
USER=test
LANG=en_PK.UTF-8
TMPDIR=/var/folders/c8/ffvmqkxd0h5dy99w9zz__jmm0000gn/T/
COMMAND_MODE=unix2003
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.mX5mSD0Wh4/Listeners
DISPLAY=/private/tmp/com.apple.launchd.JvAKfFeXgD/org.xquartz:0
XPC_FLAGS=0x0
__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
LOGNAME=test
XPC_SERVICE_NAME=application.com.install4j.7318-9294-3757-1226.70.53846856.53846862
HOME=/Users/test

Message:
java.io.IOException: Cannot run program "semgrep": error=2, No such file or directory
still the same messages and env variables!

[gand3lf]

@R34LUS3R That's really strange...
Ok, this version must work:
Semgrepper.zip

Essentially in this temporary version I have substitute all occurrences of "semgrep" with "/opt/homebrew/bin/semgrep"

[R34LUS3R]

@gand3lf this one worked! though wouldn't be for all the other users having the semgrep binary on different paths

[gand3lf]

@R34LUS3R thanks for the feedback. I have pushed a new version 1.2 downloadable in the release section.

[gbiagomba]

I am glad a solution could be worked out! @gand3lf but my question is if /opt/homebrew/bin/semgrep is now hardcoded what happens when a user such as myself tries to run it? because though am using homebrew but i am using it on mac and make stores its homebrew files somewhere else entirely if am not mistaken.