Issues running cwe_checker

[alexdetrano]

Hi there,
I am very interested in this tool but am having some problems running it. I started off using the docker method and I get the following error:

/tmp > docker run --rm -v /tmp/socat:/input fkiecad/cwe_checker /input
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("unknown variant `ROUND`, expected one of `COPY`, `LOAD`, `STORE`, `PIECE`, `SUBPIECE`, `POPCOUNT`, `INT_EQUAL`, `INT_NOTEQUAL`, `INT_LESS`, `INT_SLESS`, `INT_LESSEQUAL`, `INT_SLESSEQUAL`, `INT_ADD`, `INT_SUB`, `INT_CARRY`, `INT_SCARRY`, `INT_SBORROW`, `INT_XOR`, `INT_AND`, `INT_OR`, `INT_LEFT`, `INT_RIGHT`, `INT_SRIGHT`, `INT_MULT`, `INT_DIV`, `INT_REM`, `INT_SDIV`, `INT_SREM`, `BOOL_XOR`, `BOOL_AND`, `BOOL_OR`, `FLOAT_EQUAL`, `FLOAT_NOTEQUAL`, `FLOAT_LESS`, `FLOAT_LESSEQUAL`, `FLOAT_ADD`, `FLOAT_SUB`, `FLOAT_MULT`, `FLOAT_DIV`, `INT_NEGATE`, `INT_2COMP`, `BOOL_NEGATE`, `FLOAT_NEG`, `FLOAT_ABS`, `FLOAT_SQRT`, `FLOAT_CEIL`, `FLOAT_FLOOR`, `FLOAT_ROUND`, `FLOAT_NAN`, `INT_ZEXT`, `INT_SEXT`, `INT2FLOAT`, `FLOAT2FLOAT`, `TRUNC`", line: 45658, column: 45)', caller/src/main.rs:277:64
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The file type is a 32-bit Linux ELF for ARM:

/tmp > file /tmp/socat
/tmp/socat: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=5cfa8023dbbef69e608816f9967e981c94910f20, stripped

I also tried building cwe_checker locally and running from there, but I get the same error. I tried digging in to see what was happening under the hood, and I see cwe_checker is calling the PCodeExtractor.java file. So I create a new project in Ghidra and run the PCodeExtractor.java file directly but I get the following error:

Index 0 out of bounds for length 0
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
	at PcodeExtractor.run(PcodeExtractor.java:58)
	at ghidra.app.script.GhidraScript.executeNormal(GhidraScript.java:379)
	at ghidra.app.script.GhidraScript.doExecute(GhidraScript.java:234)
	at ghidra.app.script.GhidraScript.execute(GhidraScript.java:212)
	at ghidra.app.plugin.core.script.RunScriptTask.run(RunScriptTask.java:47)
	at ghidra.util.task.Task.monitoredRun(Task.java:124)
	at ghidra.util.task.TaskRunner.lambda$startTaskThread$0(TaskRunner.java:104)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

---------------------------------------------------
Build Date: 2020-Dec-29 1701 EST
Ghidra Version: 9.2.2
Java Home: /Library/Java/JavaVirtualMachines/jdk-11.0.7.jdk/Contents/Home
JVM Version: Oracle Corporation 11.0.7
OS: Mac OS X 10.16 x86_64
Workstation: <redacted>

I get the same error using Mac OSX and Linux Mint. Any clue why I can't run cwe_checker?

[Enkelmann]

I ran into this issue very recently myself. The problem is that Ghidra generates a P-Code instruction ROUND which is undocumented in the P-Code reference manual. Since it is undocumented, the cwe_checker doesn't yet know how to deal with it and crashes (because it assumes it is a bug in the PCodeExtractor.java).
I will have to dig into the Ghidra code to find out what ROUND is meant to do exactly, a fix should be ready next week. Fortunately, I already have example binaries for debugging. Unfortunately, until the fix is ready there is no workaround for binaries containing assembly instructions for which Ghidra generates the ROUND P-Code instruction.

Edit: Just to emphasize: The firmware image where I encountered this contained a lot of binaries with this problem, while other firmware images that I tested the cwe_checker on did not contain a single binary with this problem. So while it may seem like an issue with the execution environment, it is not! You can check that by running the cwe_checker on some regular x86-binaries.

Oh, I am also interested in your experience in running the cwe_checker on Mac OSX. Since I don't have a Mac to experiment with I don't know whether a local installation of the cwe_checker would actually work at all. Have you tried that and want to share the results?

[alexdetrano]

sure, the ROUND pcode instruction gets generated for the ARM vcvt.s32.f64 function, which converts from signed 32 bit int to floating point 64 bit. In this particular case, socat is parsing a command line argument -t, which specifies a timeout value
eg 0.5. It is strange that this PCODE instruction is undocumented...

Running locally on Mac OSx I definitely ran into some quirks. For example, when trying to run cwe_checker, I kept getting errors about files not being found. I hacked up the source code to see what file it was expecting and it was looking for the config file in a strange location - /Users/adetrano/Library/Application\ Support/cwe_checker/config.json. No idea why, especially since that file was actually installed to ~/.config/cwe_checker/config.json.

I couldn't just symlink /Users/adetrano/Library/Application\ Support/cwe_checker to ~/.config/cwe_checker either, as cwe_checker wasn't properly reading the symlink. I ended up symlinking each file like

ln -s ~/.config/cwe_checker/ghidra.json /Users/adetrano/Library/Application\ Support/cwe_checker/ghidra.json
ln -s ~/.config/cwe_checker/config.json /Users/adetrano/Library/Application\ Support/cwe_checker/config.json

but then I ran into the same issue when it came time to run the PCodeExtractor, more issues about a file not being found. So I just copied the whole directory to where cwe_checker was looking and that fixed it

cp -r ~/.local/share/cwe_checker/ghidra ~/Library/Application\ Support/cwe_checker/

Not a great solution but at least it worked

[Enkelmann]

Thanks for sharing your MacOS experiences! :-) I have opened a new issue in #142 for the incorrect lookup of the location of configuration files on MacOS. No promises on when I will have time to work on it though.

[Enkelmann]

Issue should be fixed now thanks to #143. My initial assessment was wrong, the ROUND instruction is just the FLOAT_ROUND, which is properly documented.

Coincidentally, the P-Code generated by Ghidra still seems wrong to me, since your assembly instruction should convert fp64 to signed 32-bit integer, not the other way around. If I am correct, then the ROUND should be a TRUNC instruction. But this won't affect the cwe_checker in any case, we don't have analyses for floating point variables yet.

Edit: I am closing the issue, but feel free to reopen if you still experience problems with the cwe_checker.

[ghost]

Hi, I ran into a similar issue, but this time with FLOOR / FLOAT_FLOOR. (Probably happens with more of the FLOAT_* P-Codes)

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("unknown variant `FLOOR`, expected one of `COPY`, `LOAD`, `STORE`, `PIECE`, `SUBPIECE`, `POPCOUNT`, `INT_EQUAL`, `INT_NOTEQUAL`, `INT_LESS`, `INT_SLESS`, `INT_LESSEQUAL`, `INT_SLESSEQUAL`, `INT_ADD`, `INT_SUB`, `INT_CARRY`, `INT_SCARRY`, `INT_SBORROW`, `INT_XOR`, `INT_AND`, `INT_OR`, `INT_LEFT`, `INT_RIGHT`, `INT_SRIGHT`, `INT_MULT`, `INT_DIV`, `INT_REM`, `INT_SDIV`, `INT_SREM`, `BOOL_XOR`, `BOOL_AND`, `BOOL_OR`, `FLOAT_EQUAL`, `FLOAT_NOTEQUAL`, `FLOAT_LESS`, `FLOAT_LESSEQUAL`, `FLOAT_ADD`, `FLOAT_SUB`, `FLOAT_MULT`, `FLOAT_DIV`, `INT_NEGATE`, `INT_2COMP`, `BOOL_NEGATE`, `FLOAT_NEG`, `FLOAT_ABS`, `FLOAT_SQRT`, `FLOAT_CEIL`, `FLOAT_FLOOR`, `FLOAT_ROUND`, `FLOAT_NAN`, `INT_ZEXT`, `INT_SEXT`, `INT2FLOAT`, `FLOAT2FLOAT`, `TRUNC`", line: 810373, column: 45)', caller/src/main.rs:314:64```

[Enkelmann]

Thanks for reporting it! We obviously do not have enough floating point arithmetics in our test cases... I will check all the floating point P-Code instructions this time, fix should be ready by monday.

Edit: Fix is in the master branch. I checked all the P-Code instructions and it did also occur for CEIL / FLOAT_CEIL. But now it should finally be fixed for all P-Code instructions.