Better grep/analysis

[floyd-fuh]

I think we talked about it before but because I ran EMBA again and used these tools afterwards so I thought I let you know.

Is your feature request related to a problem? Please describe.
I think the password hash search and other analysis steps after unpacking in EMBA could be improved.

Describe the solution you'd like
I guess using https://github.com/returntocorp/semgrep is the best choice at the moment because of the huge amount of rules in https://github.com/returntocorp/semgrep-rules

Describe alternatives you've considered
If semgrep doesn't work for some reason there is also my standalone https://github.com/floyd-fuh/crass/blob/master/grep-it.sh script that might be easier to implement. As an fyi, I've been maintaining it for 7 years now and even though there is semgrep I'm not planning to stop. Btw. if you would like to keep your current greps we could integrate them as rules in grep-it.sh if you think something is missing.

Additional context
I know searching for CVEs is probably a quick win in many cases, but when you really want to do proper security research to pwn a device with custom/new approaches/techniques/ideas it is good to have a tool that only points to "fishy" areas (e.g. anything related to crypto as in https://github.com/floyd-fuh/crass/blob/f2da104b073f530fbadeda7578c39a377ebd296b/grep-it.sh#L318 ). In that regards I'm not sure (I really don't know as I've just started to use it) if semgrep allows an approach that is broad enough, whereas in CRASS I'm trying. I would still rather go with semgrep. Or both. If you use semgrep with --config=auto just make sure to tell your users that you are using a third-party (metrics are sent to their servers).

[m-1-k-3]

Hi @floyd-fuh, thank you for pointing this again. We already talked on it ... probably on twitter? It is on our internal todo list but currently we are working on different other areas.

Regarding the password hash search we are currently using two approaches:

• EMBA is using STACS which gives quite good results - https://github.com/e-m-b-a/emba/blob/master/modules/S108_stacs_password_search.sh
• EMBA internal - https://github.com/e-m-b-a/emba/blob/master/modules/S107_deep_password_search.sh -> this module has different disadvantages but in case STACS is failing there is a backup in place

Have you found something that these approaches are missing? If yes, which firmware and which tool have you used for identification of the hash automatically? Probably we can tweak our current modules.

[keesj-exset]

I wonder if it could also provide some alternatives to shellcheck.

[m-1-k-3]

As far as I can see there are not a lot of *sh rules available: https://github.com/returntocorp/semgrep-rules/tree/develop/bash

[floyd-fuh]

Looking at STACS it actually looks great! But that's password/hash-related things only it seems. I guess the main focus of CRASS/semgrep is different, to also look for other vulnerable looking things. You might be right that semgrep is a little too focused on source code. Now I'm thinking maybe CRASS isn't the worst fit. Here's why I think it might be worth:

• In unpacked firmware there is sometimes a little source code for scripting languages (PHP, perl, python, JSP, JavaScript, other CGI etc.) or indirectly such as .NET binaries that are easily decompiled. I think they have a high potential for issues because they are a) sometimes called from binaries (system calls etc.) which creates a trust boundary and b) the scripting language is often not the embedded developers main language to code in. It's also very easy to analyse them because they are ASCII.
• I've found iOS source code in a appliance ISO and PHP-server-side code in an Android app before. Developers do strange things.
• Finding Cloud API keys/credentials can be a goldmine
• For me tools are also about guiding the user on what he should target during the manual analysis time. So if the tool provides a priority rated list by keyword (e.g. CRASS creates one file which includes all files that reference "SHA1")

I'll try to see if I can provide some more concrete examples next time I use EMBA.

Ultimately you can have a look from https://github.com/floyd-fuh/crass/blob/f2da104b073f530fbadeda7578c39a377ebd296b/grep-it.sh#L311 on to see what could be worth doing.

I guess it's not about what STACS or shellcheck are missing, but more what else can be done.

[m-1-k-3]

Hi @floyd-fuh, a first module using your grep-it rules is now included as s99 in this PR: #251

Give it a try ...

[m-1-k-3]

grepit module merged.