YARA malware query accelerator (web frontend)
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
cli enhancements in mquery,py Sep 13, 2018
docs re-scale mquery-flowchart May 24, 2018
kubernetes Update gen-secret.py Oct 4, 2018
lib Split mquery into React frontend and Flask API Sep 5, 2018
metadata Split mquery into React frontend and Flask API Sep 5, 2018
mqueryfront update npm packages Nov 2, 2018
static Split mquery into React frontend and Flask API Sep 5, 2018
tests Split mquery into React frontend and Flask API Sep 5, 2018
ursadb @ e732936 adjustments in kubernetes configs Oct 2, 2018
ursadb-cli @ a1dc0ee update dependencies Jul 10, 2018
.gitlab-ci.yml Fix .gitlab-ci.yml Sep 5, 2018
.gitmodules turns out it's safer to have modules linked via HTTPS ;) May 21, 2018
Dockerfile-daemon Split mquery into React frontend and Flask API Sep 5, 2018
Dockerfile-webapp Split mquery into React frontend and Flask API Sep 5, 2018
LICENSE initial public release May 21, 2018
README.md adjust user manual Sep 6, 2018
config.docker.py Split mquery into React frontend and Flask API Sep 5, 2018
config.example.py Split mquery into React frontend and Flask API Sep 5, 2018
daemon.py Split mquery into React frontend and Flask API Sep 5, 2018
docker-compose.yml Split mquery into React frontend and Flask API Sep 5, 2018
populate.py initial public release May 21, 2018
requirements.txt Split mquery into React frontend and Flask API Sep 5, 2018
util.py Split mquery into React frontend and Flask API Sep 5, 2018
uwsgi-docker.ini initial public release May 21, 2018
webapp.py Update webapp.py Sep 18, 2018
wsgi.py initial public release May 21, 2018

README.md

mquery: Blazingly fast Yara queries for malware analysts

Ever had trouble searching for particular malware samples? Our project is an analyst-friendly web GUI to look through your digital warehouse.

mquery can be used to search through terabytes of malware in a blink of an eye:

mquery web GUI

Thanks to our UrsaDB database, queries on large datasets can be extremely fast.

How does it work?

YARA is pretty fast, but searching through large dataset for given signature can take a lot of time. To countermeasure this, we have implemented a custom database called UrsaDB. It is able to pre-filter the results, so it is only necessary to run YARA against a small fraction of binaries:

mquery flowchart

Installation (Docker)

Recommended way of installing things is to build from sources using docker-compose:

git clone --recurse-submodules https://github.com/CERT-Polska/mquery.git
docker-compose up --scale daemon=3

where --scale daemon=... refers to the number of workers which will simultaneously process select/index jobs.

Hint: Your docker-compose must support v3 syntax of docker-compose.yml. Update your software if you have any problems.

Installation (manual)

  1. Run ursadb database (see ursadb project for further instructions on that topic).
  2. Install redis-server, python3 and npm.
  3. Install requirements: pip install -r requirements.txt.
  4. Run cp mqueryfront/src/config.dist.js mqueryfront/src/config.js
  5. Adjust settings in mqueryfront/src/config.js accordingly.
  6. Run cd mqueryfront && npm install && npm run build
  7. Copy config.example.py to config.py, remember to adjust the settings and set unique SECRET_KEY.
  8. Setup a flask application originating from webapp.py in your favourite web server.
  9. Run daemon.py - a standalone script which should work constantly, consider putting it in systemd.

How to use this thing

  1. Start up the whole system (see "Installation").
  2. Web interface (by default) should be available on http://localhost:80/
  3. Upload files to be indexed to the mquery_samples volume. From the host it should be visible at /var/lib/docker/volumes/mquery_samples/_data. If in doubt, debug using docker image inspect mquery_samples command.
  4. Open web interface, choose "admin" tab and click "Index /mnt/samples".
  5. While indexing, the current progress will be displayed in the "backend" section of "admin" tab (no auto refresh), ursadb will also periodically report something on the console.
  6. After successful indexing, your files should be searchable. Go to the main tab and upload some Yara, e.g.:
rule emotet4_basic: trojan
{
    meta:
        author = "psrok1/mak"
        module = "emotet"
    strings:
        $emotet4_rsa_public = { 8d ?? ?? 5? 8d ?? ?? 5? 6a 00 68 00 80 00 00 ff 35 [4] ff 35 [4] 6a 13 68 01 00 01 00 ff 15 [4] 85 }
        $emotet4_cnc_list = { 39 ?? ?5 [4] 0f 44 ?? (FF | A3)}
    condition:
        all of them
}

Maintainers

Questions/comments/pull requests are welcome.