Skip to content

CybercentreCanada/assemblyline-incident-manager

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

47 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Assemblyline incident manager

This repository contains three Python scripts used for bulk triaging file using Assemblyline.

  1. Submitter (al-incident-submitter): pushes files from a directory to an Assemblyline instance for analysis.
  2. Result analyzer (al-incident-analyzer): pulls the submissions from the Assemblyline instance and reports on if the submissions are safe/unsafe.
  3. Downloader (al-incident-downloader): downloads files submitted to Assemblyline that are under a certain score threshold, matching the folder structure of the files as they were submitted.

Prequisites

  • You will need the URL of an Assemblyline instance that you have an account on, for best results make its loaded with your best Yara rules, Sandboxes etc.
    • Want to create your own Assemblyline instance? HOW-TO
  • You will need two API keys generated by Assemblyline, ideally one with read access and another with write access.
    • The Write-only key will be used for the "Submitter" and the Read-only key will be used for the "Result Analysis" and the "Downloader".
    • This helps in the context of incident response to reduce the exposure of your Assemblyline instance.

Installation

Linux

  • Install the following packages: libffi-dev, libssl-dev
    • (APT) sudo apt-get install libffi-dev libssl-dev python3
    • (YUM) sudo yum install libffi-dev libssl-dev python3
  • Upgrade PIP: python3 -m pip install --upgrade pip
  • python3 -m pip install assemblyline-incident-manager

Windows

  • Download and install the most recent Python .msi installer from https://www.python.org/downloads/release.
  • Upgrade PIP: python -m pip install --upgrade pip
  • python -m pip install assemblyline-incident-manager

Usage

Submitter

al-incident-submitter --help
Usage: al-incident-submitter [OPTIONS] COMMAND [ARGS]...

  Example: al-incident-submitter --url="https://<domain-of-Assemblyline-
  instance>" --username="<user-name>"
  --apikey="/path/to/file/containing/apikey"
  --classification="<classification>" --service_selection="<service-
  name>,<service-name>" --path="/path/to/scan"
  --incident_num=123

Options:
  --url TEXT                The target URL that hosts Assemblyline.
                            [required]

  --username TEXT           Your Assemblyline account username.  [required]
  --apikey PATH             A path to a file that contains only your
                            Assemblyline account API key. NOTE that this API
                            key requires write access.  [required]

  --ttl INTEGER             The amount of time that you want your Assemblyline
                            submissions to live on the Assemblyline system (in
                            days).

  --classification TEXT     The classification level for each file submitted
                            to Assemblyline.  [required]

  --service_selection TEXT  A comma-separated list (no spaces!) of service
                            names (case-sensitive) to send files to. If not provided, all
                            services will be selected.

  -t, --is_test             A flag that indicates that you're running a test.
  --path PATH               The directory path containing files that you want
                            to submit to Assemblyline.  [required]

  -f, --fresh               Restart ingestion from the beginning.

  --incident_num TEXT       The incident number for each file to be associated
                            with.  [required]

  --resubmit-dynamic        All files that score higher than 500 will be
                            resubmitted for dynamic analysis.

  --alert                   Generate alerts for this submission.
  --threads INTEGER         Number of threads that will ingest files to
                            Assemblyline.

  --dedup_hashes            Only submit files with unique hashes. If you want
                            100% file coverage in a given path, do not use
                            this flag

  --priority INTEGER        Provide a priority number which will cause the
                            ingestion to go to a specific priority queue.

  --do_not_verify_ssl       Ignore SSL errors (insecure!)

  --help                    Show this message and exit.

Analyzer

al-incident-analyzer --help
Usage: al-incident-analyzer [OPTIONS] COMMAND [ARGS]...

  Example: al-incident-analyzer --url="https://<domain-of-
  Assemblyline-instance>" --username="<user-name>"
  --apikey="/path/to/file/containing/apikey" --incident_num=123

Options:
  --url TEXT           The target URL that hosts Assemblyline.  [required]
  -u, --username TEXT  Your Assemblyline account username.  [required]
  --apikey PATH        A path to a file that contains only your Assemblyline
                       account API key. NOTE that this API key requires write
                       access.  [required]

  --min_score INTEGER  The minimum score for files that we want to query from
                       Assemblyline.

  --incident_num TEXT  The incident number for each file to be associated
                       with.  [required]

  -t, --is_test        A flag that indicates that you're running a test.
  --help               Show this message and exit.

Now check the report.csv file that was created. This file will contain what files are safe/unsafe.

Downloader

al-incident-downloader --help
Usage: al-incident-downloader [OPTIONS] COMMAND [ARGS]...

  Example: al-incident-downloader --url="https://<domain-of-
  Assemblyline-instance>" --username="<user-name>"
  --apikey="/path/to/file/containing/apikey" --incident_num=123
  --max_score=100 --download_path=/path/to/where/you/want/downloads
  --upload_path=/path/from/where/files/were/uploaded/from

Options:
  --url TEXT                    The target URL that hosts Assemblyline.
                                [required]

  -u, --username TEXT           Your Assemblyline account username.
                                [required]

  --apikey PATH                 A path to a file that contains only your
                                Assemblyline account API key. NOTE that this
                                API key requires read access.  [required]

  --max_score INTEGER           The maximum score for files that we want to
                                download from Assemblyline.  [required]

  --incident_num TEXT           The incident number that each file is
                                associated with.  [required]

  --download_path PATH          The path to the folder that we will download
                                files to.  [required]

  --upload_path PATH            The base path from which the files were
                                ingested from.
                                [required]

  -t, --is_test                 A flag that indicates that you're running a
                                test.

  --num_of_downloaders INTEGER  The number of threads that will be created to
                                facilitate downloading the files.

  --do_not_verify_ssl           Verify SSL when creating and using the
                                Assemblyline Client.

  --help                        Show this message and exit.

If you check the download path you supplied, you should have all files downloaded there.


L'assistant à la réponse aux incidents d'Assemblyline

Ce répertoire contient trois scripts Python pour assisté le triage de grande quantité de fichiers avec Assemblyline.

  1. Soumission (al-incident-submitter): envoi les fichiers contenu dans un dossier vers une instance Assemblyline pour l'analyze.
  2. Résultats d'analyse (al-incident-analyzer): analyse les soumissions et génère un rapport.
  3. Téléchargeur (al-incident-downloader): télécharge les fichiers sous un certain pointage en préservant la structure original.

Prérequis

  • Vous aurez besoin d'un instance d'Assemblyline à jour et avec vos meilleurs règles Yara, "Sandboxes" etc.
    • Voici comment crée vôtre propre instance: LIEN
  • Nous vous recommandons d'utilisé deux clé d'api, un write only et une read only
    • La clé Write-only sera utilisé pour soumettre vos fichier avec le script "Submitter" et la clé Read-only sera pour "Result Analysis" et le "Downloader".
    • Cette séparation aidera a securisé vôtre instance Assemblyline dans un context de réponse aux incidents

Installation

Linux

  • Installé les packages suivants: libffi-dev, libssl-dev
    • (APT) sudo apt-get install libffi-dev libssl-dev python3
    • (YUM) sudo yum install libffi-dev libssl-dev python3
  • Mise à jour de PIP: python3 -m pip install --upgrade pip
  • python3 -m pip install assemblyline-incident-manager

Windows

Utilisation

Submitter

al-incident-submitter --help
Usage: al-incident-submitter [OPTIONS] COMMAND [ARGS]...

  Example: al-incident-submitter --url="https://<domain-of-Assemblyline-
  instance>" --username="<user-name>"
  --apikey="/path/to/file/containing/apikey"
  --classification="<classification>" --service_selection="<service-
  name>,<service-name>" --path="/path/to/scan"
  --incident_num=123

Options:
  --url TEXT                The target URL that hosts Assemblyline.
                            [required]

  --username TEXT           Your Assemblyline account username.  [required]
  --apikey PATH             A path to a file that contains only your
                            Assemblyline account API key. NOTE that this API
                            key requires write access.  [required]

  --ttl INTEGER             The amount of time that you want your Assemblyline
                            submissions to live on the Assemblyline system (in
                            days).

  --classification TEXT     The classification level for each file submitted
                            to Assemblyline.  [required]

  --service_selection TEXT  A comma-separated list (no spaces!) of service
                            names (case-sensitive) to send files to. If not provided, all
                            services will be selected.

  -t, --is_test             A flag that indicates that you're running a test.
  --path PATH               The directory path containing files that you want
                            to submit to Assemblyline.  [required]

  -f, --fresh               Restart ingestion from the beginning.

  --incident_num TEXT       The incident number for each file to be associated
                            with.  [required]

  --resubmit-dynamic        All files that score higher than 500 will be
                            resubmitted for dynamic analysis.

  --alert                   Generate alerts for this submission.
  --threads INTEGER         Number of threads that will ingest files to
                            Assemblyline.

  --dedup_hashes            Only submit files with unique hashes. If you want
                            100% file coverage in a given path, do not use
                            this flag

  --priority INTEGER        Provide a priority number which will cause the
                            ingestion to go to a specific priority queue.

  --do_not_verify_ssl       Ignore SSL errors (insecure!)

  --help                    Show this message and exit.

Analyzer

al-incident-analyzer --help
Usage: al-incident-analyzer [OPTIONS] COMMAND [ARGS]...

  Example: al-incident-analyzer --url="https://<domain-of-
  Assemblyline-instance>" --username="<user-name>"
  --apikey="/path/to/file/containing/apikey" --incident_num=123

Options:
  --url TEXT           The target URL that hosts Assemblyline.  [required]
  -u, --username TEXT  Your Assemblyline account username.  [required]
  --apikey PATH        A path to a file that contains only your Assemblyline
                       account API key. NOTE that this API key requires write
                       access.  [required]

  --min_score INTEGER  The minimum score for files that we want to query from
                       Assemblyline.

  --incident_num TEXT  The incident number for each file to be associated
                       with.  [required]

  -t, --is_test        A flag that indicates that you're running a test.
  --help               Show this message and exit.

Regardez le rapport dans report.csv. Ce fichier contient un rapport des détections.

Downloader

al-incident-downloader --help
Usage: al-incident-downloader [OPTIONS] COMMAND [ARGS]...

  Example: al-incident-downloader --url="https://<domain-of-
  Assemblyline-instance>" --username="<user-name>"
  --apikey="/path/to/file/containing/apikey" --incident_num=123
  --max_score=100 --download_path=/path/to/where/you/want/downloads
  --upload_path=/path/from/where/files/were/uploaded/from

Options:
  --url TEXT                    The target URL that hosts Assemblyline.
                                [required]

  -u, --username TEXT           Your Assemblyline account username.
                                [required]

  --apikey PATH                 A path to a file that contains only your
                                Assemblyline account API key. NOTE that this
                                API key requires read access.  [required]

  --max_score INTEGER           The maximum score for files that we want to
                                download from Assemblyline.  [required]

  --incident_num TEXT           The incident number that each file is
                                associated with.  [required]

  --download_path PATH          The path to the folder that we will download
                                files to.  [required]

  --upload_path PATH            The base path from which the files were
                                ingested from.
                                [required]

  -t, --is_test                 A flag that indicates that you're running a
                                test.

  --num_of_downloaders INTEGER  The number of threads that will be created to
                                facilitate downloading the files.

  --do_not_verify_ssl           Verify SSL when creating and using the
                                Assemblyline Client.

  --help                        Show this message and exit.

Tous les fichiers sans détections seront téléchargé dans le dossier choisi.