misp-playbooks/pb_query_ip_reputation-with_output.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ca1fbfb5",
   "metadata": {
    "tags": [
     "playbook-introduction"
    ]
   },
   "source": [
    "# Query IP reputation\n",
    "\n",
    "## Introduction\n",
    "\n",
    "- UUID: **39a029be-e4d7-40da-88b2-1d0bf9417ad0**\n",
    "- Started from [issue 12](https://github.com/MISP/misp-playbooks/issues/12)\n",
    "- State: **Published** : demo version with **output**\n",
    "- Purpose: A playbook to query for the reputation of one or more IPs. \n",
    "    - It combines the reputation scores from **VirusTotal**, **Shodan**, **Greynoise** and **AbuseIPDB** into one MISP report. \n",
    "    - The playbook adds the known associated **domains**, the **abuse contacts** and the geo information from **MMDB**.\n",
    "    - All information is added to a MISP event, summarised and send to **Mattermost** and **TheHive**.\n",
    "- Tags: [ \"ip\", \"reputation\", \"abuse\", \"whois\" ]\n",
    "- External resources: **abuse_finder**, **DNS**, **MMDB**, **Shodan**, **Greynoise**, **VirusTotal**, **AbuseIPDB**, **Mattermost, TheHive**\n",
    "- Target audience: **SOC**, **CSIRT**, **CTI**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a512a2c1",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "# Playbook\n",
    "\n",
    "- **Query IP reputation**\n",
    "    - Introduction\n",
    "- **Preparation**\n",
    "    - PR:1 Initialise environment\n",
    "    - PR:2 Verify MISP modules\n",
    "    - PR:3 Load helper functions\n",
    "    - PR:4 Set helper variables\n",
    "    - PR:5 Provide the list of IPs\n",
    "    - PR:6 MISP event details\n",
    "    - PR:7 Setup MISP event link\n",
    "- **Investigate**\n",
    "    - IN:1 Context details for the IPs\n",
    "    - IN:2 Add the IPs to the event\n",
    "- **Correlation**\n",
    "    - CR:1 Correlation with MISP events\n",
    "    - CR:2 Correlation with MISP feeds\n",
    "- **Enrichment**\n",
    "    - ER:1 Enrich with DNS information\n",
    "    - ER:2 Enrich with abuse information\n",
    "    - ER:3 Enrich with information from VirusTotal\n",
    "    - ER:4 Enrich with information from Shodan\n",
    "    - ER:5 Get reputation from Greynoise\n",
    "    - ER:6 Get geo information from MMDB\n",
    "    - ER:7 Enrich with information from AbuseIPDB\n",
    "    - ER:8 Investigation and enrichment report\n",
    "- **Summary**\n",
    "    - EN:1 MISP indicators\n",
    "    - EN:2 Create the summary of the playbook\n",
    "    - EN:3 Print the summary\n",
    "    - EN:4 Display a map of the IPs\n",
    "    - EN:5 Send a summary to Mattermost    \n",
    "    - EN:6 Send an alert to TheHive\n",
    "    - EN:7 End of the playbook\n",
    "- External references\n",
    "- Technical details"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dc24913",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Preparation\n",
    "\n",
    "## PR:1 Initialise environment\n",
    "\n",
    "This section **initialises the playbook environment** and loads the required Python libraries. \n",
    "\n",
    "The credentials for MISP (**API key**) and other services are loaded from the file `keys.py` in the directory **vault**. A [PyMISP](https://github.com/MISP/PyMISP) object is created to interact with MISP and the active MISP server is displayed. By printing out the server name you know that it's possible to connect to MISP. In case of a problem PyMISP will indicate the error with `PyMISPError: Unable to connect to MISP`.\n",
    "\n",
    "The contents of the `keys.py` file should contain at least :\n",
    "\n",
    "```\n",
    "misp_url=\"<MISP URL>\"                  # The URL to our MISP server\n",
    "misp_key=\"<MISP API KEY>\"              # The MISP API key\n",
    "misp_verifycert=<True or False>        # Indicate if PyMISP should attempt to verify the certificate or ignore errors\n",
    "urlscan_url=\"https://urlscan.io/api/v1/search\"\n",
    "urlscan_apikey=\"<URLSCAN API KEY>\"\n",
    "mattermost_playbook_user=\"<MATTERMOST USER>\"\n",
    "mattermost_hook=\"<MATTERMOST WEBHOOK>\"\n",
    "thehive_url=\"<THEHIVE URL>\"\n",
    "thehive_key=\"<THEHIVE API KEY>\"\n",
    "virustotal_apikey=\"<VIRUSTOTAL_APIKEY>\"\n",
    "shodan_apikey=\"<SHODAN_APIKEY>\"\n",
    "greynoise_apikey=\"<GREYNOISE_APIKEY>\"\n",
    "abuseipdb_apikey=<ABUSEIPDB_APIKEY>\"\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "5f94c336",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The version of PyMISP recommended by the MISP instance (2.4.176) is newer than the one you're using now (2.4.173). Please upgrade PyMISP.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The \u001b[92mPython libraries\u001b[90m are loaded and the \u001b[92mcredentials\u001b[90m are read from the keys file.\n",
      "I will use the MISP server \u001b[92mhttps://misp.demo.cudeso.be/\u001b[90m for this playbook.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Initialise Python environment\n",
    "import urllib3\n",
    "import sys\n",
    "import json\n",
    "import base64\n",
    "import uuid\n",
    "import time\n",
    "import re\n",
    "from prettytable import PrettyTable, MARKDOWN\n",
    "from IPython.display import Image, display, display_markdown, HTML\n",
    "from datetime import date\n",
    "import requests\n",
    "from pymisp import *\n",
    "from pymisp.tools import GenericObjectGenerator\n",
    "from abuse_finder import domain_abuse, ip_abuse, email_abuse, url_abuse\n",
    "import textwrap\n",
    "from datetime import datetime\n",
    "\n",
    "# Used for -optionally- drawing the maps.\n",
    "import folium\n",
    "\n",
    "# Load the credentials\n",
    "sys.path.insert(0, \"../vault/\")\n",
    "from keys import *\n",
    "if misp_verifycert is False:\n",
    "    import urllib3\n",
    "    urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)\n",
    "print(\"The \\033[92mPython libraries\\033[90m are loaded and the \\033[92mcredentials\\033[90m are read from the keys file.\")\n",
    "\n",
    "# Create the PyMISP object\n",
    "misp = PyMISP(misp_url, misp_key, misp_verifycert)\n",
    "misp_headers = {\"Authorization\": misp_key,  \"Content-Type\": \"application/json\", \"Accept\": \"application/json\"}\n",
    "print(\"I will use the MISP server \\033[92m{}\\033[90m for this playbook.\\n\".format(misp_url))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e4c6424-0056-416a-812c-04a7559cbe56",
   "metadata": {},
   "source": [
    "## PR:2 Verify MISP modules\n",
    "\n",
    "This playbook uses the MISP modules to obtain additional correlation or enrichment information. [MISP modules](https://github.com/MISP/misp-modules) are autonomous modules that can be used to extend MISP for new services such as expansion, import and export. The modules are written in Python 3 following a simple API interface. The objective is to ease the extensions of MISP functionalities without modifying core components. The API is available via a simple REST API which is independent from MISP installation or configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9a797506-c5d7-45e8-ae58-cbf9a833162c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found the \u001b[92mshodan\u001b[90m MISP module (Accepted input: ['ip-src', 'ip-dst']).\n",
      "Found the \u001b[92mreversedns\u001b[90m MISP module (Accepted input: ['ip-src', 'ip-dst', 'domain|ip']).\n",
      "Found the \u001b[92mvirustotal_public\u001b[90m MISP module (Accepted input: ['hostname', 'domain', 'ip-src', 'ip-dst', 'md5', 'sha1', 'sha256', 'url']).\n"
     ]
    }
   ],
   "source": [
    "# Where can we find the local MISP Module server? You can leave this to the default setting in most cases.\n",
    "misp_modules_url = \"http://127.0.0.1:6666\"\n",
    "\n",
    "# How long do we wait between queries when using the MISP modules (API rate limiting of external service such as VirusTotal)\n",
    "misp_modules_wait = 3\n",
    "\n",
    "# Initiliasation\n",
    "misp_modules = {}\n",
    "misp_modules_headers = {\n",
    "    \"Content-Type\": \"application/json\",\n",
    "    \"Accept\": \"application/json\"\n",
    "}\n",
    "misp_modules_in_use = [\"reversedns\", \"virustotal_public\", \"shodan\"]\n",
    "# Code block to query the MISP module server and check if our modules are enabled\n",
    "res = requests.get(\"{}/modules\".format(misp_modules_url), headers=misp_modules_headers)\n",
    "for module in res.json():\n",
    "    for module_requested in misp_modules_in_use:\n",
    "        if module.get(\"name\", False) == module_requested:\n",
    "            misp_modules[module_requested] = {\"enabled\": True, \"input\": module.get(\"mispattributes\").get(\"input\")}\n",
    "            print(\"Found the \\033[92m{}\\033[90m MISP module (Accepted input: {}).\".format(module_requested, misp_modules[module_requested][\"input\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9e92317-49fd-4746-b61e-abd84468114e",
   "metadata": {},
   "source": [
    "## PR:3 Load helper functions\n",
    "\n",
    "The next cell contains **helper functions** that are used in this playbook. \n",
    "\n",
    "Instead of distributing helper functions as separate Python files this playbook includes all the required code as one code cell. This makes portability of playbooks between instances easier. The downside is that functions defined in this playbook need to be defined again in other playbooks, which is not optimal for code re-use. For this iteration of playbooks it is chosen to include the code in the playbook (more portability), but you can easily create one \"helper\" file that contains all the helper code and then import that file in each playbook (for example by adding to the previous cell `from helpers import *`). Note that the graphical workflow image is included as an external image. A missing image would not influence the further progress of the playbook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "cfab4370-b5be-483d-9344-0b414a06cd61",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92mHelper functions loaded\u001b[90m.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "def pb_get_misp_tags(tags=[], local_tags=[]):\n",
    "    '''\n",
    "    Get a list of MISP tags based on a Python list\n",
    "\n",
    "    :param misp: MISP object\n",
    "    :param object_template: which object template to return\n",
    "    '''\n",
    "    misp_tags = []\n",
    "    for el in tags:\n",
    "        t = MISPTag()\n",
    "        t.name = el\n",
    "        t.local = False\n",
    "        misp_tags.append(t)\n",
    "\n",
    "    for el in local_tags:\n",
    "        t = MISPTag()\n",
    "        t.name = el\n",
    "        t.local = True\n",
    "        misp_tags.append(t)\n",
    "    return misp_tags\n",
    "\n",
    "\n",
    "\n",
    "def get_playbook_result(value, source, category, res_type):\n",
    "    '''\n",
    "    Lookup the playbook results for a list of entries\n",
    "    \n",
    "    : param value: The IP address under investigation\n",
    "    : param source: Which source provided the enrichment?\n",
    "    : param category: Enrichment category\n",
    "    : param res_type: Enrichment type\n",
    "    '''\n",
    "    return_value = []\n",
    "    for el in playbook_results[value]:\n",
    "        if el[\"source\"] == source and el[\"category\"] == category and el[\"type\"] == res_type:\n",
    "            if el[\"enriched\"] not in return_value:\n",
    "                if el[\"enriched\"] is list:\n",
    "                    for list_item in el[\"enriched\"]:\n",
    "                        return_value.append(str(list_item))\n",
    "                else:\n",
    "                    return_value.append(str(el[\"enriched\"]))\n",
    "    return_value = ' '.join(return_value).strip()\n",
    "    if len(return_value) < 1:\n",
    "        return_value = \"-\"\n",
    "    return return_value\n",
    "\n",
    "\n",
    "\n",
    "def pb_add_enrichment(playbook_results, field, entry, key, value):\n",
    "    '''\n",
    "    Add an enrichment (or correlation) entry but first check that the value is not already there\n",
    "    \n",
    "    : param playbook_results: all the enrichment results\n",
    "    : param field\n",
    "    : param entry\n",
    "    : param key\n",
    "    : param value\n",
    "    '''\n",
    "    skip_field = False\n",
    "    for existing_entry in playbook_results.get(field, []):\n",
    "        if existing_entry.get(key, False) == value:\n",
    "            skip_field = True\n",
    "            print(\" Not adding to playbook results because of duplicate. Already added via {}\".format(existing_entry.get(\"source\", False)))\n",
    "    if not skip_field:\n",
    "        if field in playbook_results:\n",
    "            playbook_results[field].append(entry)\n",
    "        else:\n",
    "            playbook_results[field] = [entry]\n",
    "    return playbook_results\n",
    "\n",
    "\n",
    "print(\"\\033[92mHelper functions loaded\\033[90m.\\n\".format(misp_url))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b0fe9f4-23b1-4eec-bc61-8bc965ffc619",
   "metadata": {},
   "source": [
    "## PR:4 Set helper variables\n",
    "\n",
    "This cell contains **helper variables** for this playbook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "645d8ee5-e07f-4bab-953c-8d71f9c1b171",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Dictionary to playbook results and some of the core objects that are created\n",
    "playbook_results = {}\n",
    "case_objects = {}\n",
    "\n",
    "# A set of regular expressions that we use to determine the attribute type in a warninglist\n",
    "regular_expressions = {\"sha256\": \"^[a-fA-F0-9]{64}$\",\n",
    "                       \"md5\": \"^[a-fA-F0-9]{32}$\",\n",
    "                       \"hostname\": \"^[a-zA-Z0-9.\\-_]+\\.[a-zA-Z]{2,}$\",\n",
    "                       \"sha1\": \"^[a-fA-F0-9]{40}$\",\n",
    "                       \"url\": \"^(http|https):\\/\\/[-a-zA-Z0-9-]{2,256}\\.[-a-zA-Z0-9-]{2,256}\",\n",
    "                       \"ip-src\": \"[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\",\n",
    "                       \"ip-dst\": \"[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\"\n",
    "                      }"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62f98308-a0c2-486d-9675-44f51f9a538e",
   "metadata": {},
   "source": [
    "## PR:5 Provide the list of IPs\n",
    "\n",
    "Add the **IPs** that you want to investigate with this playbook to the variable `query`. You can provide one IP or a list of IPs. You can supply an IPv4 or IPv6 address but remember that not all enrichment services provide sufficient results for IPv6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "01fd3a79-9ab2-4fde-a9d3-2a242a621a8f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Provide one or more IPs\n",
    "query = [\"91.219.237.56\", \"185.194.93.128\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b67481c1-3e9f-4f18-82b8-284c132057cf",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The playbook will do the IP reputation query for \u001b[92m['91.219.237.56', '185.194.93.128']\u001b[90m\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Simple function to make a list. Makes it more consistent to work with the user input in the next cells.\n",
    "if type(query) == str:\n",
    "    query =[query]\n",
    "print(\"The playbook will do the IP reputation query for \\033[92m{}\\033[90m\\n\".format(query))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bfd0045-13fe-4f83-a6a7-2d70621b7423",
   "metadata": {},
   "source": [
    "## PR:6 MISP event details\n",
    "\n",
    "### Event title\n",
    "\n",
    "In this playbook we create a **new** MISP event with title **IP reputation _IPlist_**. You get the chance to override this default title but remember that it is good practice to choose a self-explanatory **event title**. This event title is shown in the MISP event index and should provide you the necessary information what the event is about. You should avoid using generic event titles. Read the [Best Practices in Threat Intelligence](https://www.misp-project.org/best-practices-in-threat-intelligence.html) for further guidance.\n",
    "\n",
    "### Contexualisation\n",
    "\n",
    "This playbook adds event contexualisation via the **tags** that are defined in `event_additional_global_tags` (for *global* tags) and `event_additional_local_tags` (for *local* tags). As a reminder, whereas *global* tags remain attached to the events that you share with your community, the *local* tags are not shared outside your organisation. It's also a good idea to primarily use tags that are part of a [taxonomy](https://github.com/MISP/misp-taxonomies), this allows you to make the contexualisation more portable accross multiple MISP instances.\n",
    "\n",
    "In this playbook the list of tags is build via one of the helper functions `pb_get_misp_tags`. This function takes two arguments, first a list of tags to convert as *global* tags, and secondly a list of tags to convert as *local* tags. It then returns a Python list of MISPTag objects.\n",
    "\n",
    "### Traffic Light Protocol\n",
    "\n",
    "The default **TLP** for this event is **<span style='color:#FFBF00'>tlp:amber</span>**. The Traffic Light Protocol (TLP) facilitates sharing of potentially sensitive information and allows for more effective collaboration. TLP is a set of four standard labels to indicate the sharing boundaries to be applied by the recipients. TLP is always set by the creator of information. You can find more information at [FIRST](https://www.first.org/tlp/). You can specify the TLP via `event_tlp`.\n",
    "\n",
    "### MISP Galaxies\n",
    "\n",
    "This playbook can also add MISP galaxies to the event with the variable `event_galaxies`. You can also leave the list empty if you do not want to add galaxies in this stage of the investigation.\n",
    "\n",
    "### MISP distribution, threat level and analysis level\n",
    "\n",
    "Optionally you can specifiy a MISP **distribution** (with `event_distribution`), **threat level** (with `event_threat_level_id`) or **analysis state** (with `event_analysis`). The event **date** is set to today via `event_date`.\n",
    "\n",
    "If you cannot remember the options for distribution, threat level or the analysis state then use the next cell to guide you. This cell is set as **raw**. If you **change its type to code** and execute the cell you get an overview of the options available for creating a MISP event."
   ]
  },
  {
   "cell_type": "raw",
   "id": "0f74da01-e0db-4140-9386-09be040d7fbd",
   "metadata": {
    "tags": []
   },
   "source": [
    "# This cell is 'raw' by default. Change it to 'code' to execute it.\n",
    "\n",
    "print(\"Distribution value\")\n",
    "for e in Distribution:\n",
    "    print(\" \", e.value, \"=\", e)\n",
    "print(\"\\n\")\n",
    "print(\"Threat level\")\n",
    "for e in ThreatLevel:\n",
    "    print(\" \", e.value, \"=\", e)\n",
    "print(\"\\n\")\n",
    "print(\"Analysis level\")\n",
    "for e in Analysis:\n",
    "    print(\" \", e.value, \"=\", e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f46eaf74-a42e-41c5-8f10-e8882d5dd31d",
   "metadata": {},
   "source": [
    "## PR:7 Setup MISP event link\n",
    "\n",
    "By default the playbook will generate a **title** with a prefix and the IPs you want to investigate. You can override this event title with the variable `event_title`. If you leave `event_title` empty the playbook will generate the MISP event title for you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4f2f2568-9ee6-4f82-a267-075e98f4e477",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Provide the event title for a new event. Leave blank for the playbook to auto generate one\n",
    "event_title = \"\"\n",
    "\n",
    "# Prefix for auto generate event title\n",
    "event_title_default_prefix = \"IP reputation\"\n",
    "\n",
    "# Optionally, you can change TLP, add additional event (local and global) tags, threatlevel, analysis state or distribution level\n",
    "event_tlp = \"tlp:amber\"\n",
    "\n",
    "# Event context\n",
    "event_additional_global_tags = []                                 # This needs to be a Python list\n",
    "event_additional_local_tags = [\"workflow:state=\\\"incomplete\\\"\"]   # This needs to be a Python list\n",
    "\n",
    "# Event galaxies\n",
    "event_galaxies = [\"misp-galaxy:mitre-attack-pattern=\\\"IP Addresses - T1590.005\\\"\"]\n",
    "\n",
    "# Additional MISP event settings\n",
    "event_threat_level_id = ThreatLevel.low\n",
    "event_analysis = Analysis.ongoing\n",
    "event_distribution = Distribution.your_organisation_only\n",
    "event_date = date.today()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a6b266f-577f-4a4c-882b-10542107987e",
   "metadata": {},
   "source": [
    "### Create MISP event\n",
    "\n",
    "The next cell **creates the MISP event** and stores the event object in `misp_event`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "864d7c62-a923-472a-937a-ba196c148b77",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Continue the playbook with the new \u001b[92mcreated\u001b[90m MISP event ID 3125 with title \u001b[92mIP reputation for 185.194.93.128 91.219.237.56\u001b[90m and UUID d4929a85-1590-46d2-900a-615113572ee0\n",
      " \u001b[92mAdded\u001b[90m event tag <MISPTag(name=tlp:amber)>\n",
      " \u001b[92mAdded\u001b[90m event tag <MISPTag(name=['misp-galaxy:mitre-attack-pattern=\"IP Addresses - T1590.005\"'])>\n",
      " \u001b[92mAdded\u001b[90m event tag <MISPTag(name=workflow:state=\"incomplete\")>\n"
     ]
    }
   ],
   "source": [
    "# Code block to create the event or add data to an existing event\n",
    "event_title = event_title.strip()\n",
    "\n",
    "if not(len(event_title) > 0):\n",
    "    query_sample = \"\"\n",
    "    for key in query:\n",
    "        query_sample = \"{} {}\".format(key, query_sample)\n",
    "    event_title = \"{} for {}\".format(event_title_default_prefix, query_sample)\n",
    "\n",
    "# Construct the event tags\n",
    "event_additional_global_tags.append(event_tlp)\n",
    "if len(event_galaxies) > 0:\n",
    "    event_additional_global_tags.append(event_galaxies)\n",
    "event_tags = pb_get_misp_tags(event_additional_global_tags, event_additional_local_tags)\n",
    "\n",
    "# Create the PyMISP object for an event\n",
    "event = MISPEvent()\n",
    "event.info = event_title\n",
    "event.distribution = event_distribution\n",
    "event.threat_level_id = event_threat_level_id\n",
    "event.analysis = event_analysis\n",
    "event.set_date(event_date)\n",
    "\n",
    "# Create the MISP event on the server side\n",
    "misp_event = misp.add_event(event, pythonify=True)\n",
    "print(\"Continue the playbook with the new \\033[92mcreated\\033[90m MISP event ID {} with title \\033[92m{}\\033[90m and UUID {}\".format(misp_event.id, misp_event.info, misp_event.uuid))\n",
    "for tag in event_tags:\n",
    "    if len(tag) > 0:\n",
    "        misp.tag(misp_event.uuid, tag, local=tag.local)\n",
    "        print(\" \\033[92mAdded\\033[90m event tag {}\".format(tag))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2243ae8b-cb77-4835-a0cd-2738ee8c9bec",
   "metadata": {},
   "source": [
    "# Investigate"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9da9ca98-401e-4205-a31b-1a571f681f9b",
   "metadata": {},
   "source": [
    "## IN:1 Context details for the IPs\n",
    "\n",
    "Set the contextualisation (tags) for the IPs with `attribute_tags`. These tags are added to the IPs, and their corresponding reverse DNS entries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "171aee2b-bf8b-4668-9992-3c64f04e7ec3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "attribute_tags = pb_get_misp_tags([\"PAP:GREEN\", \"course-of-action:passive=\\\"discover\\\"\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4be95a1-67ee-4c5a-85e3-a9607f335f40",
   "metadata": {},
   "source": [
    "## IN:2 Add the IPs to the event\n",
    "\n",
    "Add the IPs to the MISP event."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "d17bcaf4-07d9-4329-bfb4-ec04a1530294",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Add IPs to MISP event.\n",
      " Added domain-ip object for \u001b[92m91.219.237.56\u001b[90m with UUID 7d7faf28-866b-4830-9883-229e093ee67d\n",
      " Added domain-ip object for \u001b[92m185.194.93.128\u001b[90m with UUID 140bd150-79d5-4f6f-bbb4-504e5f6eada2\n"
     ]
    }
   ],
   "source": [
    "print(\"Add IPs to MISP event.\")\n",
    "for ip in query:\n",
    "    domain_ip_object = MISPObject(\"domain-ip\")\n",
    "    domain_ip_object.add_attribute(\"ip\", ip, tags=attribute_tags, comment=\"Added by playbook\")\n",
    "    result = misp.add_object(misp_event.uuid, domain_ip_object, pythonify=True)\n",
    "    if not \"errors\" in result:\n",
    "        case_objects[ip] = result\n",
    "        playbook_results[ip] = []\n",
    "        print(\" Added domain-ip object for \\033[92m{}\\033[90m with UUID {}\".format(ip, result.uuid))\n",
    "    else:\n",
    "        print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7aefa562-90fb-449d-be2e-cccf76060e74",
   "metadata": {},
   "source": [
    "# Correlation\n",
    "\n",
    "## CR:1 Correlation with MISP events\n",
    "\n",
    "Search the **MISP server** for events that match with one of the IPs you specified in `query`. A summary is also shown at the end of the playbook.\n",
    "\n",
    "Only **published** events (`correlation_published`) and attributes that have the **to_ids** flag (`correlation_to_ids`) are considered. You can limit the number of results (`correlation_limit`) and limit the search with tags (`correlation_match_tags`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "69ad9cc1-ab8a-4bc1-a485-11565b849785",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Only query for published MISP events\n",
    "correlation_published = None #True\n",
    "\n",
    "# Only consider those values that have the to_ids field set to True\n",
    "correlation_to_ids = True\n",
    "\n",
    "# Limit the returned results. Use \"None\" to include all results\n",
    "correlation_limit = None\n",
    "\n",
    "# Only return results corresponding with these tags\n",
    "correlation_match_tags = [\"tlp:amber\", \"tlp:white\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "ba6ac8f8-cd37-4db6-be8a-12e39275d63b",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Search for correlating MISP events\n",
      "Finished searching for correlations\n"
     ]
    }
   ],
   "source": [
    "print(\"Search for correlating MISP events\")\n",
    "# Code block to query MISP and find the correlations\n",
    "if len(query) > 0:\n",
    "    search_match = misp.search(\"attributes\", to_ids=correlation_to_ids, value=query, tags=correlation_match_tags,\n",
    "                                   published=correlation_published, limit=correlation_limit, pythonify=True)\n",
    "    if len(search_match) > 0:\n",
    "        for attribute in search_match:\n",
    "            if attribute.Event.id != misp_event.id:   # Skip the event we just created for this playbook\n",
    "                print(\" Found \\033[92m{}\\033[90m in event \\033[92m{}\\033[90m - \\033[92m{}\\033[90m\".format(attribute.value, attribute.event_id, attribute.Event.info))\n",
    "                entry = {\"source\": \"MISP\", \"category\": attribute.category, \"type\": attribute.type, \"event_id\": attribute.Event.id, \"event_info\": attribute.Event.info, \"enriched\": attribute.value}\n",
    "                playbook_results = pb_add_enrichment(playbook_results, attribute.value, entry, \"event_id\", attribute.Event.id)\n",
    "print(\"Finished searching for correlations\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c10d315-123f-4622-b764-13c1a036ebb5",
   "metadata": {},
   "source": [
    "### MISP events correlation table\n",
    "\n",
    "The correlation results are now stored in `playbook_results`. Execute the next cell to display them in a table format. The table is also included in the summary for Mattermost and TheHive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "7d098b83-1733-4804-939e-c9d370d8e596",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+-------+----------+------+-------+----------+\n",
      "| Source | Value | Category | Type | Event | Event ID |\n",
      "+--------+-------+----------+------+-------+----------+\n",
      "+--------+-------+----------+------+-------+----------+\n"
     ]
    }
   ],
   "source": [
    "# Put the correlations in a pretty table. We can use this table later also for the summary\n",
    "table = PrettyTable()\n",
    "table.field_names = [\"Source\", \"Value\", \"Category\", \"Type\", \"Event\", \"Event ID\"]\n",
    "table.align[\"Value\"] = \"l\"\n",
    "table.align[\"Category\"] = \"l\"\n",
    "table.align[\"Type\"] = \"l\"\n",
    "table.align[\"Event\"] = \"l\"\n",
    "table.align[\"Event ID\"] = \"l\"\n",
    "table._max_width = {\"Event\": 50}\n",
    "for ip in playbook_results:\n",
    "    for match in playbook_results[ip]:\n",
    "        if match[\"source\"] == \"MISP\":\n",
    "            table.add_row([match[\"source\"], ip, match[\"category\"], match[\"type\"], match[\"event_info\"], match[\"event_id\"]])\n",
    "print(table.get_string(sortby=\"Value\"))\n",
    "table_mispevents = table"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0469a986-b85e-4f99-ac16-8957447ec76e",
   "metadata": {},
   "source": [
    "## CR:2 Correlation with MISP feeds\n",
    "\n",
    "Search the **MISP feeds** for events that match with one of the IPs you specified in `query`. The results highly depend on the feeds you have enabled. For good results, it's advised you enable at least\n",
    "- **Tor exit nodes**\n",
    "- **Feodo IP Blocklist**\n",
    "\n",
    "Be sure to check if other [MISP feeds](https://www.misp-project.org/feeds/) provide valuable results for your environment as well. Note that the correlation lookup in the MISP feeds does not return the name of the MISP event, it returns the UUID of the event as title."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "21a234ec-0ecd-493c-bfac-c1c6aaa6ce4d",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Search in MISP feeds\n",
      "Found \u001b[92m91.219.237.56\u001b[90m in \u001b[92mTor exit nodes\u001b[90m\n",
      "Finished searching in MISP feeds\n"
     ]
    }
   ],
   "source": [
    "print(\"Search in MISP feeds\")\n",
    "if len(query) > 0:\n",
    "    misp_cache_url = \"{}/feeds/searchCaches/\".format(misp_url)\n",
    "    match = False\n",
    "    for ip in query:\n",
    "        # Instead of GET, use POST (https://github.com/MISP/MISP/issues/7478)\n",
    "        cache_results = requests.post(misp_cache_url, headers=misp_headers, verify=misp_verifycert, json={\"value\": ip})\n",
    "        for result in cache_results.json():\n",
    "            if \"Feed\" in result:\n",
    "                match = True\n",
    "                print(\"Found \\033[92m{}\\033[90m in \\033[92m{}\\033[90m\".format(ip, result[\"Feed\"][\"name\"]))\n",
    "                for match in result[\"Feed\"][\"direct_urls\"]:\n",
    "                    entry = {\"source\": \"Feeds\", \"feed_name\": result[\"Feed\"][\"name\"], \"match_url\": match[\"url\"]}\n",
    "                    playbook_results = pb_add_enrichment(playbook_results, ip, entry, \"match_url\", match[\"url\"])\n",
    "print(\"Finished searching in MISP feeds\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "597185dd-bfd2-4dbb-9a93-46dff4f68945",
   "metadata": {},
   "source": [
    "### MISP feed correlations table\n",
    "\n",
    "The correlation results are now stored in `playbook_results`. Execute the next cell to display them in a table format. The table is also included in the summary for Mattermost and TheHive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "0e1734e3-b012-44ab-b6fa-d7090758bdaf",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+---------------+----------------+--------------------------------------------------+\n",
      "| Source | Value         | Feed           |                       URL                        |\n",
      "+--------+---------------+----------------+--------------------------------------------------+\n",
      "| Feeds  | 91.219.237.56 | Tor exit nodes | https://misp.demo.cudeso.be/feeds/previewIndex/5 |\n",
      "+--------+---------------+----------------+--------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "# Put the correlations in a pretty table. We can use this table later also for the summary\n",
    "table = PrettyTable()\n",
    "table.field_names = [\"Source\", \"Value\", \"Feed\", \"URL\"]\n",
    "table.align[\"Value\"] = \"l\"\n",
    "table.align[\"Feed\"] = \"l\"\n",
    "table.align[\"Feed URL\"] = \"l\"\n",
    "table._max_width = {\"Feed\": 50}\n",
    "for ip in playbook_results:\n",
    "    for match in playbook_results[ip]:\n",
    "        if match[\"source\"] == \"Feeds\":\n",
    "            table.add_row([match[\"source\"], ip, match[\"feed_name\"], match[\"match_url\"]])\n",
    "print(table.get_string(sortby=\"Value\"))\n",
    "table_mispfeeds = table"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0540bacc-19b1-4efd-840f-7bc099a4e789",
   "metadata": {},
   "source": [
    "# Enrichment"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ab4bd03-1000-4ed8-a37e-2ad420db9ba0",
   "metadata": {
    "tags": []
   },
   "source": [
    "## ER:1 Enrich with DNS information\n",
    "\n",
    "Query the ReverseDNS module and add the **reverse DNS** results to the previously created `domain-ip` object(s). This module uses the default DNS resolver defined in the MISP module (8.8.8.8) but you can specify your own DNS server with `module_dnsserver`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "c139b5cb-51bb-4043-aabf-764462f1c39b",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start reversedns.\n",
      "Query \u001b[92m91.219.237.56\u001b[90m\n",
      " Got \u001b[92mqeohmonlocbe-dedicated.serverastra.com\u001b[90m\n",
      "Query \u001b[92m185.194.93.128\u001b[90m\n",
      " Got \u001b[92mcppz.circl.lu\u001b[90m\n",
      "Finished reversedns.\n"
     ]
    }
   ],
   "source": [
    "# DNS server to use. Leave blank to use 8.8.8.8 (default MISP module)\n",
    "module_dnsserver = \"\"\n",
    "\n",
    "module_name = \"reversedns\"\n",
    "attribute_type = \"ip-dst\"\n",
    "\n",
    "print(\"Start {}.\".format(module_name))\n",
    "for value in query:\n",
    "    module_comment = \"From {} for {}\".format(module_name, value)    \n",
    "    data = {\n",
    "        attribute_type: f\"{value}\",\n",
    "        \"module\": module_name\n",
    "    }\n",
    "    if len(module_dnsserver) > 0:\n",
    "        data[\"config\"] = { \"nameserver\": module_dnsserver }\n",
    "    print(\"Query \\033[92m{}\\033[90m\".format(value))    \n",
    "    result = requests.post(\"{}/query\".format(misp_modules_url), headers=misp_modules_headers, json=data)\n",
    "    if \"results\" in result.json() and len(result.json()[\"results\"]) > 0:\n",
    "        result_json = result.json()[\"results\"]\n",
    "        for entry in result_json:\n",
    "            if \"values\" in entry:\n",
    "                for module_value in entry[\"values\"]:\n",
    "                    # Avoid hostname validation errors when ending in a \".\"\n",
    "                    if module_value.endswith('.'):\n",
    "                        module_value = module_value[:-1]\n",
    "                    ip_object = misp.get_object(case_objects[value].uuid, pythonify=True)\n",
    "                    ip_object.add_attribute(\"hostname\", module_value, comment=module_comment, tags=attribute_tags)\n",
    "                    ip_object_misp = misp.update_object(ip_object, pythonify=True)\n",
    "                    if not \"errors\" in ip_object_misp:\n",
    "                        print(\" Got \\033[92m{}\\033[90m\".format(module_value))\n",
    "                        entry = {\"source\": module_name, \"category\": \"Network activity\", \"type\": \"hostname\", \"enriched\": module_value}\n",
    "                        playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", module_value)\n",
    "\n",
    "print(\"Finished {}.\".format(module_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70bc578c-b648-45d4-becb-9630db8c5650",
   "metadata": {},
   "source": [
    "## ER:2 Enrich with abuse information\n",
    "\n",
    "Use [abuse_finder](https://github.com/certsocietegenerale/abuse_finder) information to get the most appropriate contact details for **abuse reports** and add them as MISP attributes (either `whois-registrant-name` or `whois-registrant-email`).\n",
    "\n",
    "Note that the current Python module for abuse_finder does not yet [support IPv6](https://github.com/certsocietegenerale/abuse_finder/pull/5)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "6c9bcba7-2bc4-4716-bc94-f5d41e46c208",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start abuse_finder.\n",
      "Trying \u001b[92m91.219.237.56\u001b[90m\n",
      " Got \u001b[92mabuse@serverastra.com\u001b[90m\n",
      "Trying \u001b[92m185.194.93.128\u001b[90m\n",
      " Got \u001b[92minfo@circl.lu\u001b[90m\n",
      "Finished abuse_finder.\n"
     ]
    }
   ],
   "source": [
    "# Relationship type for abuse records\n",
    "relationtype = \"associated-with\"\n",
    "\n",
    "module_name = \"abuse_finder\"\n",
    "\n",
    "# Lookup abuse details for the IPs\n",
    "print(\"Start {}.\".format(module_name))\n",
    "for value in query:\n",
    "    ipv6_pattern = r'^(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]))$'\n",
    "    if re.match(ipv6_pattern, value) is not None:\n",
    "        print(\"Current abuse_finder module does not support IPv6.\")\n",
    "        continue\n",
    "\n",
    "    module_comment = \"From {} for {}\".format(module_name, value)\n",
    "    print(\"Trying \\033[92m{}\\033[90m\".format(value))\n",
    "    details = ip_abuse(value)\n",
    "    if \"names\" in details:\n",
    "        attribute_category = \"Attribution\"\n",
    "        attribute_type = \"whois-registrant-name\"\n",
    "        for name in details[\"names\"]:\n",
    "            print(\" Got \\033[92m{}\\033[90m\".format(name))\n",
    "            entry = {\"source\": module_name, \"category\": attribute_category, \"type\": attribute_type, \"enriched\": name}\n",
    "            playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", name)\n",
    "            attribute = MISPAttribute()\n",
    "            attribute.value = name\n",
    "            attribute.to_ids = False\n",
    "            attribute.category = attribute_category\n",
    "            attribute.type = attribute_type\n",
    "            attribute.disable_correlation = True\n",
    "            attribute.comment = \"{}. Information for {}\".format(module_comment, value)\n",
    "            attribute_misp = misp.add_attribute(misp_event.uuid, attribute, pythonify=True)\n",
    "            if not \"errors\" in attribute_misp:\n",
    "                misp.add_object_reference(case_objects[value].add_reference(attribute_misp.uuid, relationtype))\n",
    "            else:\n",
    "                print(attribute_misp)\n",
    "    if \"abuse\" in details:\n",
    "        attribute_category = \"Attribution\"\n",
    "        attribute_type = \"whois-registrant-email\"\n",
    "        for name in details[\"abuse\"]:\n",
    "            print(\" Got \\033[92m{}\\033[90m\".format(name))\n",
    "            entry = {\"source\": module_name, \"category\": \"Attribution\", \"type\": \"whois-registrant-email\", \"enriched\": name}\n",
    "            playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", name)\n",
    "            attribute = MISPAttribute()\n",
    "            attribute.value = name\n",
    "            attribute.to_ids = False\n",
    "            attribute.category = attribute_category\n",
    "            attribute.type = attribute_type\n",
    "            attribute.disable_correlation = True\n",
    "            attribute.comment = \"{}. Information for {}\".format(module_comment, value)\n",
    "            attribute_misp = misp.add_attribute(misp_event.uuid, attribute, pythonify=True)\n",
    "            if not \"errors\" in attribute_misp:\n",
    "                misp.add_object_reference(case_objects[value].add_reference(attribute_misp.uuid, relationtype))\n",
    "            else:\n",
    "                print(attribute_misp)\n",
    "\n",
    "print(\"Finished {}.\".format(module_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03d862f0-2462-49fe-b83d-f5f96bae907a",
   "metadata": {},
   "source": [
    "## ER:3 Enrich with information from VirusTotal\n",
    "\n",
    "Next we query [VirusTotal](https://www.virustotal.com/gui/). First we use the MISP module because it returns MISP attributes and objects that can then be easily added to the event.\n",
    "\n",
    "Then we query VirusTotal directly (without using the module) to get the **tags**, **reputation**, **last analysis stats** and **alternative names** from certiicates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "51e58415-82b2-4144-ae97-2cb74705991b",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start virustotal_public.\n",
      "Query \u001b[92m91.219.237.56\u001b[90m as \u001b[92mip-dst\u001b[90m\n",
      " Got \u001b[92mvirustotal-report\u001b[90m \n",
      " Got \u001b[92masn\u001b[90m \n",
      " Got \u001b[92mdomain-ip\u001b[90m \n",
      " Query VirusTotal directly to get IP details\n",
      " Got tags: \u001b[92m['tor']\u001b[90m \n",
      " Got last_analysis_stats: \u001b[92m{'harmless': 62, 'malicious': 8, 'suspicious': 2, 'undetected': 17, 'timeout': 0}\u001b[90m \n",
      " Finished query VirusTotal directly\n",
      "Query \u001b[92m185.194.93.128\u001b[90m as \u001b[92mip-dst\u001b[90m\n",
      " Got \u001b[92mvirustotal-report\u001b[90m \n",
      " Got \u001b[92masn\u001b[90m \n",
      " Got \u001b[92mdomain-ip\u001b[90m \n",
      " Query VirusTotal directly to get IP details\n",
      " Got last_analysis_stats: \u001b[92m{'harmless': 49, 'malicious': 0, 'suspicious': 0, 'undetected': 41, 'timeout': 0}\u001b[90m \n",
      " Got certificate alt name \u001b[92mclapper.misp-community.org\u001b[90m \n",
      " Not adding to playbook results because of duplicate. Already added via virustotal_public\n",
      " Finished query VirusTotal directly\n",
      "Finished virustotal_public.\n"
     ]
    }
   ],
   "source": [
    "# Code block to query VirusTotal\n",
    "\n",
    "module_name = \"virustotal_public\"\n",
    "attribute_type = \"ip-dst\"\n",
    "scan_count = 0\n",
    "\n",
    "headers = {\n",
    "    \"accept\": \"application/json\",\n",
    "    \"x-apikey\": virustotal_apikey\n",
    "}\n",
    "\n",
    "print(\"Start {}.\".format(module_name))\n",
    "\n",
    "for value in query:\n",
    "    module_comment = \"From {} for {}\".format(module_name, value)\n",
    "\n",
    "    if attribute_type in misp_modules[module_name][\"input\"]:\n",
    "        data = {\n",
    "            \"attribute\": {\n",
    "                \"type\": f\"{attribute_type}\",\n",
    "                \"uuid\": str(uuid.uuid4()),\n",
    "                \"value\": f\"{value}\",\n",
    "            },\n",
    "            \"module\": module_name,\n",
    "            \"config\": {\"apikey\": virustotal_apikey}\n",
    "        }\n",
    "        print(\"Query \\033[92m{}\\033[90m as \\033[92m{}\\033[90m\".format(value, attribute_type))\n",
    "        result = requests.post(\"{}/query\".format(misp_modules_url), headers=misp_modules_headers, json=data)\n",
    "        if \"results\" in result.json() and len(result.json()[\"results\"]) > 0:\n",
    "            result_json = result.json()[\"results\"]\n",
    "            for misp_attribute in result_json.get(\"Attribute\", []):\n",
    "                misp_attribute[\"comment\"] = \"{}{}\".format(module_comment, misp_attribute.get(\"comment\", \"\"))\n",
    "                created_attribute = misp.add_attribute(misp_event.uuid, misp_attribute, pythonify=True)\n",
    "                if not \"errors\" in created_attribute:\n",
    "                    misp.add_object_reference(case_objects[value].add_reference(created_attribute.uuid, \"related-to\"))\n",
    "                    print(\" Got {} \\033[92m{}\\033[90m\".format(misp_attribute[\"type\"], misp_attribute[\"value\"]))\n",
    "                    entry = {\"source\": module_name, \"category\": misp_attribute[\"category\"], \"type\": misp_attribute[\"type\"], \"enriched\": misp_attribute[\"value\"]}\n",
    "                    playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", misp_attribute[\"value\"])\n",
    "                else:\n",
    "                    print(\" Unable to add {} \\033[92m{}\\033[90m to MISP event\".format(misp_attribute[\"type\"], misp_attribute[\"value\"]))\n",
    "            for misp_object in result_json.get(\"Object\", []):\n",
    "                misp_object[\"comment\"] = \"{}{}\".format(module_comment, misp_object.get(\"comment\", \"\"))\n",
    "                if \"Attribute\" in misp_object and len(misp_object[\"Attribute\"]) > 0:\n",
    "                    created_object = misp.add_object(misp_event.uuid, misp_object, pythonify=True)\n",
    "                    if not \"errors\" in created_object:\n",
    "                        for misp_attribute in misp_object.get(\"Attribute\", []):\n",
    "                            entry = {\"source\": module_name, \"category\": misp_attribute[\"object_relation\"], \"type\": misp_attribute[\"type\"], \"enriched\": misp_attribute[\"value\"]}\n",
    "                            playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", misp_attribute[\"value\"])\n",
    "                        print(\" Got \\033[92m{}\\033[90m \".format(misp_object[\"name\"]))\n",
    "                        misp.add_object_reference(case_objects[value].add_reference(created_object.uuid, \"related-to\"))\n",
    "                    else:\n",
    "                        print(\" Unable to add \\033[92m{}\\033[90m to MISP event\".format(misp_object[\"name\"]))\n",
    "\n",
    "        #################################################################\n",
    "\n",
    "        print(\" Query VirusTotal directly to get IP details\")\n",
    "        vt_url = \"https://www.virustotal.com/api/v3/ip_addresses/{}\".format(value)\n",
    "        result = requests.get(vt_url, headers=headers)\n",
    "        if \"data\" in result.json() and len(result.json()[\"data\"]) > 0:\n",
    "            result_json = result.json()[\"data\"][\"attributes\"]\n",
    "\n",
    "            if result_json.get(\"tags\", False):\n",
    "                print(\" Got tags: \\033[92m{}\\033[90m \".format(result_json.get(\"tags\")))\n",
    "                entry = {\"source\": module_name, \"category\": \"tags\", \"type\": \"tags\", \"enriched\": result_json.get(\"tags\", [])}\n",
    "                playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", result_json.get(\"tags\", []))\n",
    "\n",
    "            if result_json.get(\"last_analysis_stats\", False):\n",
    "                print(\" Got last_analysis_stats: \\033[92m{}\\033[90m \".format(result_json.get(\"last_analysis_stats\")))\n",
    "                entry = {\"source\": module_name, \"category\": \"last_analysis_stats\", \"type\": \"last_analysis_stats\", \"enriched\": result_json.get(\"last_analysis_stats\", [])}\n",
    "                playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", result_json.get(\"last_analysis_stats\", []))\n",
    "\n",
    "            if result_json.get(\"reputation\", False):\n",
    "                print(\" Got \\033[92m{}\\033[90m \".format(result_json.get(\"reputation\")))\n",
    "                entry = {\"source\": module_name, \"category\": \"reputation\", \"type\": \"reputation\", \"enriched\": result_json.get(\"reputation\", 0)}\n",
    "                playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", result_json.get(\"reputation\", []))\n",
    "\n",
    "            if result_json.get(\"last_https_certificate\", False):\n",
    "                subject_alternative_name = result_json.get(\"last_https_certificate\", []).get(\"extensions\", []).get(\"subject_alternative_name\", [])\n",
    "                for alt_name in subject_alternative_name:\n",
    "                    print(\" Got certificate alt name \\033[92m{}\\033[90m \".format(alt_name))\n",
    "                    entry = {\"source\": module_name, \"category\": \"domain\", \"type\": \"domain\", \"enriched\": alt_name}\n",
    "                    playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", alt_name)\n",
    "        print(\" Finished query VirusTotal directly\")\n",
    "\n",
    "        scan_count += 1\n",
    "        if scan_count > 5:\n",
    "            print(\"Sleeping for {} seconds\".format(misp_modules_wait))\n",
    "            time.sleep(misp_modules_wait)\n",
    "            scan_count = 0\n",
    "    else:\n",
    "        print(\"Skipping \\033[91m{}\\033[90m. Not a valid query type ({}).\".format(value, misp_modules[module_name][\"input\"]))\n",
    "\n",
    "print(\"Finished {}.\".format(module_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c5c6b14-e6de-469a-8988-1f5b1e9ed3d2",
   "metadata": {},
   "source": [
    "### VirusTotal enrichment table\n",
    "\n",
    "The results are now stored in `playbook_results`. Execute the next cell to display them in a table format. The table is also included in the summary for Mattermost and TheHive.\n",
    "\n",
    "This table returns only those matches corresponding with the source **VirusTotal**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "4b6cd04e-7361-4c24-9305-7c34a1b909c1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-------------------+----------------+---------------------+---------------------+----------------------------------------------------+\n",
      "|       Source      | Value          | Category            | Type                | Enriched                                           |\n",
      "+-------------------+----------------+---------------------+---------------------+----------------------------------------------------+\n",
      "| virustotal_public | 185.194.93.128 | asn                 | AS                  | 197869                                             |\n",
      "| virustotal_public | 185.194.93.128 | country             | text                | LU                                                 |\n",
      "| virustotal_public | 185.194.93.128 | detection-ratio     | text                | 0/90                                               |\n",
      "| virustotal_public | 185.194.93.128 | domain              | domain              | 2023.hack.lu                                       |\n",
      "| virustotal_public | 185.194.93.128 | domain              | domain              | clapper.misp-community.org                         |\n",
      "| virustotal_public | 185.194.93.128 | domain              | domain              | demo.misp-community.org                            |\n",
      "| virustotal_public | 185.194.93.128 | domain              | domain              | doc.cerebrate-project.org                          |\n",
      "| virustotal_public | 185.194.93.128 | domain              | domain              | hack.lu                                            |\n",
      "| virustotal_public | 185.194.93.128 | domain              | domain              | misp-lea.org                                       |\n",
      "| virustotal_public | 185.194.93.128 | domain              | domain              | monitor.circl.lu                                   |\n",
      "| virustotal_public | 185.194.93.128 | domain              | domain              | phishtankapi.circl.lu                              |\n",
      "| virustotal_public | 185.194.93.128 | domain              | domain              | typosquatting-finder.circl.lu                      |\n",
      "| virustotal_public | 185.194.93.128 | domain              | domain              | www.hack.lu                                        |\n",
      "| virustotal_public | 185.194.93.128 | ip                  | ip-dst              | 185.194.93.128                                     |\n",
      "| virustotal_public | 185.194.93.128 | last_analysis_stats | last_analysis_stats | {'harmless': 49, 'malicious': 0, 'suspicious': 0,  |\n",
      "|                   |                |                     |                     | 'undetected': 41, 'timeout': 0}                    |\n",
      "| virustotal_public | 185.194.93.128 | permalink           | link                | https://www.virustotal.com/gui/ip_address/185.194. |\n",
      "|                   |                |                     |                     | 93.128                                             |\n",
      "| virustotal_public | 185.194.93.128 | subnet-announced    | ip-src              | 185.194.92.0/22                                    |\n",
      "| virustotal_public | 91.219.237.56  | asn                 | AS                  | 56322                                              |\n",
      "| virustotal_public | 91.219.237.56  | country             | text                | HU                                                 |\n",
      "| virustotal_public | 91.219.237.56  | detection-ratio     | text                | 8/89                                               |\n",
      "| virustotal_public | 91.219.237.56  | domain              | domain              | already5.ddns.net                                  |\n",
      "| virustotal_public | 91.219.237.56  | domain              | domain              | brightsolarpower.net.http.s11.wbprx.com            |\n",
      "| virustotal_public | 91.219.237.56  | domain              | domain              | facebook.com.http.s11.wbprx.com                    |\n",
      "| virustotal_public | 91.219.237.56  | domain              | domain              | facebook.com.https.s11.wbprx.com                   |\n",
      "| virustotal_public | 91.219.237.56  | domain              | domain              | lolol.hopto.org                                    |\n",
      "| virustotal_public | 91.219.237.56  | domain              | domain              | no-hostname.serverastra.com                        |\n",
      "| virustotal_public | 91.219.237.56  | domain              | domain              | podpricelom.com.ua.s11.wbprx.com                   |\n",
      "| virustotal_public | 91.219.237.56  | domain              | domain              | secure.gravatar.com.https.s11.wbprx.com            |\n",
      "| virustotal_public | 91.219.237.56  | domain              | domain              | simaiiii1.ddns.net                                 |\n",
      "| virustotal_public | 91.219.237.56  | domain              | domain              | www.brightsolarpower.net.http.s11.wbprx.com        |\n",
      "| virustotal_public | 91.219.237.56  | ip                  | ip-dst              | 91.219.237.56                                      |\n",
      "| virustotal_public | 91.219.237.56  | last_analysis_stats | last_analysis_stats | {'harmless': 62, 'malicious': 8, 'suspicious': 2,  |\n",
      "|                   |                |                     |                     | 'undetected': 17, 'timeout': 0}                    |\n",
      "| virustotal_public | 91.219.237.56  | permalink           | link                | https://www.virustotal.com/gui/ip_address/91.219.2 |\n",
      "|                   |                |                     |                     | 37.56                                              |\n",
      "| virustotal_public | 91.219.237.56  | subnet-announced    | ip-src              | 91.219.236.0/22                                    |\n",
      "| virustotal_public | 91.219.237.56  | tags                | tags                | ['tor']                                            |\n",
      "+-------------------+----------------+---------------------+---------------------+----------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "# Put the correlations in a pretty table. We can use this table later also for the summary\n",
    "table = PrettyTable()\n",
    "table.field_names = [\"Source\", \"Value\", \"Category\", \"Type\", \"Enriched\"]\n",
    "table.align[\"Value\"] = \"l\"\n",
    "table.align[\"Category\"] = \"l\"\n",
    "table.align[\"Type\"] = \"l\"\n",
    "table.align[\"Enriched\"] = \"l\"\n",
    "table._max_width = {\"Enriched\": 50}\n",
    "for value in playbook_results:\n",
    "    for match in playbook_results[value]:\n",
    "        if match[\"source\"] == \"virustotal_public\":\n",
    "            table.add_row([match[\"source\"], value, match[\"category\"], match[\"type\"], match[\"enriched\"]])\n",
    "print(table.get_string(sortby=\"Value\"))\n",
    "table_virustotal = table"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9b6b0f4-d99d-4db1-8bee-944466c6cfaf",
   "metadata": {},
   "source": [
    "## ER:4 Enrich with information from Shodan\n",
    "\n",
    "Use **Shodan** to get more information associated with the IPs defined in `query`, such as **ASN**, **open ports** or discovered **vulnerabilities**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "c6819a95-3f4a-4911-ab1f-442ddfcff900",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start shodan.\n",
      " Query \u001b[92m91.219.237.56\u001b[90m as \u001b[92mip-dst\u001b[90m\n",
      " Got \u001b[92mip-api-address\u001b[90m \n",
      " Not adding to playbook results because of duplicate. Already added via virustotal_public\n",
      " Not adding to playbook results because of duplicate. Already added via shodan\n",
      " Not adding to playbook results because of duplicate. Already added via virustotal_public\n",
      " Got \u001b[92mip-port\u001b[90m \n",
      " Not adding to playbook results because of duplicate. Already added via virustotal_public\n",
      " Not adding to playbook results because of duplicate. Already added via reversedns\n",
      " Got \u001b[92mx509\u001b[90m \n",
      " Query \u001b[92m185.194.93.128\u001b[90m as \u001b[92mip-dst\u001b[90m\n",
      " Got \u001b[92mip-api-address\u001b[90m \n",
      " Not adding to playbook results because of duplicate. Already added via virustotal_public\n",
      " Not adding to playbook results because of duplicate. Already added via shodan\n",
      " Not adding to playbook results because of duplicate. Already added via shodan\n",
      " Not adding to playbook results because of duplicate. Already added via virustotal_public\n",
      " Not adding to playbook results because of duplicate. Already added via virustotal_public\n",
      " Got \u001b[92mip-port\u001b[90m \n",
      " Not adding to playbook results because of duplicate. Already added via virustotal_public\n",
      " Not adding to playbook results because of duplicate. Already added via reversedns\n",
      " Not adding to playbook results because of duplicate. Already added via virustotal_public\n",
      " Got \u001b[92mx509\u001b[90m \n",
      "Finished shodan.\n"
     ]
    }
   ],
   "source": [
    "# Code block to query Shodan\n",
    "\n",
    "module_name = \"shodan\"\n",
    "attribute_type = \"ip-dst\"\n",
    "\n",
    "print(\"Start {}.\".format(module_name))\n",
    "for value in query:\n",
    "    module_comment = \"From {} for {}\".format(module_name, value)\n",
    "\n",
    "    if attribute_type in misp_modules[module_name][\"input\"]:\n",
    "        data = {\n",
    "            \"attribute\": {\n",
    "                \"type\": f\"{attribute_type}\",\n",
    "                \"uuid\": str(uuid.uuid4()),\n",
    "                \"value\": f\"{value}\",\n",
    "            },\n",
    "            \"module\": module_name,\n",
    "            \"config\": {\"apikey\": shodan_apikey}\n",
    "        }\n",
    "        print(\" Query \\033[92m{}\\033[90m as \\033[92m{}\\033[90m\".format(value, attribute_type))\n",
    "        result = requests.post(\"{}/query\".format(misp_modules_url), headers=misp_modules_headers, json=data)\n",
    "        if \"results\" in result.json() and len(result.json()[\"results\"]) > 0:\n",
    "            result_json = result.json()[\"results\"]\n",
    "            for misp_object in result_json[\"Object\"]:\n",
    "                misp_object[\"comment\"] = \"{}{}\".format(module_comment, misp_object.get(\"comment\", \"\"))\n",
    "                if \"Attribute\" in misp_object and len(misp_object[\"Attribute\"]) > 0:\n",
    "                    if misp_object[\"name\"] == \"ip-api-address\":\n",
    "                        # ip-api-address requires a SRC-IP\n",
    "                        ip_src_attribute = {\n",
    "                            \"uuid\": str(uuid.uuid4()),\n",
    "                            \"object_relation\": \"ip-src\",\n",
    "                            \"value\": f\"{value}\",\n",
    "                            \"type\": \"ip-src\",\n",
    "                            \"disable_correlation\": 0,\n",
    "                            \"to_ids\": 1,\n",
    "                            \"category\": \"Network activity\"\n",
    "                        }\n",
    "                        misp_object[\"Attribute\"].append(ip_src_attribute)\n",
    "                    created_object = misp.add_object(misp_event.uuid, misp_object, pythonify=True)\n",
    "                    if not \"errors\" in created_object:\n",
    "                        print(\" Got \\033[92m{}\\033[90m \".format(misp_object[\"name\"]))\n",
    "                        for misp_attribute in misp_object.get(\"Attribute\", []):\n",
    "                            entry = {\"source\": module_name, \"category\": misp_attribute[\"category\"], \"type\": misp_attribute[\"type\"], \"enriched\": misp_attribute[\"value\"]}\n",
    "                            if misp_object[\"name\"] == \"ip-api-address\":\n",
    "                                if misp_attribute[\"object_relation\"] == \"city\":\n",
    "                                    entry = {\"source\": module_name, \"category\": \"ip-api-address\", \"type\": \"city\", \"enriched\": misp_attribute[\"value\"]}\n",
    "                                elif misp_attribute[\"object_relation\"] == \"ISP\":\n",
    "                                    entry = {\"source\": module_name, \"category\": \"ip-api-address\", \"type\": \"ISP\", \"enriched\": misp_attribute[\"value\"]}\n",
    "                                elif misp_attribute[\"object_relation\"] == \"organization\":\n",
    "                                    entry = {\"source\": module_name, \"category\": \"ip-api-address\", \"type\": \"organization\", \"enriched\": misp_attribute[\"value\"]}\n",
    "                            playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", misp_attribute[\"value\"])\n",
    "                            misp.add_object_reference(case_objects[value].add_reference(created_object.uuid, \"related-to\"))\n",
    "                    else:\n",
    "                        print(\" Unable to add \\033[92m{}\\033[90m to MISP event\".format(misp_object[\"name\"]))\n",
    "\n",
    "    else:\n",
    "        print(\"Skipping \\033[91m{}\\033[90m. Not a valid query type ({}).\".format(value, misp_modules[module_name][\"input\"]))\n",
    "\n",
    "print(\"Finished {}.\".format(module_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9159c241-8dc3-4d59-912e-7bab805d1fd1",
   "metadata": {},
   "source": [
    "### Shodan enrichment table\n",
    "\n",
    "The results are now stored in `playbook_results`. Execute the next cell to display them in a table format. The table is also included in the summary for Mattermost and TheHive.\n",
    "\n",
    "This table returns only those matches corresponding with the source **Shodan**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "4bfa6be2-082d-4728-a9f2-f959d8d5fe28",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------+----------------+------------------+-------------------------+----------------------------------------------------+\n",
      "| Source | Value          | Category         | Type                    | Enriched                                           |\n",
      "+--------+----------------+------------------+-------------------------+----------------------------------------------------+\n",
      "| shodan | 185.194.93.128 | Network activity | AS                      | AS197869                                           |\n",
      "| shodan | 185.194.93.128 | Network activity | domain                  | circl.lu                                           |\n",
      "| shodan | 185.194.93.128 | Network activity | domain                  | misp-community.org                                 |\n",
      "| shodan | 185.194.93.128 | Network activity | port                    | 443                                                |\n",
      "| shodan | 185.194.93.128 | Network activity | port                    | 53                                                 |\n",
      "| shodan | 185.194.93.128 | Network activity | port                    | 80                                                 |\n",
      "| shodan | 185.194.93.128 | Network activity | x509-fingerprint-sha1   | 96b2c7ff82d1eea417dfe67fac3baa9f6e1d0cc1           |\n",
      "| shodan | 185.194.93.128 | Network activity | x509-fingerprint-sha256 | 264a8df1cfae07e8995445ea4747fff64b9834842f9683aa59 |\n",
      "|        |                |                  |                         | f41cfb2adb6ba0                                     |\n",
      "| shodan | 185.194.93.128 | Other            | datetime                | 2023-10-28T09:00:14                                |\n",
      "| shodan | 185.194.93.128 | Other            | datetime                | 2024-01-26T09:00:13                                |\n",
      "| shodan | 185.194.93.128 | Other            | float                   | 6.13                                               |\n",
      "| shodan | 185.194.93.128 | Other            | float                   | 49.61167                                           |\n",
      "| shodan | 185.194.93.128 | Other            | text                    | /C=US/CN=R3/O=Let's Encrypt                        |\n",
      "| shodan | 185.194.93.128 | Other            | text                    | /CN=clapper.misp-community.org                     |\n",
      "| shodan | 185.194.93.128 | Other            | text                    | 2                                                  |\n",
      "| shodan | 185.194.93.128 | Other            | text                    | 2048                                               |\n",
      "| shodan | 185.194.93.128 | Other            | text                    | 299625435784545754531043904687971546037349         |\n",
      "| shodan | 185.194.93.128 | Other            | text                    | rsa                                                |\n",
      "| shodan | 185.194.93.128 | Other            | text                    | sha256WithRSAEncryption                            |\n",
      "| shodan | 185.194.93.128 | ip-api-address   | ISP                     | Luxembourg House of Cybersecurity                  |\n",
      "| shodan | 185.194.93.128 | ip-api-address   | city                    | Luxembourg                                         |\n",
      "| shodan | 91.219.237.56  | Network activity | AS                      | AS56322                                            |\n",
      "| shodan | 91.219.237.56  | Network activity | domain                  | serverastra.com                                    |\n",
      "| shodan | 91.219.237.56  | Network activity | port                    | 22                                                 |\n",
      "| shodan | 91.219.237.56  | Network activity | port                    | 443                                                |\n",
      "| shodan | 91.219.237.56  | Network activity | port                    | 80                                                 |\n",
      "| shodan | 91.219.237.56  | Network activity | x509-fingerprint-sha1   | 93ffc6f7b3435982596f3a23b6b50bcc4da27f75           |\n",
      "| shodan | 91.219.237.56  | Network activity | x509-fingerprint-sha256 | 03b4d92fe372e5b87353bdd822c352330783ec8f2e13c58212 |\n",
      "|        |                |                  |                         | 61dee669b79739                                     |\n",
      "| shodan | 91.219.237.56  | Other            | datetime                | 2023-07-29T00:00:00                                |\n",
      "| shodan | 91.219.237.56  | Other            | datetime                | 2023-11-18T00:00:00                                |\n",
      "| shodan | 91.219.237.56  | Other            | float                   | 19.04045                                           |\n",
      "| shodan | 91.219.237.56  | Other            | float                   | 47.49835                                           |\n",
      "| shodan | 91.219.237.56  | Other            | text                    | /CN=www.6stxgvoz.net                               |\n",
      "| shodan | 91.219.237.56  | Other            | text                    | /CN=www.6zr4dmrhfsqbrfker.com                      |\n",
      "| shodan | 91.219.237.56  | Other            | text                    | 14097940683106547645                               |\n",
      "| shodan | 91.219.237.56  | Other            | text                    | 2                                                  |\n",
      "| shodan | 91.219.237.56  | Other            | text                    | 2048                                               |\n",
      "| shodan | 91.219.237.56  | Other            | text                    | BU                                                 |\n",
      "| shodan | 91.219.237.56  | Other            | text                    | Hungary                                            |\n",
      "| shodan | 91.219.237.56  | Other            | text                    | rsa                                                |\n",
      "| shodan | 91.219.237.56  | Other            | text                    | sha256WithRSAEncryption                            |\n",
      "| shodan | 91.219.237.56  | ip-api-address   | ISP                     | ServerAstra Kft.                                   |\n",
      "| shodan | 91.219.237.56  | ip-api-address   | city                    | Budapest                                           |\n",
      "+--------+----------------+------------------+-------------------------+----------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "# Put the correlations in a pretty table. We can use this table later also for the summary\n",
    "table = PrettyTable()\n",
    "table.field_names = [\"Source\", \"Value\", \"Category\", \"Type\", \"Enriched\"]\n",
    "table.align[\"Value\"] = \"l\"\n",
    "table.align[\"Category\"] = \"l\"\n",
    "table.align[\"Type\"] = \"l\"\n",
    "table.align[\"Enriched\"] = \"l\"\n",
    "table._max_width = {\"Enriched\": 50}\n",
    "for value in playbook_results:\n",
    "    for match in playbook_results[value]:\n",
    "        if match[\"source\"] == \"shodan\":\n",
    "            table.add_row([match[\"source\"], value, match[\"category\"], match[\"type\"], match[\"enriched\"]])\n",
    "print(table.get_string(sortby=\"Value\"))\n",
    "table_shodan = table"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdd8c3b4-058d-4bba-b585-fc33724b8575",
   "metadata": {},
   "source": [
    "## ER:5 Get reputation from Greynoise\n",
    "\n",
    "Query [Greynoise](https://viz.greynoise.io/) to get the **reputation** of the IPs. The results indicate if the IP is part of [RIOT](https://docs.greynoise.io/docs/riot-data) (*a GreyNoise feature that informs users about IPs used by common business services that are almost certainly not attacking you*) or noise (*when the IP has been observed scanning the internet in the last 90 days*)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "1cebaada-0c93-4915-98c6-b4a02ec320b2",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start greynoise.\n",
      "Query \u001b[92m91.219.237.56\u001b[90m\n",
      " Got \u001b[92m{'ip': '91.219.237.56', 'noise': True, 'riot': False, 'classification': 'unknown', 'name': 'unknown', 'link': 'https://viz.greynoise.io/ip/91.219.237.56', 'last_seen': '2023-11-05', 'message': 'Success'}\u001b[90m \n",
      "Query \u001b[92m185.194.93.128\u001b[90m\n",
      " Got \u001b[92m{'ip': '185.194.93.128', 'noise': True, 'riot': False, 'classification': 'unknown', 'name': 'unknown', 'link': 'https://viz.greynoise.io/ip/185.194.93.128', 'last_seen': '2023-10-23', 'message': 'Success'}\u001b[90m \n",
      "Finished greynoise.\n"
     ]
    }
   ],
   "source": [
    "# Code block to query Greynoise\n",
    "\n",
    "greynoise_url = \"https://api.greynoise.io/v3/community/\"\n",
    "headers = {\n",
    "  'key': greynoise_apikey\n",
    "}\n",
    "module_name = \"greynoise\"\n",
    "print(\"Start {}.\".format(module_name))\n",
    "for value in query:\n",
    "    print(\"Query \\033[92m{}\\033[90m\".format(value))\n",
    "    response = requests.request(\"GET\", \"{}{}\".format(greynoise_url, value), headers=headers)\n",
    "    if response.status_code == 200:\n",
    "        result_json = response.json()\n",
    "        print(\" Got \\033[92m{}\\033[90m \".format(result_json))\n",
    "        entry = {\"source\": module_name, \"category\": \"domain\", \"type\": \"reputation\", \"enriched\": result_json}\n",
    "        playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", result_json)\n",
    "\n",
    "        ip_object = misp.get_object(case_objects[value].uuid, pythonify=True)\n",
    "        module_comment = \"From {} for {}\".format(module_name, value)\n",
    "        ip_object.add_attribute(\"text\", \"noise: {}\".format(result_json.get(\"noise\")), comment=module_comment, tags=[])\n",
    "        ip_object.add_attribute(\"text\", \"riot: {}\".format(result_json.get(\"riot\")), comment=module_comment, tags=[])\n",
    "        ip_object.add_attribute(\"text\", \"classification: {}\".format(result_json.get(\"classification\")), comment=module_comment, tags=[])\n",
    "        ip_object.add_attribute(\"text\", \"name: {}\".format(result_json.get(\"name\")), comment=module_comment, tags=[])\n",
    "        ip_object_misp = misp.update_object(ip_object, pythonify=True)\n",
    "    elif response.status_code == 404:\n",
    "        print(\" Received 404 message: \\033[91mIP not found\\033[90m.\")\n",
    "    elif response.status_code == 400:\n",
    "        print(\" Received 400 message: \\033[91mInvalid IP\\033[90m.\")\n",
    "    elif response.status_code == 429:\n",
    "        print(\" Received 429 message: \\033[91mRatelimit error\\033[90m.\")\n",
    "    else:\n",
    "        print(\" Received \\033[91munexpected error\\033[90m.\")\n",
    "print(\"Finished {}.\".format(module_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc6add72-7180-448c-a8dd-f1e8cb6cc43a",
   "metadata": {},
   "source": [
    "## ER:6 Get geo information from MMDB\n",
    "\n",
    "Add the **geolocation** information from an mmdb server, such as [ip.circl.lu](https://ip.circl.lu/geolookup/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "1df37ff8-1a5e-4eba-bc74-d6736ea6995e",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start mmdb.\n",
      "Query \u001b[92m91.219.237.56\u001b[90m\n",
      " Got iso_code \u001b[92mHU\u001b[90m \n",
      " Not adding to playbook results because of duplicate. Already added via virustotal_public\n",
      " Got latitude and longitude \u001b[92m47\u001b[90m \u001b[92m20\u001b[90m\n",
      "Query \u001b[92m185.194.93.128\u001b[90m\n",
      " Got iso_code \u001b[92mLU\u001b[90m \n",
      " Not adding to playbook results because of duplicate. Already added via virustotal_public\n",
      " Got latitude and longitude \u001b[92m49.75\u001b[90m \u001b[92m6.1667\u001b[90m\n",
      "Finished mmdb.\n"
     ]
    }
   ],
   "source": [
    "# Code block to query MMDB\n",
    "\n",
    "mmdb_url = \"https://ip.circl.lu/geolookup/\"\n",
    "headers = {}\n",
    "module_name = \"mmdb\"\n",
    "print(\"Start {}.\".format(module_name))\n",
    "for value in query:\n",
    "    print(\"Query \\033[92m{}\\033[90m\".format(value))\n",
    "    response = requests.request(\"GET\", \"{}{}\".format(mmdb_url, value), headers=headers)\n",
    "    if response.status_code == 200:\n",
    "        result_json = response.json()[0]\n",
    "        iso_code = result_json.get(\"country\", False).get(\"iso_code\", False)\n",
    "        lat = result_json.get(\"country_info\", False).get(\"Latitude (average)\", False)\n",
    "        long = result_json.get(\"country_info\", False).get(\"Longitude (average)\", False)\n",
    "\n",
    "        ip_object = misp.get_object(case_objects[value].uuid, pythonify=True)\n",
    "        module_comment = \"From {} for {}\".format(module_name, value)\n",
    "\n",
    "        if iso_code:\n",
    "            print(\" Got iso_code \\033[92m{}\\033[90m \".format(iso_code))\n",
    "            entry = {\"source\": module_name, \"category\": \"geo\", \"type\": \"iso_code\", \"enriched\": iso_code}\n",
    "            playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", iso_code)\n",
    "            ip_object.add_attribute(\"text\", \"iso_code: {}\".format(iso_code), comment=module_comment, tags=[])\n",
    "        if lat and long:\n",
    "            print(\" Got latitude and longitude \\033[92m{}\\033[90m \\033[92m{}\\033[90m\".format(lat, long))\n",
    "            entry = {\"source\": module_name, \"category\": \"geo\", \"type\": \"lat\", \"enriched\": lat}\n",
    "            playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", lat)\n",
    "            ip_object.add_attribute(\"text\", \"latitude: {}\".format(lat), comment=module_comment, tags=[])\n",
    "            entry = {\"source\": module_name, \"category\": \"geo\", \"type\": \"long\", \"enriched\": long}\n",
    "            playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", long)\n",
    "            ip_object.add_attribute(\"text\", \"longitude: {}\".format(long), comment=module_comment, tags=[])\n",
    "\n",
    "        ip_object_misp = misp.update_object(ip_object, pythonify=True)\n",
    "print(\"Finished {}.\".format(module_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e61060c-db05-441f-a683-f015b46be427",
   "metadata": {},
   "source": [
    "## ER:7 Enrich with information from AbuseIPDB\n",
    "\n",
    "Use additional information on the IP reputation from [AbuseIPDB](https://www.abuseipdb.com/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "eb2fdccc-2109-46c1-8cf6-bb2b81d02e84",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start abuseipdb.\n",
      "Query \u001b[92m91.219.237.56\u001b[90m\n",
      " Got abuseConfidenceScore \u001b[92m100\u001b[90m\n",
      " Got usageType \u001b[92mData Center/Web Hosting/Transit\u001b[90m\n",
      " Got totalReports \u001b[92m96\u001b[90m\n",
      "Query \u001b[92m185.194.93.128\u001b[90m\n",
      " Got abuseConfidenceScore \u001b[92m0\u001b[90m\n",
      " Got usageType \u001b[92mCommercial\u001b[90m\n",
      " Got totalReports \u001b[92m0\u001b[90m\n",
      " Not adding to playbook results because of duplicate. Already added via abuseipdb\n",
      "Finished abuseipdb.\n"
     ]
    }
   ],
   "source": [
    "# Code block to query AbuseIPDB\n",
    "\n",
    "abuseipdb_url = \"https://api.abuseipdb.com/api/v2/check\"\n",
    "headers = {\"Accept\": \"application/json\",\n",
    "           \"Key\": abuseipdb_apikey\n",
    "            }\n",
    "ipdb_maxage = 90\n",
    "module_name = \"abuseipdb\"\n",
    "print(\"Start {}.\".format(module_name))\n",
    "for value in query:\n",
    "    print(\"Query \\033[92m{}\\033[90m\".format(value))\n",
    "    querystring = {\n",
    "        \"ipAddress\": value,\n",
    "        \"maxAgeInDays\": ipdb_maxage\n",
    "    }\n",
    "    response = requests.request(\"GET\", url=abuseipdb_url, headers=headers, params=querystring)\n",
    "    if response.status_code == 200 and \"data\" in response.json():\n",
    "        result_json = response.json()[\"data\"]\n",
    "        abuseConfidenceScore = result_json.get(\"abuseConfidenceScore\")\n",
    "        print(\" Got abuseConfidenceScore \\033[92m{}\\033[90m\".format(abuseConfidenceScore))\n",
    "        entry = {\"source\": module_name, \"category\": \"abuseConfidenceScore\", \"type\": \"abuseConfidenceScore\", \"enriched\": abuseConfidenceScore}\n",
    "        playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", abuseConfidenceScore)\n",
    "\n",
    "        usageType = result_json.get(\"usageType\")\n",
    "        print(\" Got usageType \\033[92m{}\\033[90m\".format(usageType))\n",
    "        entry = {\"source\": module_name, \"category\": \"usageType\", \"type\": \"usageType\", \"enriched\": usageType}\n",
    "        playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", usageType)\n",
    "\n",
    "        totalReports = result_json.get(\"totalReports\")\n",
    "        print(\" Got totalReports \\033[92m{}\\033[90m\".format(totalReports))\n",
    "        entry = {\"source\": module_name, \"category\": \"totalReports\", \"type\": \"totalReports\", \"enriched\": totalReports}\n",
    "        playbook_results = pb_add_enrichment(playbook_results, value, entry, \"enriched\", totalReports)\n",
    "\n",
    "        ip_object = misp.get_object(case_objects[value].uuid, pythonify=True)\n",
    "        module_comment = \"From {} for {}\".format(module_name, value)\n",
    "        ip_object.add_attribute(\"text\", \"abuseConfidenceScore: {}\".format(abuseConfidenceScore), comment=module_comment, tags=[])\n",
    "        ip_object.add_attribute(\"text\", \"usageType: {}\".format(usageType), comment=module_comment, tags=[])\n",
    "        ip_object.add_attribute(\"text\", \"totalReports: {}\".format(totalReports), comment=module_comment, tags=[])\n",
    "        ip_object_misp = misp.update_object(ip_object, pythonify=True)\n",
    "print(\"Finished {}.\".format(module_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97b6e2d7-b6d6-4b71-8154-33c0cc219b48",
   "metadata": {},
   "source": [
    "## ER:8 Investigation and enrichment report\n",
    "\n",
    "The next cell summarises the investigation and enrichment details into a **MISP report**. The report is attached to the MISP event, and also used at the end of the playbook to create the summary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "6a6988c4-058b-41d9-96d1-0427dcfb72fd",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating MISP report \u001b[92mIP reputation\u001b[90m\n",
      " Report ID: \u001b[92m780\u001b[90m\n"
     ]
    }
   ],
   "source": [
    "summary_iv = \"## Investigation and enrichment report\\n\\n\"\n",
    "current_date = datetime.now()\n",
    "formatted_date = current_date.strftime(\"%Y-%m-%d\")\n",
    "virustotal = \"virustotal_public\"    # put in a variable in case we later want to change hits\n",
    "for key, sample in playbook_results.items():\n",
    "    summary_iv += \"## Analysis for {}\\n\".format(key)\n",
    "    summary_iv += \"- Date: **{}**\\n\".format(formatted_date)\n",
    "    summary_iv += \"- MISP event: **{}** ({})\\n\".format(misp_event.info, misp_event.id)\n",
    "    summary_iv += \"\\n\\n\"\n",
    "    summary_iv += \"### IP details\\n\"\n",
    "    summary_iv += \"- ASN: **{}**\\n\".format(get_playbook_result(key, virustotal, \"asn\", \"AS\"))\n",
    "    summary_iv += \"- ASN ownername: **{}**\\n\".format(get_playbook_result(key, \"abuse_finder\", \"Attribution\", \"whois-registrant-name\"))\n",
    "    summary_iv += \"- Subnet: **{}**\\n\".format(get_playbook_result(key, virustotal, \"subnet-announced\", \"ip-src\"))\n",
    "    summary_iv += \"- ISP: **{}**\\n\".format(get_playbook_result(key, \"shodan\", \"ip-api-address\", \"ISP\"))\n",
    "    summary_iv += \"- Country: **{}**\\n\".format(get_playbook_result(key, virustotal, \"country\", \"text\"))\n",
    "    summary_iv += \"- City: **{}**\\n\".format(get_playbook_result(key, \"shodan\", \"ip-api-address\", \"city\"))\n",
    "    summary_iv += \"- Usage: **{}**\\n\".format(get_playbook_result(key, \"abuseipdb\", \"usageType\", \"usageType\"))\n",
    "    summary_iv += \"- Abuse contact: **{}**\\n\".format(get_playbook_result(key, \"abuse_finder\", \"Attribution\", \"whois-registrant-email\"))\n",
    "    summary_iv += \"\\n\\n\"\n",
    "    summary_iv += \"### Ratings\\n\"\n",
    "    summary_iv += \"- VirusTotal detection rate: **{}**\\n\".format(get_playbook_result(key, virustotal, \"detection-ratio\", \"text\"))\n",
    "    last_analysis_stats = False\n",
    "    for el in playbook_results[key]:\n",
    "        if el[\"source\"] == virustotal and el[\"category\"] == \"last_analysis_stats\" and el[\"type\"] == \"last_analysis_stats\":\n",
    "            last_analysis_stats = el[\"enriched\"]\n",
    "    if last_analysis_stats:\n",
    "        summary_iv += \"     - Harmless: **{}**\\n\".format(last_analysis_stats[\"harmless\"])\n",
    "        summary_iv += \"     - Malicious: **{}**\\n\".format(last_analysis_stats[\"malicious\"])\n",
    "        summary_iv += \"     - Suspicious: **{}**\\n\".format(last_analysis_stats[\"suspicious\"])\n",
    "        summary_iv += \"     - Undetected: **{}**\\n\".format(last_analysis_stats[\"undetected\"])\n",
    "        summary_iv += \"     - Timeout: **{}**\\n\".format(last_analysis_stats[\"timeout\"])\n",
    "    summary_iv += \"- AbuseIPDB confidence score: **{}**\\n\".format(get_playbook_result(key, \"abuseipdb\", \"abuseConfidenceScore\", \"abuseConfidenceScore\"))\n",
    "    greynoise = False\n",
    "    for el in playbook_results[key]:\n",
    "        if el[\"source\"] == \"greynoise\" and el[\"category\"] == \"domain\" and el[\"type\"] == \"reputation\":\n",
    "            greynoise = el[\"enriched\"]\n",
    "    if greynoise:\n",
    "        summary_iv += \"- Greynoise:\\n\"\n",
    "        summary_iv += \"     - Classification: **{}**\\n\".format(greynoise[\"classification\"])\n",
    "        summary_iv += \"     - In noise?: **{}**\\n\".format(greynoise[\"noise\"])\n",
    "        summary_iv += \"     - In RIOT?: **{}**\\n\".format(greynoise[\"riot\"])\n",
    "    summary_iv += \"\\n\\n\"\n",
    "    summary_iv += \"### Associated hostnames\\n\"\n",
    "    summary_iv += \"- Reverse DNS: **{}**\\n\".format(get_playbook_result(key, \"reversedns\", \"Network activity\", \"hostname\"))\n",
    "    summary_iv += \"- Hostnames\\n\"\n",
    "    for el in get_playbook_result(key, virustotal, \"domain\", \"domain\").split(\" \"):\n",
    "        summary_iv += \"     - {}\\n\".format(el)\n",
    "    for el in get_playbook_result(key, \"shodan\", \"Network activity\", \"domain\").split(\" \"):\n",
    "        summary_iv += \"     - {}\\n\".format(el)\n",
    "    summary_iv += \"\\n\\n\"\n",
    "    vulnerability = get_playbook_result(key, \"shodan\", \"External analysis\", \"vulnerability\")\n",
    "    if vulnerability and len(vulnerability) > 1:\n",
    "        summary_iv += \"### Associated vulnerabilities\\n\"\n",
    "        summary_iv += \"- **{}**\\n\".format(vulnerability)\n",
    "        summary_iv += \"\\n\\n\"\n",
    "\n",
    "    ports = get_playbook_result(key, \"shodan\", \"Network activity\", \"port\")\n",
    "    if ports and len(ports) > 1:\n",
    "        summary_iv += \"### Associated open ports\\n\"\n",
    "        for el in ports.split(\" \"):\n",
    "            summary_iv += \"- **{}**\\n\".format(el)\n",
    "        summary_iv += \"\\n\\n\"\n",
    " \n",
    "event_title = \"IP reputation\"\n",
    "print(\"Creating MISP report \\033[92m{}\\033[90m\".format(event_title))\n",
    "chunk_size = 61500\n",
    "for i in range(0, len(summary_iv), chunk_size):\n",
    "    chunk = summary_iv[i:i + chunk_size]\n",
    "    event_report = MISPEventReport()\n",
    "    event_title_edit = event_title\n",
    "    if i > 0:\n",
    "        event_title_edit = \"{} ({} > {})\".format(event_title, i, i + chunk_size)\n",
    "    event_report.name = event_title_edit\n",
    "    event_report.content = chunk\n",
    "    result = misp.add_event_report(misp_event.id, event_report)\n",
    "    if \"EventReport\" in result:\n",
    "        print(\" Report ID: \\033[92m{}\\033[90m\".format(result.get(\"EventReport\", []).get(\"id\", 0)))\n",
    "    else:\n",
    "        print(\"Failed to create report for \\033[91m{}\\033[90m.\".format(event_title))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70dea5ca",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Closure\n",
    "\n",
    "In this **closure** or end step we create a **summary** of the actions that were performed by the playbook. The summary is printed in the playbook and can also be send to a chat channel. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf86d67a-de23-4121-a26b-b8fb1c67c44e",
   "metadata": {},
   "source": [
    "## EN:1 MISP indicators\n",
    "\n",
    "The next section first **queries MISP for the indicators added to the MISP event** that is linked to the execution of this playbook.\n",
    "\n",
    "The indicators are stored in the variable `indicator_table` (table format) and `indicator_raw_list` (in raw format) which is used in a later section to create the playbook summary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "16d50d2c-078c-476f-acfc-bb536934f049",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Got \u001b[92m33\u001b[90m indicator(s) from the event \u001b[92mIP reputation for 185.194.93.128 91.219.237.56\u001b[90m (3125).\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Get all the indicators for our event and store this is in a table. We can also use this for the summary.\n",
    "indicator_search = misp.search(\"attributes\", uuid=misp_event.uuid, to_ids=True, pythonify=True)\n",
    "indicator_raw_list = []\n",
    "indicator_table = PrettyTable()\n",
    "if len(indicator_search) > 0:\n",
    "    indicator_table.field_names = [\"Type\", \"Category\", \"Indicator\", \"Comment\"]\n",
    "    indicator_table.align[\"Type\"] = \"l\"\n",
    "    indicator_table.align[\"Category\"] = \"l\"\n",
    "    indicator_table.align[\"Indicator\"] = \"l\"\n",
    "    indicator_table.align[\"Comment\"] = \"l\"\n",
    "    indicator_table.border = True\n",
    "    for indicator in indicator_search:\n",
    "        if indicator.value not in indicator_raw_list:\n",
    "            indicator_table.add_row([indicator.type, indicator.category, indicator.value, indicator.comment])\n",
    "            indicator_raw_list.append(indicator.value)\n",
    "    print(\"Got \\033[92m{}\\033[90m indicator(s) from the event \\033[92m{}\\033[90m ({}).\\n\\n\".format(len(indicator_raw_list), misp_event.info, misp_event.id))\n",
    "else:\n",
    "    print(\"\\033[93mNo indicators found in the event \\033[92m{}\\033[90m ({})\".format(misp_event.info, misp_event.id))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d26d1c3b-dec4-4191-92fe-4c130e3f9686",
   "metadata": {},
   "source": [
    "### Raw list of MISP indicators\n",
    "\n",
    "The indicators are now stored in `indicator_search` (as Python objects) and `indicator_raw_list` (in raw format, only the indicators). Execute the next cell to display them in a table format. The table is also included in the summary sent to Mattermost and TheHive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "72cd3e2a-afad-4b9c-9f3a-9dbe9e8e135d",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-------------------------+------------------+------------------------------------------------------------------+------------------------------------+\n",
      "| Type                    | Category         | Indicator                                                        | Comment                            |\n",
      "+-------------------------+------------------+------------------------------------------------------------------+------------------------------------+\n",
      "| domain                  | Network activity | 2023.hack.lu                                                     |                                    |\n",
      "| domain                  | Network activity | already5.ddns.net                                                |                                    |\n",
      "| domain                  | Network activity | brightsolarpower.net.http.s11.wbprx.com                          |                                    |\n",
      "| domain                  | Network activity | circl.lu                                                         |                                    |\n",
      "| domain                  | Network activity | clapper.misp-community.org                                       |                                    |\n",
      "| domain                  | Network activity | demo.misp-community.org                                          |                                    |\n",
      "| domain                  | Network activity | doc.cerebrate-project.org                                        |                                    |\n",
      "| domain                  | Network activity | facebook.com.http.s11.wbprx.com                                  |                                    |\n",
      "| domain                  | Network activity | facebook.com.https.s11.wbprx.com                                 |                                    |\n",
      "| domain                  | Network activity | hack.lu                                                          |                                    |\n",
      "| domain                  | Network activity | lolol.hopto.org                                                  |                                    |\n",
      "| domain                  | Network activity | misp-community.org                                               |                                    |\n",
      "| domain                  | Network activity | misp-lea.org                                                     |                                    |\n",
      "| domain                  | Network activity | monitor.circl.lu                                                 |                                    |\n",
      "| domain                  | Network activity | no-hostname.serverastra.com                                      |                                    |\n",
      "| domain                  | Network activity | phishtankapi.circl.lu                                            |                                    |\n",
      "| domain                  | Network activity | podpricelom.com.ua.s11.wbprx.com                                 |                                    |\n",
      "| domain                  | Network activity | secure.gravatar.com.https.s11.wbprx.com                          |                                    |\n",
      "| domain                  | Network activity | serverastra.com                                                  |                                    |\n",
      "| domain                  | Network activity | simaiiii1.ddns.net                                               |                                    |\n",
      "| domain                  | Network activity | typosquatting-finder.circl.lu                                    |                                    |\n",
      "| domain                  | Network activity | www.brightsolarpower.net.http.s11.wbprx.com                      |                                    |\n",
      "| domain                  | Network activity | www.hack.lu                                                      |                                    |\n",
      "| hostname                | Network activity | cppz.circl.lu                                                    | From reversedns for 185.194.93.128 |\n",
      "| hostname                | Network activity | qeohmonlocbe-dedicated.serverastra.com                           | From reversedns for 91.219.237.56  |\n",
      "| ip-dst                  | Network activity | 185.194.93.128                                                   | Added by playbook                  |\n",
      "| ip-dst                  | Network activity | 91.219.237.56                                                    | Added by playbook                  |\n",
      "| ip-src                  | Network activity | 185.194.92.0/22                                                  |                                    |\n",
      "| ip-src                  | Network activity | 91.219.236.0/22                                                  |                                    |\n",
      "| x509-fingerprint-sha1   | Network activity | 93ffc6f7b3435982596f3a23b6b50bcc4da27f75                         |                                    |\n",
      "| x509-fingerprint-sha1   | Network activity | 96b2c7ff82d1eea417dfe67fac3baa9f6e1d0cc1                         |                                    |\n",
      "| x509-fingerprint-sha256 | Network activity | 03b4d92fe372e5b87353bdd822c352330783ec8f2e13c5821261dee669b79739 |                                    |\n",
      "| x509-fingerprint-sha256 | Network activity | 264a8df1cfae07e8995445ea4747fff64b9834842f9683aa59f41cfb2adb6ba0 |                                    |\n",
      "+-------------------------+------------------+------------------------------------------------------------------+------------------------------------+\n",
      "\n",
      "\n",
      "Indicator list in raw format:\n",
      "---------------------------------------------------\n",
      "91.219.237.56\n",
      "185.194.93.128\n",
      "qeohmonlocbe-dedicated.serverastra.com\n",
      "cppz.circl.lu\n",
      "91.219.236.0/22\n",
      "no-hostname.serverastra.com\n",
      "lolol.hopto.org\n",
      "already5.ddns.net\n",
      "simaiiii1.ddns.net\n",
      "brightsolarpower.net.http.s11.wbprx.com\n",
      "facebook.com.http.s11.wbprx.com\n",
      "facebook.com.https.s11.wbprx.com\n",
      "www.brightsolarpower.net.http.s11.wbprx.com\n",
      "podpricelom.com.ua.s11.wbprx.com\n",
      "secure.gravatar.com.https.s11.wbprx.com\n",
      "185.194.92.0/22\n",
      "www.hack.lu\n",
      "clapper.misp-community.org\n",
      "demo.misp-community.org\n",
      "misp-lea.org\n",
      "monitor.circl.lu\n",
      "hack.lu\n",
      "2023.hack.lu\n",
      "doc.cerebrate-project.org\n",
      "phishtankapi.circl.lu\n",
      "typosquatting-finder.circl.lu\n",
      "serverastra.com\n",
      "03b4d92fe372e5b87353bdd822c352330783ec8f2e13c5821261dee669b79739\n",
      "93ffc6f7b3435982596f3a23b6b50bcc4da27f75\n",
      "misp-community.org\n",
      "circl.lu\n",
      "264a8df1cfae07e8995445ea4747fff64b9834842f9683aa59f41cfb2adb6ba0\n",
      "96b2c7ff82d1eea417dfe67fac3baa9f6e1d0cc1\n",
      "---------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "if len(indicator_raw_list) > 0:\n",
    "    print(indicator_table.get_string(sortby=\"Type\"))\n",
    "    print(\"\\n\\nIndicator list in raw format:\")\n",
    "    print(\"---------------------------------------------------\")\n",
    "    for el in indicator_raw_list:\n",
    "        print(\"{}\".format(el))\n",
    "    print(\"---------------------------------------------------\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b82e2327-1e11-4efb-8869-ccbcc896bfcc",
   "metadata": {},
   "source": [
    "## EN:2 Create the summary of the playbook\n",
    "\n",
    "The next section creates a summary and stores the output in the variable `summary` in Markdown format. It also stores an intro text in the variable `intro`. These variables are later used when sending information to Mattermost or TheHive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "ad25bb9f-aa60-4621-b1a4-153444cea7c5",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The \u001b[92msummary\u001b[90m of the playbook is available.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "summary = \"# MISP Playbook summary\\nQuery IP reputation with MISP event: **{}** ({}/events/view/{}). \".format(misp_event.info, misp_url, misp_event.id)\n",
    "\n",
    "summary += \"\\n\"\n",
    "summary += summary_iv\n",
    "summary += \"\\n\"\n",
    "intro = summary\n",
    "\n",
    "summary += \"## Indicators\\n\\n\"\n",
    "summary += \"### Indicators table\\n\\n\"\n",
    "if len(indicator_raw_list) > 0:\n",
    "    indicator_table.set_style(MARKDOWN)\n",
    "    summary += indicator_table.get_string(sortby=\"Type\")\n",
    "    summary += \"\\n\\n\\n\"\n",
    "    summary += \"### Indicators in **raw format**\\n\\n\"\n",
    "    for indicator in indicator_raw_list:\n",
    "        summary += \"{}\\n\\n\".format(indicator)\n",
    "    summary += \"\\n\"\n",
    "else:\n",
    "    summary += \"There are no indicators\"\n",
    "summary += \"\\n\\n\"\n",
    "\n",
    "summary += \"## Correlations\\n\\n\"\n",
    "summary += \"### MISP event matches\\n\\n\"\n",
    "table_mispevents.set_style(MARKDOWN)\n",
    "summary += table_mispevents.get_string()\n",
    "summary += \"\\n\\n\"\n",
    "\n",
    "summary += \"### MISP feed matches\\n\\n\"\n",
    "table_mispfeeds.set_style(MARKDOWN)\n",
    "summary += table_mispfeeds.get_string()\n",
    "summary += \"\\n\\n\"\n",
    "\n",
    "summary += \"\\n\\n\"\n",
    "\n",
    "print(\"The \\033[92msummary\\033[90m of the playbook is available.\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d603257f-0cbe-47c8-876f-9bbe52216c0d",
   "metadata": {},
   "source": [
    "## EN:3 Print the summary\n",
    "\n",
    "Apart from the full summary of the investigation and enrichment, the IP triage summary provides the necessary input for the analyst. This summary was previously also added as a MISP report."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "12691048-4122-4ba3-aab0-203226137e3b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Comment the printing of the summary to avoid cluttering the playbook by accident\n",
    "#display_markdown(summary, raw=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25228e86-b2c1-41cd-8ce2-b9f1993c5d26",
   "metadata": {},
   "source": [
    "## EN:4 Display a map of the IPs\n",
    "\n",
    "Now use folium to render the IPs on a world map. The geographical locations are obtained from the MMDB server."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "b77a31a6-a3b7-4dc0-886b-f6e60a3e4dd6",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"width:100%;\"><div style=\"position:relative;width:100%;height:0;padding-bottom:60%;\"><span style=\"color:#565656\">Make this Notebook Trusted to load map: File -> Trust Notebook</span><iframe srcdoc=\"&lt;!DOCTYPE html&gt;\n",
       "&lt;html&gt;\n",
       "&lt;head&gt;\n",
       "    \n",
       "    &lt;meta http-equiv=&quot;content-type&quot; content=&quot;text/html; charset=UTF-8&quot; /&gt;\n",
       "    \n",
       "        &lt;script&gt;\n",
       "            L_NO_TOUCH = false;\n",
       "            L_DISABLE_3D = false;\n",
       "        &lt;/script&gt;\n",
       "    \n",
       "    &lt;style&gt;html, body {width: 100%;height: 100%;margin: 0;padding: 0;}&lt;/style&gt;\n",
       "    &lt;style&gt;#map {position:absolute;top:0;bottom:0;right:0;left:0;}&lt;/style&gt;\n",
       "    &lt;script src=&quot;https://cdn.jsdelivr.net/npm/leaflet@1.9.3/dist/leaflet.js&quot;&gt;&lt;/script&gt;\n",
       "    &lt;script src=&quot;https://code.jquery.com/jquery-1.12.4.min.js&quot;&gt;&lt;/script&gt;\n",
       "    &lt;script src=&quot;https://cdn.jsdelivr.net/npm/bootstrap@5.2.2/dist/js/bootstrap.bundle.min.js&quot;&gt;&lt;/script&gt;\n",
       "    &lt;script src=&quot;https://cdnjs.cloudflare.com/ajax/libs/Leaflet.awesome-markers/2.0.2/leaflet.awesome-markers.js&quot;&gt;&lt;/script&gt;\n",
       "    &lt;link rel=&quot;stylesheet&quot; href=&quot;https://cdn.jsdelivr.net/npm/leaflet@1.9.3/dist/leaflet.css&quot;/&gt;\n",
       "    &lt;link rel=&quot;stylesheet&quot; href=&quot;https://cdn.jsdelivr.net/npm/bootstrap@5.2.2/dist/css/bootstrap.min.css&quot;/&gt;\n",
       "    &lt;link rel=&quot;stylesheet&quot; href=&quot;https://netdna.bootstrapcdn.com/bootstrap/3.0.0/css/bootstrap.min.css&quot;/&gt;\n",
       "    &lt;link rel=&quot;stylesheet&quot; href=&quot;https://cdn.jsdelivr.net/npm/@fortawesome/fontawesome-free@6.2.0/css/all.min.css&quot;/&gt;\n",
       "    &lt;link rel=&quot;stylesheet&quot; href=&quot;https://cdnjs.cloudflare.com/ajax/libs/Leaflet.awesome-markers/2.0.2/leaflet.awesome-markers.css&quot;/&gt;\n",
       "    &lt;link rel=&quot;stylesheet&quot; href=&quot;https://cdn.jsdelivr.net/gh/python-visualization/folium/folium/templates/leaflet.awesome.rotate.min.css&quot;/&gt;\n",
       "    \n",
       "            &lt;meta name=&quot;viewport&quot; content=&quot;width=device-width,\n",
       "                initial-scale=1.0, maximum-scale=1.0, user-scalable=no&quot; /&gt;\n",
       "            &lt;style&gt;\n",
       "                #map_6e88592fa138393c5c337762e387bc5c {\n",
       "                    position: relative;\n",
       "                    width: 100.0%;\n",
       "                    height: 100.0%;\n",
       "                    left: 0.0%;\n",
       "                    top: 0.0%;\n",
       "                }\n",
       "                .leaflet-container { font-size: 1rem; }\n",
       "            &lt;/style&gt;\n",
       "        \n",
       "&lt;/head&gt;\n",
       "&lt;body&gt;\n",
       "    \n",
       "    \n",
       "            &lt;div class=&quot;folium-map&quot; id=&quot;map_6e88592fa138393c5c337762e387bc5c&quot; &gt;&lt;/div&gt;\n",
       "        \n",
       "&lt;/body&gt;\n",
       "&lt;script&gt;\n",
       "    \n",
       "    \n",
       "            var map_6e88592fa138393c5c337762e387bc5c = L.map(\n",
       "                &quot;map_6e88592fa138393c5c337762e387bc5c&quot;,\n",
       "                {\n",
       "                    center: [0.0, 0.0],\n",
       "                    crs: L.CRS.EPSG3857,\n",
       "                    zoom: 2,\n",
       "                    zoomControl: true,\n",
       "                    preferCanvas: false,\n",
       "                }\n",
       "            );\n",
       "\n",
       "            \n",
       "\n",
       "        \n",
       "    \n",
       "            var tile_layer_3944dc75aa6139c909a7ef4d8e6568d2 = L.tileLayer(\n",
       "                &quot;https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png&quot;,\n",
       "                {&quot;attribution&quot;: &quot;Data by \\u0026copy; \\u003ca target=\\&quot;_blank\\&quot; href=\\&quot;http://openstreetmap.org\\&quot;\\u003eOpenStreetMap\\u003c/a\\u003e, under \\u003ca target=\\&quot;_blank\\&quot; href=\\&quot;http://www.openstreetmap.org/copyright\\&quot;\\u003eODbL\\u003c/a\\u003e.&quot;, &quot;detectRetina&quot;: false, &quot;maxNativeZoom&quot;: 18, &quot;maxZoom&quot;: 18, &quot;minZoom&quot;: 0, &quot;noWrap&quot;: false, &quot;opacity&quot;: 1, &quot;subdomains&quot;: &quot;abc&quot;, &quot;tms&quot;: false}\n",
       "            ).addTo(map_6e88592fa138393c5c337762e387bc5c);\n",
       "        \n",
       "    \n",
       "            var marker_faca5bdbb4cd61afbb90b9a149d46d2e = L.marker(\n",
       "                [47.0, 20.0],\n",
       "                {}\n",
       "            ).addTo(map_6e88592fa138393c5c337762e387bc5c);\n",
       "        \n",
       "    \n",
       "        var popup_71c4cc5826eb57cc250227ad9ccda7de = L.popup({&quot;maxWidth&quot;: &quot;100%&quot;});\n",
       "\n",
       "        \n",
       "            \n",
       "                var html_1c5601caf81ce711bd8a248c5e6907ea = $(`&lt;div id=&quot;html_1c5601caf81ce711bd8a248c5e6907ea&quot; style=&quot;width: 100.0%; height: 100.0%;&quot;&gt;91.219.237.56&lt;/div&gt;`)[0];\n",
       "                popup_71c4cc5826eb57cc250227ad9ccda7de.setContent(html_1c5601caf81ce711bd8a248c5e6907ea);\n",
       "            \n",
       "        \n",
       "\n",
       "        marker_faca5bdbb4cd61afbb90b9a149d46d2e.bindPopup(popup_71c4cc5826eb57cc250227ad9ccda7de)\n",
       "        ;\n",
       "\n",
       "        \n",
       "    \n",
       "    \n",
       "            var marker_d5c1813eb5264e8f87dca3853a367b52 = L.marker(\n",
       "                [49.75, 6.1667],\n",
       "                {}\n",
       "            ).addTo(map_6e88592fa138393c5c337762e387bc5c);\n",
       "        \n",
       "    \n",
       "        var popup_e2bd17b354f830642c574399fde4a07c = L.popup({&quot;maxWidth&quot;: &quot;100%&quot;});\n",
       "\n",
       "        \n",
       "            \n",
       "                var html_1957b8bd0868dee811e0ebc583c89846 = $(`&lt;div id=&quot;html_1957b8bd0868dee811e0ebc583c89846&quot; style=&quot;width: 100.0%; height: 100.0%;&quot;&gt;185.194.93.128&lt;/div&gt;`)[0];\n",
       "                popup_e2bd17b354f830642c574399fde4a07c.setContent(html_1957b8bd0868dee811e0ebc583c89846);\n",
       "            \n",
       "        \n",
       "\n",
       "        marker_d5c1813eb5264e8f87dca3853a367b52.bindPopup(popup_e2bd17b354f830642c574399fde4a07c)\n",
       "        ;\n",
       "\n",
       "        \n",
       "    \n",
       "&lt;/script&gt;\n",
       "&lt;/html&gt;\" style=\"position:absolute;width:100%;height:100%;left:0;top:0;border:none !important;\" allowfullscreen webkitallowfullscreen mozallowfullscreen></iframe></div></div>"
      ],
      "text/plain": [
       "<folium.folium.Map at 0x7fc8e41b6910>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "world_map = folium.Map(location=[0, 0], zoom_start=2)\n",
    "\n",
    "for value in query:\n",
    "    lat = get_playbook_result(value, \"mmdb\", \"geo\", \"lat\")\n",
    "    long = get_playbook_result(value, \"mmdb\", \"geo\", \"long\")\n",
    "    \n",
    "    marker = folium.Marker(location=[lat, long], popup=value)\n",
    "    marker.add_to(world_map)\n",
    "    \n",
    "world_map"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eba96062-deaa-4fb9-8302-7b3ef829201e",
   "metadata": {},
   "source": [
    "## EN:5 Send a summary to Mattermost\n",
    "\n",
    "Now you can send the summary to Mattermost. You can send the summary in two ways by selecting one of the options for the variable `send_to_mattermost_option` in the next cell.\n",
    "\n",
    "- The default option where the entire summary is in the **chat**, or\n",
    "- a short intro and the summary in a **card**\n",
    "\n",
    "For this playbook we rely on a webhook in Mattermost. You can add a webhook by choosing the gear icon in Mattermost, then choose Integrations and then **Incoming Webhooks**. Set a channel for the webhook and lock the webhook to this channel with *\"Lock to this channel\"*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "8c455bc3-35cc-436d-8237-95c81b57208f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "send_to_mattermost_option = \"via a chat message\"\n",
    "#send_to_mattermost_option = \"via a chat message with card\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "ed2f2cdd-4eea-4125-8e52-8addc47bba5b",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Summary is \u001b[92msent to Mattermost.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "message = False\n",
    "if send_to_mattermost_option == \"via a chat message\":\n",
    "    message = {\"username\": mattermost_playbook_user, \"text\": summary}\n",
    "elif send_to_mattermost_option == \"via a chat message with card\":\n",
    "    message = {\"username\": mattermost_playbook_user, \"text\": intro, \"props\": {\"card\": summary}}\n",
    "\n",
    "if message:\n",
    "    r = requests.post(mattermost_hook, data=json.dumps(message))\n",
    "    r.raise_for_status()\n",
    "if message and r.status_code == 200:\n",
    "    print(\"Summary is \\033[92msent to Mattermost.\\n\")\n",
    "else:\n",
    "    print(\"\\033[91mFailed to sent summary\\033[90m to Mattermost.\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9c0f780-d8df-4f9b-9a11-230eb3e4f42d",
   "metadata": {},
   "source": [
    "## EN:6 Send an alert to TheHive\n",
    "\n",
    "Next to informing your colleagues via Mattermost you can also send an **alert** to TheHive. The alert contains the summary, and a list of indicators as 'observables'.\n",
    "\n",
    "You can change the alert title with `thehive_alert_title` and provide a reference type with `thehive_alert_reference`. Note that this reference needs to be **unique** in TheHive. If you want to create multiple alerts for the same MISP event then add some random value at the end."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "87edbdbf-e6b0-47b7-b70a-4ddb4af67e55",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# The title of the TheHive alert\n",
    "thehive_alert_title = \"MISP Playbook Summary - Query IP reputation\"\n",
    "\n",
    "# A unique reference for the TheHive (we include the MISP event UUID)\n",
    "thehive_alert_reference = \"MISP event - {} - {}\".format(misp_event.info, misp_event.uuid)\n",
    "\n",
    "# Alert type in TheHive\n",
    "thehive_alert_type = \"MISP Playbook alert\"\n",
    "\n",
    "# TLP:Amber for TheHive\n",
    "thehive_tlp = 2\n",
    "\n",
    "# PAP:GREEN for TheHive\n",
    "thehive_pap = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "0da33569-cd50-414a-ad0b-70bcdb69174d",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The TheHive \u001b[92malert ~643224 is added\n"
     ]
    }
   ],
   "source": [
    "# Code block to send an alert to TheHive\n",
    "# We use the Python requests library\n",
    "thehive_headers = {'Content-Type': 'application/json', 'Authorization': f'Bearer {thehive_key}'}\n",
    "thehive_url_create = \"{}/api/v1/alert\".format(thehive_url)\n",
    "\n",
    "thehive_observables = []\n",
    "\n",
    "for value in indicator_raw_list:\n",
    "    attribute_type = False\n",
    "    for expr in regular_expressions:\n",
    "        if re.match(r\"{}\".format(regular_expressions[expr]), value):\n",
    "            attribute_type = expr\n",
    "            break\n",
    "    if attribute_type:\n",
    "        if attribute_type == \"ip-src\" or attribute_type == \"ip-dst\":\n",
    "            dataType = \"ip\"\n",
    "        elif attribute_type == \"md5\" or attribute_type == \"sha256\" or attribute_type == \"sha1\":\n",
    "            dataType = \"hash\"\n",
    "        else:\n",
    "            dataType = attribute_type\n",
    "\n",
    "        if dataType:\n",
    "            thehive_observables.append({\"dataType\": dataType, \"data\": value, \"pap\": thehive_pap, \"tlp\": thehive_tlp})\n",
    "\n",
    "thehive_alert = {\"title\": thehive_alert_title,\n",
    "                 \"description\": intro,\n",
    "                 \"summary\": summary[0:1048576],\n",
    "                 \"type\": thehive_alert_type,\n",
    "                 \"source\": \"playbook\",\n",
    "                 \"sourceRef\": thehive_alert_reference,\n",
    "                 \"tlp\": thehive_tlp, \"pap\": thehive_pap,\n",
    "                 \"observables\": thehive_observables}\n",
    "\n",
    "result = requests.post(thehive_url_create, headers=thehive_headers, data=json.dumps(thehive_alert))\n",
    "if result.status_code == 201 and result.json()['status'] == 'New':\n",
    "    thehive_alert_id = result.json()['_id']\n",
    "    print('The TheHive \\033[92malert {} is added'.format(thehive_alert_id))\n",
    "else:\n",
    "    print('\\033[91mFailed\\033[90m to add TheHive alert')\n",
    "    print(result.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "498b8eff-7d1b-47b7-90c4-e13cc738b36c",
   "metadata": {},
   "source": [
    "## EN:7 End of the playbook "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "4c03bb30",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m End of the playbook\n"
     ]
    }
   ],
   "source": [
    "print(\"\\033[92m End of the playbook\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b17af9f2",
   "metadata": {},
   "source": [
    "## External references <a name=\"extreferences\"></a>\n",
    "\n",
    "- [The MISP Project](https://www.misp-project.org/)\n",
    "- [Mattermost](https://mattermost.com/)\n",
    "- [TheHive](https://thehive-project.org/)\n",
    "- [ip.circl.lu](https://ip.circl.lu/geolookup/)\n",
    "- [abuse_finder](https://github.com/certsocietegenerale/abuse_finder)\n",
    "- [AbuseIPDB](https://www.abuseipdb.com/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1c813f6",
   "metadata": {},
   "source": [
    "## Technical details \n",
    "\n",
    "### Documentation\n",
    "\n",
    "This playbook requires these Python **libraries** to exist in the environment where the playbook is executed. You can install them with `pip install <library>`.\n",
    "\n",
    "```\n",
    "pyfaup\n",
    "chardet\n",
    "PrettyTable\n",
    "ipywidgets\n",
    "mattermostdriver\n",
    "folium\n",
    "\n",
    "```\n",
    "\n",
    "### Colour codes\n",
    "\n",
    "The output from Python displays some text in different colours. These are the colour codes\n",
    "\n",
    "```\n",
    "Red = '\\033[91m'\n",
    "Green = '\\033[92m'\n",
    "Blue = '\\033[94m'\n",
    "Cyan = '\\033[96m'\n",
    "White = '\\033[97m'\n",
    "Yellow = '\\033[93m'\n",
    "Magenta = '\\033[95m'\n",
    "Grey = '\\033[90m'\n",
    "Black = '\\033[90m'\n",
    "Default = '\\033[99m'\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14dce812-e432-4c18-8358-f8b66e5b94a5",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Attachments",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "toc-showcode": false,
  "toc-showmarkdowntxt": false
 },
 "nbformat": 4,
 "nbformat_minor": 5
}