ioc-extractor

Pull indicators of compromise (IOCs) out of unstructured text — threat reports, incident notes, blog posts, emails — and export them as JSON, CSV, or a STIX 2.1 bundle.

It runs entirely offline (no network calls), handles defanged indicators like hxxp://evil[.]com, and is conservative about matching so analysts aren't buried in false positives.

Why

Analysts constantly copy indicators out of PDFs and reports by hand. This tool does the tedious part in one pass and hands back clean, deduplicated, normalised output ready to feed into a TIP or a blocklist.

Extracted indicator types

Type	Notes
IPv4/IPv6	Validated with ipaddress, so 999.1.1.1 is rejected
URLs	Refanged and trailing punctuation stripped
Domains	TLD-checked; file names like payload.exe are excluded
Emails	Domains already covered by an email are not double-counted
Hashes	MD5 / SHA-1 / SHA-256
CVEs	Normalised to upper case (CVE-2021-44228)
Files	Common malware/document extensions

Install

git clone https://github.com/Divolika/ioc-extractor.git
cd ioc-extractor
pip install -e .

Pure standard library — no third-party runtime dependencies. pytest is only needed to run the tests.

Usage

# From a file, JSON output
ioc-extractor samples/threat_report.txt

# CSV to a file
ioc-extractor samples/threat_report.txt -f csv -o iocs.csv

# STIX 2.1 bundle
ioc-extractor samples/threat_report.txt -f stix -o iocs.json

# From stdin
cat report.txt | ioc-extractor -f json

Or as a library:

from ioc_extractor import extract_iocs

result = extract_iocs(open("report.txt").read())
print(result.ipv4, result.cves)
print(result.as_dict())

Example

Input (excerpt from samples/threat_report.txt):

The loader beaconed to hxxp://cdn-update[.]example-bad[.]com/gate.php
and resolved secondary C2 at 185.220.101[.]45.
Initial access leveraged CVE-2023-23397.

Output:

{
  "indicators": {
    "ipv4": ["185.220.101.45"],
    "urls": ["http://cdn-update.example-bad.com/gate.php"],
    "cves": ["CVE-2023-23397"]
  }
}

Running the tests

pip install pytest
pytest -q

Design notes

• Refanging reverses common obfuscation ([.], hxxp, [at], …) before matching. Disable with --no-refang.
• Conservative domain matching: a candidate must end in a known TLD and must not already be the host of an extracted URL or email domain.
• STIX output emits indicator objects for network/file IOCs and vulnerability objects for CVEs, which is how most platforms expect them.