Skip to content

Repository files navigation

MalwareWorld

MalwareWorld aggregates public threat-intel blacklists into a worldwide static dataset (IPs, domains, IP ranges, E.164 phone numbers, mobile app IDs, file hashes, TLS/code-signing certificate fingerprints, and known exploited CVEs) and publishes it as a website on GitHub Pages.

  • Website: https://malwareworld.com/
  • HackTricks tools: https://tools.hacktricks.wiki/
  • Blacklists used (URLs): https://malwareworld.com/data/blacklists.txt

What this repo contains

  • docs/: the GitHub Pages web UI (no backend).
  • scripts/: a generator that downloads blacklists and produces:
    • text lists to download (suspiciousIPs.txt, suspiciousDomains.txt, suspiciousRanges.txt, suspiciousPhones.txt, suspiciousAppIds.txt, suspiciousFileHashes.txt, suspiciousSigningCertificates.txt, and knownExploitedVulnerabilities.txt)
    • per-category downloads (type_<Category>_<indicator-kind>.txt)
    • sharded JSON for exact lookups from the UI (IP/domain/range/phone/app ID/file hash/certificate fingerprint/CVE)
    • maps + stats per category
    • monthly archive artifacts (optional; see below)

Generation uses SQLite on disk to keep memory usage low.

Supported indicators and categories

The generated dataset supports these first-class searchable indicator kinds:

Indicator Accepted search input Main download
IPv4 address Exact IPv4 address suspiciousIPs.txt
IPv4 CIDR range Exact CIDR suspiciousRanges.txt
Domain Domain or subdomain; URL input is reduced to its hostname suspiciousDomains.txt
International phone number E.164 or a formatted international number; country codes are never guessed suspiciousPhones.txt
Mobile app ID Android package name or iOS bundle ID suspiciousAppIds.txt
Malware file hash SHA-1 or SHA-256 suspiciousFileHashes.txt
TLS or code-signing certificate SHA-1 or SHA-256 fingerprint, with optional colon separators suspiciousSigningCertificates.txt
Known exploited vulnerability CVE identifier knownExploitedVulnerabilities.txt

The web search displays a current, clickable example for every indicator kind plus URL input. All lookups are static and download only the relevant JSON shard. URLs are checked using their host; domain lookup checks the exact hostname and then its parent domains.

Indicators can belong to more than one category. Current categories are:

  • BadReputation, Malware, Ransomware, KnownAttacker, Spammer, Phishing, Adware, DGA, HideSource, CryptoCurrencies, and Whitelist
  • PhoneSpamSmishing for worldwide spam-call, robocall, SMS-spam, and smishing numbers
  • EmailSpamPhishing for domains used in spam and phishing email
  • CommandAndControl for C2 domains and IP addresses
  • ScamFraud for scam and fraud infrastructure
  • DisposableEmail for temporary-email domains
  • BruteForce for brute-force source IP addresses
  • MobileSpyware for malicious Android/iOS infrastructure, app IDs, binaries, and signing certificates
  • MaliciousCertificate for malware C2 TLS certificates and manually vetted malicious code-signing certificates
  • KnownExploitedVulnerability for the compact CISA catalog of vulnerabilities with evidence of exploitation in the wild; entries marked by CISA as used in known ransomware campaigns also receive Ransomware

New sources are selected for clear malicious semantics, redistribution-compatible licensing, continued maintenance, and bounded size. MalwareWorld deliberately avoids giant combined feeds when they would mostly duplicate existing data or make the generated GitHub Pages and Release assets unmanageable.

Per-category text files use type_<Category>_<kind>.txt, where <kind> is domains, ips, ranges, phones, apps, hashes, certificates, or vulnerabilities. Empty combinations are still emitted as valid text files so automated consumers can use a stable filename matrix.

Browser and client lookup shards

manifest.json describes the current shard patterns; clients should prefer it over hard-coding filenames. The default layout is:

Indicator Shard layout
Domains domains_<first-character>.json
IPv4 addresses ips_<first-octet-group>.json
IPv4 ranges ranges_<first-octet-group>.json
Phone numbers phones_<first-two-E.164-digits>.json
Mobile app IDs apps_<first-character>.json
File hashes hashes_<first-hex-character>.json
Certificate fingerprints certificates_<first-hex-character>.json
Known exploited CVEs vulnerabilities_<CVE-year>.json

This keeps exact browser and library lookups bounded to one small shard instead of downloading a complete list. File-hash input is checked against both the file-hash and certificate shards because both use hexadecimal fingerprints.

GitHub Pages setup

  1. Repo → Settings → Pages
  2. Source: GitHub Actions
  3. Run the workflow .github/workflows/publish-release-assets.yml once (or wait for the schedule).

The workflows publish a GitHub Release with the generated artifacts and also deploy the same artifacts under the Pages site at /data/ so the UI can fetch them without CORS issues.

Run locally (web UI + data generation)

  1. Install:

npm ci

  1. Generate the site data:

npm run generate:site

Useful env vars:

MW_LIMIT_BLACKLISTS=10 MW_CONCURRENCY=10 MW_OUTPUT_DIR=release-assets npm run generate:site

The parser does not impose a global per-blacklist match cap. Individual limit values in blacklists_list.js are still honored where a source intentionally selects only recent entries.

To avoid removing a malicious IP or domain after a transient feed omission, generation writes retention-state.sqlite. Seed the next run with it:

MW_RETENTION_SEED_PATH=path/to/retention-state.sqlite npm run generate:site

An item absent from all of its successfully downloaded sources is retained after the first absence and removed after the second consecutive absence. Failed or skipped sources do not advance that counter. A source that fails six consecutive scheduled downloads expires, and data attributable only to that source is removed. Configure this threshold with MW_RETENTION_MAX_SOURCE_FAILURES. The publish workflow downloads and decompresses the state automatically from the latest release. Releases store it losslessly as retention-state.sqlite.zst; it is excluded from the Pages artifact.

The same state database tracks source freshness. For GitHub-hosted lists, generation records the latest commit affecting the feed path. Other feeds use HTTP Last-Modified metadata when available, with content-hash changes as the universal fallback. The website exposes these values in an expandable table sorted by most recent update and flags repeatedly unchanged or failing sources for review.

For flaky sources (common on CI):

MW_CONCURRENCY=12 MW_RETRY_COUNT=3 MW_RETRY_DELAY_MS=90000 npm run generate:site

Monthly archive (union of all non-whitelist items seen during the month):

MW_MONTHLY=1 MW_MONTHLY_DB_PATH=release-assets/monthly-YYYY-MM-archive.sqlite npm run generate:site

Optional seed to merge the current run into an existing monthly archive:

MW_MONTHLY_SEED_PATH=path/to/monthly-YYYY-MM-archive.sqlite

  1. Serve the UI:

python3 -m http.server 4173 --directory docs

Option A (recommended local dev): generate into docs/data/ so the UI auto-detects it:

MW_OUTPUT_DIR=docs/data npm run generate:site

Open http://127.0.0.1:4173/

Option B: serve release-assets/ separately and point the UI to it:

python3 -m http.server 4174 --directory release-assets

Open http://127.0.0.1:4173/?releaseBase=http://127.0.0.1:4174/

  1. Quick end-to-end smoke test (generates a small subset and verifies outputs + HTTP fetches):

npm test

CLI lookup tools (Node + Python)

The same sharded JSON layout used by the UI can be queried from the command line. Scripts live in tools/lookup/.

Local data (uses docs/data/ if present):

node tools/lookup/lookup.js example.com

python3 tools/lookup/lookup.py 1.1.1.1

node tools/lookup/lookup.js '+34 600 000 000'

node tools/lookup/lookup.js com.example.suspicious

node tools/lookup/lookup.js 203.0.113.0/24

node tools/lookup/lookup.js https://suspicious.example/path

python3 tools/lookup/lookup.py <sha256-or-certificate-fingerprint>

node tools/lookup/lookup.js CVE-2021-44228

Remote release assets:

node tools/lookup/lookup.js example.com --base https://github.com/<owner>/<repo>/releases/latest/download/

python3 tools/lookup/lookup.py 1.1.1.1 --base https://malwareworld.com/data/

Note about removals

This repo previously exposed a Node.js “library” API (including external intelligence lookups). That code path is removed: MalwareWorld is now focused on static data generation + GitHub Pages.

Monthly archives (how it works)

MalwareWorld can keep a monthly union of all non-whitelist indicators and their associated URLs, categories, and available IP geolocation. When enabled:

  • scripts/generate-site-data.js attaches/creates a monthly SQLite archive: monthly-YYYY-MM-archive.sqlite.
  • Each run merges the current dataset into the archive (non‑whitelist only).
  • It also generates monthly map and stats assets: monthly-YYYY-MM-map_{Type}.geojson and monthly-YYYY-MM-stats_{Type}.json.
  • The workflow .github/workflows/publish-release-assets.yml publishes those files as a release and publishes monthly-index.json with the month → release URL mapping. Pages exposes that asset as /data/monthly/index.json.

The UI reads /data/monthly/index.json and shows a Month selector in the Maps section. When a month is selected, the UI loads the monthly map/stats files from the release URL. Because GitHub Releases do not provide CORS headers, the UI fetches monthly files through a configurable CORS proxy (default: https://api.allorigins.win/raw?url=). Override via:

?corsProxy=https://your-proxy/?url= or disable with ?corsProxy=none.

Downloadable files (from the web UI)

These files are published under /data/ on GitHub Pages and also as release assets:

Monthly archive artifacts (not in Pages; only in Releases):

Monthly archive workflow notes

  • The Pages site always serves the latest dataset under /data/.
  • Monthly archives and monthly map/stats are kept in Releases (not in Pages) to avoid unbounded Pages growth.
  • The monthly index is a release asset copied to /data/monthly/index.json during deployment, so the UI can discover months without calling the GitHub API or creating an extra Git commit/run.

License

The MalwareWorld code is MIT licensed. The aggregated blacklist data remains subject to each upstream provider's license and usage terms; the source URL and available license metadata are published in blacklists.json. Some feeds restrict commercial use or require attribution and share-alike distribution, so downstream users must review those terms before redistributing the generated datasets.

About

System based on +500 blacklists to detect internet potencialy malicious hosts

Topics

Resources

Stars

157 stars

Watchers

4 watching

Forks

Releases

Packages

Used by

Contributors

Languages