Skip to content

su6osec/graphite

Repository files navigation

graphite

High-performance, cross-platform subdomain discovery & validation CLI

Build Go Report Card License: MIT Go Version Release

  ██████╗ ██████╗  █████╗ ██████╗ ██╗  ██╗██╗████████╗███████╗
 ██╔════╝ ██╔══██╗██╔══██╗██╔══██╗██║  ██║██║╚══██╔══╝██╔════╝
 ██║  ███╗██████╔╝███████║██████╔╝███████║██║   ██║   █████╗
 ██║   ██║██╔══██╗██╔══██║██╔═══╝ ██╔══██║██║   ██║   ██╔══╝
 ╚██████╔╝██║  ██║██║  ██║██║     ██║  ██║██║   ██║   ███████╗
  ╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝  ╚═╝╚═╝   ╚═╝   ╚══════╝

Graphite aggregates 25+ free passive OSINT sources, deduplicates results, and runs fast concurrent DNS / TLS / HTTP validation with a deterministic confidence scoring system — reporting only verified subdomains by default.


Getting Started

1. Install (requires Go 1.21+)

go install github.com/su6osec/graphite/cmd/graphite@latest

2. Run

graphite -d example.com

That's it. One command, one flag.


Quick Install

Method Command
go install go install github.com/su6osec/graphite/cmd/graphite@latest
Pre-built binary Releases page
Build from source git clone … && make build

Optional Python TUI

From PyPI:

pip install graphite-tui

From source:

pip install -r tui/requirements.txt

Single-Command Example

graphite -d hackerone.com
  graphite  domain=hackerone.com  workers=50  min-confidence=60

  [*] Validating [========================================] 143/143

   #   Subdomain                   Score   Confidence   DNS   TLS   HTTP   Status   Sources
  ─── ──────────────────────────── ─────── ──────────── ───── ───── ────── ──────── ──────────────
   1   api.hackerone.com             95     high          ✓     ✓     ✓      200      crtsh,urlscan
   2   docs.hackerone.com            92     high          ✓     ✓     ✓      200      crtsh,wayback
   3   hacktivity.hackerone.com      88     high          ✓     ✓     ✓      200      crtsh,anubis
   4   gslink.hackerone.com          72     medium        ✓     ✓     ✓      301      hackertarget
   ...

  → 89 subdomain(s) discovered

CLI Reference

Flag Short Default Description
--domain -d (required) Target domain
--output -o table Format (table|json|csv|txt) or file path (e.g. subs.txt, ~/recon/zoho.json)
--confidence -c 60 Minimum confidence score (0–100)
--timeout -t 5m Global timeout (e.g. 3m, 10m)
--workers -w 50 Concurrent validation workers
--verbosity 1 Verbosity: 0=quiet, 1=normal, 2=debug
--no-ui false Disable Python TUI auto-launch
--ports false Enable port checks (80/443/8080/8443)
--show-all false Include low-confidence / unverified results
--disable-source Disable a source by name (repeatable)
--version -v Show version + update status (green=latest, red=outdated)
--update -u Update graphite to the latest version

Sub-commands

graphite sources
graphite tui -d example.com
graphite version
graphite update

Usage examples

graphite -d example.com
graphite -d example.com -o json | jq '.[] | select(.score > 80)'
graphite -d example.com -o results.json
graphite -d example.com -o ~/bugbounty/zoho/subs.txt
graphite -d example.com --verbosity 2 -c 40
graphite -d example.com --verbosity 0 -o json -t 2m
graphite -d example.com --show-all -c 0
graphite -d example.com --ports
graphite -d example.com --disable-source dnsdumpster --disable-source sitedossier
graphite tui -d example.com
graphite -v
graphite update

JSON Output Sample

[
  {
    "subdomain": "api.example.com",
    "score": 95,
    "confidence": "high",
    "verified": true,
    "passive_sources": ["crtsh", "hackertarget", "urlscan", "alienvault"],
    "ips": ["93.184.216.34"],
    "tls_valid": true,
    "tls_subject": "api.example.com",
    "http_status": 200,
    "http_title": "API Gateway",
    "http_server": "nginx/1.24.0",
    "open_ports": [80, 443]
  }
]

Architecture

graphite -d example.com
         │
         ▼
┌─────────────────────────────────────────────────────────┐
│  Engine (internal/engine)                               │
│                                                         │
│  ┌─────────────────────────────────────────────────┐   │
│  │  Passive Collection  (concurrent goroutines)    │   │
│  │                                                  │   │
│  │  crtsh  certspotter  hackertarget  alienvault    │   │
│  │  urlscan  threatcrowd  threatminer  rapiddns     │   │
│  │  anubis  robtex  dnsdumpster  wayback            │   │
│  │  commoncrawl  virustotal  github  bufferover     │   │
│  │  bevigil  leakix  omnisint  chaos  recondev      │   │
│  │  netcraft  securitytrails  phonebook  riddler    │   │
│  │  shrewdeye  dnsrepo  fullhunt  c99  …            │   │
│  └──────────────────────┬──────────────────────────┘   │
│                          │ deduplicated candidates       │
│                          ▼                               │
│  ┌────────────────────────────────────────────────┐     │
│  │  Wildcard / Sinkhole Baseline Probe            │     │
│  │  (xn--random-probe.<domain> → detect wildcard) │     │
│  └──────────────────────┬─────────────────────────┘     │
│                          │                               │
│                          ▼                               │
│  ┌────────────────────────────────────────────────┐     │
│  │  Validation Worker Pool  (–w 50 default)        │     │
│  │                                                  │     │
│  │  DNS validator  → A/AAAA/CNAME (retries x2)     │     │
│  │  TLS validator  → handshake + SAN check          │     │
│  │  HTTP validator → GET/HEAD + title + hash        │     │
│  │  Port validator → connect(80/443/8080/8443)      │     │
│  └──────────────────────┬─────────────────────────┘     │
│                          │                               │
│                          ▼                               │
│  ┌────────────────────────────────────────────────┐     │
│  │  Confidence Scoring + FP Suppression           │     │
│  └──────────────────────┬─────────────────────────┘     │
│                          │                               │
│                          ▼                               │
│        Output: table | json | csv                        │
└─────────────────────────────────────────────────────────┘
         │
         ▼  (optional, parallel)
┌─────────────────────────────────────────────────────────┐
│  Python Textual TUI  (tui/)                             │
│  Reads JSON stream from Go core over stdout              │
│  Live table, filter by score, open-in-browser, copy      │
└─────────────────────────────────────────────────────────┘

Passive Sources

Name Description Key Required
crtsh Certificate Transparency via crt.sh No
certspotter CT logs via certspotter.com No
hackertarget HackerTarget free hostsearch No
alienvault AlienVault OTX passive DNS No
urlscan urlscan.io public scan results No
threatcrowd ThreatCrowd domain API No
threatminer ThreatMiner passive DNS No
rapiddns RapidDNS.io HTML scraping No
anubis Anubis-DB (jldc.me) No
robtex Robtex NDJSON API No
dnsdumpster DNSDumpster form scraping No
wayback Wayback Machine CDX API No
commoncrawl Common Crawl latest index No
virustotal VirusTotal public endpoint No
github GitHub public code search No
bufferover dns.bufferover.run dataset No
tlsbufferover tls.bufferover.run CT data No
bevigil BeVigil mobile-app OSINT No
leakix LeakIX host search No
omnisint Project Sonar FDNS data No
chaos ProjectDiscovery Chaos dataset No
recondev Recon.dev free API No
netcraft Netcraft search DNS No
securitytrails SecurityTrails free endpoint No
phonebook Phonebook.cz / IntelX search No
riddler Riddler.io CN/SAN search No
shrewdeye ShrewdEye.app database No
dnsrepo dnsrepo.noc.org passive DNS No
sitedossier SiteDossier.com pages No
fullhunt FullHunt.io attack surface No
subdomaincenter subdomain.center API No
c99 c99.nl subdomainfinder No
synapsint Synapsint OSINT aggregator No
whoisxmlsubs WhoisXML API subdomain lookup No
digitorus Digitorus CT search No
googlect Google CT transparency report No
archivesubs Archive.org unique domains No
passivetotal RiskIQ PassiveTotal community No
shodan Shodan free DNS dataset No

View all at runtime: graphite sources


Validation Pipeline

Candidate subdomain
       │
       ▼
1. DNS A/AAAA/CNAME  ──fail──▶  score += 0 (passive only)
       │ success
       ▼
2. TLS Handshake     ──fail──▶  score += 0 for TLS
       │ success + SAN match
       ▼
3. HTTP GET/HEAD     ──fail──▶  score += 0 for HTTP
       │ success
       ▼
4. Response fingerprint  ──▶  wildcard / sinkhole check
       │
       ▼
5. Port connect (opt.)  ──▶  score += 10 if any open
       │
       ▼
  Confidence score → filter → output

Confidence Scoring Algorithm

score = passive_score + dns_score + tls_score + http_score + port_score

passive_score = min(len(passive_sources) × 5, 30)
dns_score     = 25  if DNS A/AAAA resolved
tls_score     = 20  if TLS handshake succeeded AND cert SAN covers subdomain
http_score    = 15  if HTTP 2xx response
              =  8  if any HTTP response (3xx/4xx/5xx)
port_score    = 10  if at least one port open (requires --ports)

max total     = 30 + 25 + 20 + 15 + 10 = 100
Band Score Meaning
high 80–100 DNS + TLS + HTTP all pass, multiple sources
medium 60–79 DNS resolves + one more signal
low 40–59 DNS only or HTTP only
speculative 0–39 Passive-only, no active validation

Default minimum display threshold: --confidence 60 (medium+).


False-Positive Mitigation

Graphite employs multiple layers of FP suppression:

  1. Wildcard DNS detection — before validating candidates, a random probe (xn--graphite-probe-zz9h42.<domain>) is resolved. If it returns IPs, those IPs are flagged as wildcard IPs and any candidate resolving to them is suppressed.

  2. HTTP wildcard fingerprinting — if wildcard DNS is active, an HTTP probe is sent to the random host. Any candidate whose response body hash matches the wildcard response is suppressed.

  3. Known sinkhole IPs — a built-in list of well-known sinkhole/placeholder IPs (0.0.0.0, 127.0.0.1, etc.) suppresses matching candidates.

  4. CDN/parking page detection — HTTP title strings matching patterns like "domain for sale", "account suspended", "parking" etc. are suppressed.

  5. TLS SAN verification — TLS score is only awarded when the certificate SANs explicitly cover the subdomain (wildcard certs handled correctly).

  6. At least one active check required for verified=true — passive-only results can still appear with --show-all but are never marked verified.


Rate-Limiting & Caching

  • Each source adapter declares a RateLimit() duration; the engine inserts a courtesy sleep between consecutive requests to the same source.
  • Results are cached in ~/.cache/graphite/ as JSON files keyed by sha256(domain|source). Default TTL: 24 hours.
  • Purge stale cache: rm -rf ~/.cache/graphite/

Python TUI

# Launch TUI (auto-started unless --no-ui)
graphite tui -d example.com
┌─ graphite ──────────────────────────────────────────────────────────────────┐
│ Filter:  [______________]                       total=143 verified=89 high=61│
├─────────────────────────────────────────────────────────────────────────────┤
│  #   Subdomain              Score  DNS  TLS  HTTP  Status  Title            │
│ ─── ───────────────────────────── ──── ──── ───── ─────── ──────────────── │
│  1   api.example.com          95    ✓    🔒   ✓     200    API Gateway      │
│  2   docs.example.com         92    ✓    🔒   ✓     200    Documentation    │
│ ▶ 3   mail.example.com         88    ✓    🔒   ✓     301    Redirecting...   │
│  4   staging.example.com      71    ✓         ~     403    Forbidden        │
├─────────────────────────────────────────────────────────────────────────────┤
│  api.example.com                                                             │
│                                                                              │
│  Score:        95 / 100  (high)                                             │
│  Verified:     yes                                                           │
│  TLS:          valid                                                         │
│  HTTP status:  200                                                           │
│  HTTP title:   API Gateway                                                   │
│  Server:       nginx/1.24.0                                                 │
│  IPs:          93.184.216.34                                                 │
│  Sources:      crtsh, hackertarget, urlscan, alienvault                     │
└─────────────────────────────────────────────────────────────────────────────┘
 q Quit  o Open browser  c Copy  / Filter  Esc Clear filter

TUI keyboard shortcuts:

Key Action
q Quit
o Open selected subdomain in browser
c Copy subdomain to clipboard
/ Focus filter input
Esc Clear filter

Communication model:
The TUI launches graphite -d <domain> -o json --no-ui as a subprocess and parses the JSON array output. It can also load a pre-saved JSON file offline:

from graphite_tui.app import GraphiteApp
app = GraphiteApp()
app.load_json_file("results.json")
app.run()

How to Add a Passive Source Adapter

Drop a new file in internal/sources/:

// internal/sources/myapi.go
package sources

import (
    "context"
    "encoding/json"
    "fmt"
    "net/http"
    "time"
)

// Register at package init – no changes needed elsewhere.
func init() { Register(&MyAPI{}) }

type MyAPI struct{}

func (m *MyAPI) Name() string             { return "myapi" }
func (m *MyAPI) Description() string      { return "MyAPI.io free subdomain endpoint" }
func (m *MyAPI) RateLimit() time.Duration { return 1 * time.Second }
func (m *MyAPI) NeedsKey() bool           { return false }

func (m *MyAPI) Enumerate(ctx context.Context, domain string) (<-chan Result, error) {
    ch := make(chan Result, 256)
    go func() {
        defer close(ch)

        url := fmt.Sprintf("https://api.myapi.io/subdomains/%s", domain)
        req, _ := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
        resp, err := defaultClient(20 * time.Second).Do(req)
        if err != nil {
            return
        }
        defer resp.Body.Close()

        var subs []string
        if err := json.NewDecoder(resp.Body).Decode(&subs); err != nil {
            return
        }
        for _, sub := range subs {
            SendResult(ctx, ch, m.Name(), sub, domain)
        }
    }()
    return ch, nil
}

That's all — the engine auto-discovers it via init().
Verify with: graphite sources | grep myapi


Running Tests

Unit tests only (no network required)

make test-short

All tests including network (DNS/HTTP)

make test

With race detector + coverage

make test-cover

Test locations:

  • tests/sources_test.go — source registry, normalisation
  • tests/scoring_test.go — confidence scoring formula
  • tests/validators_test.go — DNS/HTTP/TLS/port + cache round-trip
  • tests/filter_test.go — wildcard/sinkhole/CDN suppression
  • tests/output_test.go — JSON/CSV/table formatter correctness

Build & Release

Build for current platform

make build
./graphite -d example.com

Cross-compile (all platforms)

make build-all

Release (requires goreleaser + git tag)

git tag v1.0.0
make release

CI/CD runs automatically on every push and pull request via .github/workflows/ci.yml (lint → test matrix → cross-platform build → goreleaser on tag).


Responsible Use

Important: Graphite is designed for authorised security assessments, bug-bounty hunting, and defensive security research only.

  • Always obtain written permission before scanning domains you do not own.
  • Check and comply with your organisation's security policies before running active probes.
  • Graphite respects rate limits; do not modify them to hammer third-party APIs.
  • Do not use Graphite to enumerate targets for unauthorised access, DoS, or any illegal purpose.
  • Discovered subdomains may expose sensitive infrastructure — handle results responsibly and do not share without authorisation.

Graphite never exfiltrates or uploads discovered data to any third-party service. All results remain local.


Roadmap

  • 39 passive OSINT sources (zero API keys)
  • Active DNS / TLS / HTTP / Port validation
  • Confidence scoring + false-positive suppression
  • Output to file: JSON, CSV, plain TXT (one subdomain per line)
  • Self-update (graphite update)
  • Version check with colored outdated/latest badge
  • Subdomain takeover detection
  • ASN-based subdomain correlation
  • MX / NS / TXT record collection
  • DNSSEC validation flag
  • Streaming NDJSON output mode
  • Docker image
  • Plugin system for custom validators

Contributing

See CONTRIBUTING.md for guidelines, adapter skeleton, and the pull request checklist.

Security

To report a vulnerability, do not open a public GitHub issue. Please open a GitHub Security Advisory or email the maintainer directly.

License

MIT © 2024 su6osec