emailvalidator

A production-oriented Go email validation library, CLI, HTTP service, and authenticated ESP webhook receiver.

The module combines conservative address parsing, DNS mail-routing checks, active SMTP recipient probing, catch-all detection, disposable-domain detection, role-account flags, and privacy-preserving reputation signals.

What is included

• Syntax and normalization
  • 254-byte address and 64-byte local-part limits
  • dot-atom validation and optional quoted local parts
  • optional SMTPUTF8 local parts
  • Unicode-domain conversion to Punycode
  • optional pluggable full IDNA/UTS #46 mapper
  • domain-literal support only when explicitly enabled
  • control-character and SMTP command-injection rejection
• DNS verification
  • MX lookup and preference ordering
  • Null MX detection
  • RFC 5321 implicit MX fallback to A/AAAA
  • bounded positive/negative caches and duplicate-request coalescing
• SMTP verification
  • direct SMTP protocol client with EHLO/HELO fallback
  • RCPT TO mailbox classification
  • enhanced status-code parsing
  • MX failover, deadlines, bounded responses, concurrency limits, and per-domain pacing
  • STARTTLS with certificate validation and TLS 1.2 minimum
  • SMTPUTF8 capability enforcement
  • catch-all sampling using cryptographically random recipients
  • positive, negative, unknown, and catch-all caches
  • DNS-rebinding/SSRF defense: MX hosts are resolved once, public IPs are pinned, and non-public targets are blocked by default
• Disposable detection
  • 22,000+ embedded domains from an MIT-licensed community dataset
  • parent-domain matching
  • lock-free reads, atomic blocklist replacement, and application allowlist override
• Role and provider flags
  • generic role addresses such as info@, support@, sales@, admin@, and noreply@
  • common free-mail provider flag
• Reputation tracking
  • delivered, hard/soft bounce, complaint, unsubscribe, and deferral signals
  • clear-text addresses are accepted only transiently
  • persisted keys are HMAC-SHA-256 hashes using an operator-owned 32+ byte pepper
  • address and domain aggregates
  • provider event-ID idempotency, including across restarts
  • CRC32C append log, torn-tail recovery, exclusive lock, optional fsync, atomic compaction, and retention of hashed idempotency markers
• Webhook adapters
  • generic HMAC endpoint
  • Twilio SendGrid ECDSA signature verification
  • Mailgun timestamp/token HMAC verification
  • AWS SES/SNS certificate and message-signature verification with strict certificate-URL validation
  • Postmark operator-configured secret header or HTTP Basic authentication
  • body limits, timestamp windows, replay rejection, provider event normalization, and no recipient echo in responses
• Operational surfaces
  • library API
  • bulk validation preserving input order
  • CLI
  • HTTP daemon with API-key authentication, rate limiting, strict JSON, request/body limits, security headers, health check, TLS option, and graceful shutdown
  • Dockerfile, Compose example, Makefile, tests, race tests, and dataset update script

Important SMTP limitation

An SMTP 250 response means the remote server accepted the recipient command at that moment. It is not absolute proof that a human-controlled mailbox exists: providers may accept then bounce, tarpitting and greylisting are common, and some providers intentionally obscure mailbox state. Treat exists as strong evidence, does_not_exist as strong negative evidence, and temporary/blocked/unknown results as inconclusive. Never send message content during verification.

Active SMTP probing is disabled by default. Enable it only from infrastructure permitted to make outbound TCP/25 connections, use a valid HELO identity and envelope sender under your control, apply strict rate limits, and review provider policies and applicable law.

Requirements

• Go 1.23 or newer for source compatibility; use a currently security-supported Go toolchain for production builds (the Dockerfile pins Go 1.26.5)
• Network DNS access for MX checks
• Outbound TCP/25 for active SMTP checks
• A stable, secret 32-byte-or-longer reputation pepper for durable reputation storage

The core module has no third-party runtime dependencies.

Quick start: library

package main

import (
    "context"
    "encoding/json"
    "log"
    "os"
    "time"

    ev "github.com/oarkflow/ev"
)

func main() {
    cfg := ev.DefaultConfig() // DNS on, SMTP off

    smtp := cfg.SMTP.Options
    smtp.Enabled = true
    smtp.HELODomain = "validator.example.com"
    smtp.MailFrom = "email-validation@example.com"
    smtp.RequireTLS = false // many MX servers still do not require STARTTLS
    smtp.MaxConcurrent = 16
    smtp.MinDomainInterval = 500 * time.Millisecond
    cfg.SMTP = ev.NewSMTPChecker(smtp)

    validator := ev.New(cfg)
    defer validator.Close()

    ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
    defer cancel()

    result, err := validator.Validate(ctx, "person@example.com")
    if err != nil {
        log.Fatal(err)
    }
    json.NewEncoder(os.Stdout).Encode(result)
}

Full IDNA mapping without a core dependency

The default normalizer is conservative and performs direct Unicode-to-Punycode conversion. Applications needing complete UTS #46 mapping can supply any object with ToASCII(string) (string, error), for example a profile from golang.org/x/net/idna:

cfg := emailvalidator.DefaultConfig()
cfg.Address.DomainMapper = idna.Lookup

Syntax-only validation

result := emailvalidator.ValidateSyntax("person@example.com", emailvalidator.AddressOptions{})
if result.Status != emailvalidator.StatusPass {
    // result.Error contains the reason
}

Durable privacy-preserving reputation

Generate the pepper once and store it in a secret manager. Never rotate or lose it without a migration plan; a different pepper produces different keys and makes existing aggregates unreachable.

openssl rand -base64 32

pepper := []byte("at-least-32-random-secret-bytes...")
store, err := emailvalidator.OpenFileReputationStore(emailvalidator.FileReputationOptions{
    Path:                 "/var/lib/emailvalidator/reputation.db",
    Pepper:               pepper,
    SyncWrites:           true,
    CompactAfterRecords:  100_000,
    MaxLogBytes:          64 << 20,
    IdempotencyRetention: 30 * 24 * time.Hour,
})
if err != nil {
    log.Fatal(err)
}

cfg := emailvalidator.DefaultConfig()
cfg.Reputation = store
validator := emailvalidator.New(cfg) // Close closes the store

Record signals directly:

err := validator.RecordReputationEvent(ctx, emailvalidator.ReputationEvent{
    ID:    "my-esp:event-unique-id",
    Email: "person@example.com",
    Type:  emailvalidator.EventHardBounce,
    At:    time.Now().UTC(),
})

The file contains HMAC keys and aggregate counters only. It does not persist email addresses, domains, webhook bodies, SMTP responses, or bounce reasons. Hashing is pseudonymization, not anonymization: protect the pepper, restrict file access, define retention, and apply your privacy obligations.

The bundled file store is intentionally single-process and uses an exclusive lock. For a replicated service, implement the small ReputationStore interface against a transactional shared database or route all reputation writes to one owner.

CLI

Build and validate arguments:

go build -o bin/emailvalidator ./cmd/emailvalidator
./bin/emailvalidator person@example.com support@example.org

Read newline-delimited addresses and emit JSON Lines:

cat emails.txt | ./bin/emailvalidator -json -workers 16

Enable SMTP explicitly:

./bin/emailvalidator \
  -smtp \
  -helo validator.example.com \
  -mail-from email-validation@example.com \
  person@example.com

HTTP service

cp .env.example .env
# Edit secrets and identities, then:
docker compose up --build

Or run directly:

export EV_LISTEN=:8080
export EV_API_KEY='replace-with-a-long-random-key'
go run ./cmd/emailvalidatord

Validate one address:

curl -sS http://127.0.0.1:8080/v1/validate \
  -H 'Content-Type: application/json' \
  -H 'X-API-Key: replace-with-a-long-random-key' \
  -d '{"email":"person@example.com"}'

Bulk validation:

curl -sS http://127.0.0.1:8080/v1/validate/bulk \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer replace-with-a-long-random-key' \
  -d '{"emails":["a@example.com","support@example.org"],"workers":8}'

Endpoints:

Method	Path	Authentication	Purpose
GET	/healthz	none	liveness
POST	/v1/validate	API key when configured	one address
POST	/v1/validate/bulk	API key when configured	bounded batch
POST	/webhooks/generic	HMAC signature	generic reputation events
POST	/webhooks/sendgrid	SendGrid ECDSA signature	SendGrid events
POST	/webhooks/mailgun	Mailgun HMAC signature	Mailgun events
POST	/webhooks/postmark	secret header or Basic Auth	Postmark events
POST	/webhooks/ses	SNS signature	SES events

See docs/CONFIGURATION.md and docs/WEBHOOKS.md.

Verdict model

The API deliberately avoids a misleading boolean result:

• deliverable: mail routing passed and SMTP explicitly accepted the mailbox without catch-all behavior.
• risky: disposable, role account, catch-all, or meaningful negative reputation signals.
• undeliverable: invalid syntax, non-mail-routable domain, Null MX, or a strong permanent SMTP rejection.
• unknown: checks were disabled or remote systems returned inconclusive/transient/policy-blocked outcomes.

Use risk_score, stable reasons[].code, and individual stage results to make business-specific decisions. Do not reject users solely because an address is a role account or because an SMTP probe was blocked.

Security defaults

• SMTP disabled by default.
• Private, loopback, link-local, multicast, and unspecified SMTP targets blocked by default.
• MX names are resolved once and dials are pinned to the reviewed IP.
• STARTTLS validates the MX hostname and enforces TLS 1.2 or newer.
• SMTP commands reject CR/LF injection.
• SMTP response lines and total responses are bounded.
• HTTP JSON rejects unknown fields, trailing values, and oversized bodies.
• API secrets are compared in constant time.
• SNS certificate downloads accept only strict AWS SNS HTTPS hosts and paths, reject redirects, validate certificate chains, and cache briefly.
• Durable reputation uses HMAC-SHA-256 rather than unsalted hashes.
• On-disk files and lock files are created with owner-only permissions.

For deployment guidance, see SECURITY.md.

Disposable-domain data

The embedded compressed list currently contains 22,102 domains, substantially more than the requested 3,000. It is derived from the MIT-licensed disposable/disposable-email-domains project. Lists can be stale or contain false positives, so use SetAllowlist for domains you trust and update the snapshot regularly:

make update-disposable

The updater validates, sorts, deduplicates, requires at least 3,000 entries, and creates deterministic gzip output.

Customization

The main extension points are:

• DNSResolver for a private resolver, DNS-over-HTTPS gateway, or deterministic test resolver.
• SMTPDialer and SMTPIPResolver for controlled networking.
• DomainMapper for full IDNA policy.
• ReputationStore for PostgreSQL, MySQL, Redis, or another transactional backend.
• RoleAccountDetector and FreeProviderDetector for organization-specific classification.
• Scorer / ScorerFunc for business-specific weights and verdict policy.
• Observer for metrics and traces. Observer callbacks are synchronous and must return quickly.
• DisposableDetector.ReplaceBlocklist and SetAllowlist for local policy.

Build and verification

make check

This runs formatting verification, go vet, unit/integration tests, race tests, and builds both executables.

Individual commands:

go test ./...
go test -race ./...
go vet ./...
go build ./cmd/emailvalidator ./cmd/emailvalidatord

The SMTP tests use a local fake server and never contact third-party mail systems.

Package layout

.
├── address.go                 syntax, SMTPUTF8, domain normalization
├── dns.go                     MX/Null-MX/implicit-MX checks
├── smtp.go                    SMTP protocol and catch-all verification
├── disposable.go              embedded and replaceable domain detection
├── reputation*.go             HMAC aggregate stores and durable log
├── webhook*.go                authenticated ESP adapters
├── validator.go / score.go    orchestration and verdict model
├── httpapi/                   hardened HTTP handler
├── cmd/emailvalidator/        CLI
├── cmd/emailvalidatord/       HTTP daemon
├── examples/library/          embedding example
└── data/                      embedded dataset and license

Non-goals

• Sending verification emails or message bodies
• Bypassing providers that intentionally block mailbox enumeration
• Claiming a mailbox belongs to a person
• Guaranteeing future delivery
• Replacing consent, double opt-in, suppression lists, or real delivery telemetry

License

The Go code is MIT licensed. The disposable-domain snapshot has its own included MIT notice in data/LICENSE-disposable-email-domains.

High-throughput bulk validation

ValidateBulk is the recommended in-memory batch API. It uses a bounded worker pool, collapses duplicate normalized addresses, distributes domains across workers, preserves input order, supports per-item deadlines, continues past individual failures when configured, and returns throughput/verdict statistics.

bulk, err := validator.ValidateBulk(ctx, emails, emailvalidator.BulkOptions{
    Workers:         64,
    QueueSize:       256,
    Deduplicate:     true,
    ContinueOnError: true,
    PerItemTimeout:  15 * time.Second,
    ProgressEvery:   1000,
    OnProgress: func(p emailvalidator.BulkProgress) {
        log.Printf("completed=%d/%d rate=%.0f/s", p.Completed, p.Total, p.RatePerSecond)
    },
})

For lists too large to retain in memory, use ValidateStream. It consumes a channel and emits completion-order results using bounded memory and backpressure.

input := make(chan string, 256)
go readEmails(input)
for item := range validator.ValidateStream(ctx, input, emailvalidator.BulkOptions{
    Workers: 64, QueueSize: 256, PerItemTimeout: 15 * time.Second,
}) {
    // write item immediately to NDJSON, CSV, a queue, or a database
}

The daemon exposes:

• POST /v1/validate/bulk — JSON request, ordered JSON response with summary.
• POST /v1/validate/bulk/stream — newline-delimited email input and NDJSON output as results complete.

Example streaming request:

printf 'a@example.com\nb@example.com\n' | curl -N \
  -H 'Content-Type: text/plain' \
  -H "X-API-Key: $EV_API_KEY" \
  --data-binary @- http://localhost:8080/v1/validate/bulk/stream

Performance guidance

• Keep SMTP concurrency below provider and network limits; EV_SMTP_MAX_CONCURRENT=32 is intentionally conservative.
• For syntax, disposable, role, reputation, and cached DNS validation, start with 4–8 workers per CPU.
• Enable de-duplication for imported lists; duplicate results reuse the original validation without repeated DNS/SMTP work.
• Streaming is preferable above tens of thousands of addresses because memory remains bounded.
• Domain round-robin scheduling avoids letting one heavily represented provider monopolize workers while SMTP pacing is active.
• DNS, SMTP mailbox, and catch-all caches are shared across all bulk workers.