DomainsByCerts

Domains by Certificates

Note

This module utilizes multiple external data sources and APIs to query Certificate Transparency (CT) logs. It implements robust fallback mechanisms, HTTP retry logic, and timeouts. These settings (e.g., MaxRetriesCert, RetryBaseDelay, CrtshPGTimeout) are governed by your global configuration. For comprehensive instructions on adjusting these parameters, refer to the Network Configuration page.

The Domains by Certificates module leverages Certificate Transparency (CT) logs to discover subdomains, emails, and target lifespans associated with a given domain. By extracting Subject Alternative Names (SANs) from SSL/TLS certificates, it can uncover hidden or unlinked infrastructure that is not publicly routable or linked from anywhere else.

Capabilities

The module accepts Domains as input and routes requests to a sequential chain of cert fetchers, extracting the following intelligence:

Extracted Information

• Subdomains: Extracts all valid subdomains listed in the certificate's SAN fields. Automatically parses wildcard certificates (e.g., *.example.com becomes example.com with a Wildcard tag). All discovered subdomain nodes are automatically tagged with SAN and CT Log.
• Emails: Extracts email addresses embedded in certificates (often found in older certificates or specific enterprise PKI setups), validating and returning them as new Email nodes (also tagged with SAN and CT Log).
• Certificate Lifespan: Extracts the notAfter timestamp to determine the maximum known validity of the target's certificates, returning this as a date property.

Historical Data & Expiration Logic

Certificate Transparency logs are append-only, meaning they contain both active and expired certificates. This module intelligently classifies historical data based on the certificate's notAfter timestamp:

• Historical Tags: If a certificate associated with a subdomain/email is expired (its notAfter date is in the past relative to the time of execution), the resulting node is automatically tagged as Historical, and an Expired status property is attached.
• Noise Reduction: By default, to prevent cluttering the graph with outdated infrastructure, expired subdomains are aggregated into a single Expired Certificates summary property rather than generating individual nodes. If the DisableCertExpiredSubdomains network option is enabled (true), this aggregation behavior is overridden, and every historical subdomain is explicitly emitted into the system as an individual node.

Fallback Logic & Data Sources

To ensure high availability and bypass potential rate limits or service outages, the module implements a robust sequential fallback chain. If a source fails or returns no data, it gracefully falls back to the next available provider in the chain. All HTTP requests utilize randomized User-Agents and the retry delays outlined in your network settings.

1. Certspotter API (Primary)

Data Source: Certspotter API

Queries the commercial-grade Certspotter endpoint (api.certspotter.com) for highly accurate and fast issuance records. It expands the response to capture all associated DNS names.

• This source can be explicitly bypassed by setting DisableCertspotter=true in your configuration.

2. crt.sh Direct PostgreSQL (Secondary)

Data Source: crt.sh

If the primary fails, the module connects directly to the crt.sh PostgreSQL database (postgres://guest@crt.sh:5432/certwatch). This direct database connection circumvents the often rate-limited or overloaded crt.sh web frontend, executing specialized text-search queries for rapid discovery.

• This source can be explicitly bypassed by setting DisableCrtshPG=true in your configuration.

3. crt.sh HTTP API (Fallback)

Data Source: crt.sh API

If the direct database connection fails (e.g., due to port blocking, connection limits, or database maintenance), the module gracefully falls back to standard HTTP JSON queries against the crt.sh web endpoint (https://crt.sh/?q=%25.domain&output=json).