MerchantGuard is a modular Python-based merchant risk assessment engine that automatically analyzes merchant websites, extracts publicly available business and technical signals, correlates them with external infrastructure intelligence, and generates an explainable merchant risk assessment using a configurable rule-based evaluation engine.
Unlike a traditional web scraper that only collects website data, MerchantGuard transforms extracted information into structured evidence, evaluates it through transparent business rules, and produces a comprehensive merchant risk report that can support merchant verification workflows.
This project is an educational implementation inspired by real-world merchant onboarding and risk assessment processes used across the payments industry. It demonstrates how publicly available website intelligence, infrastructure analysis, and explainable business rules can be combined to automate a portion of merchant verification.
Disclaimer
MerchantGuard is an educational project created for learning and demonstration purposes. It is not affiliated with or endorsed by Stripe, Visa, Mastercard, Razorpay, PayPal, Adyen, or any other payment provider. Commercial merchant underwriting systems incorporate proprietary datasets, transaction history, fraud intelligence, behavioural analytics, and machine learning models that are outside the scope of this project.
- Overview
- The Problem
- Why MerchantGuard?
- Key Features
- Technology Stack
- High-Level Architecture
- Risk Evaluation Pipeline
MerchantGuard combines multiple software engineering disciplines into a single workflow:
- Backend Development
- Web Crawling & HTML Parsing
- Browser Automation
- Security Automation
- Risk Analytics
- Rule-Based Decision Engine
- Open Source Intelligence (OSINT)
- Python Automation
Instead of treating a merchant website as a collection of HTML pages, MerchantGuard treats it as a source of evidence.
The engine extracts merchant information, validates technical infrastructure, correlates external intelligence, applies configurable business rules, and produces an explainable merchant risk assessment.
The result is a transparent evaluation process where every finding is supported by evidence rather than a black-box score.
Before a merchant is allowed to process online payments, financial institutions and payment service providers typically perform a series of verification checks to determine whether the business appears legitimate and whether additional review is required.
Many of these checks involve manually collecting information such as:
- Business identity
- Contact information
- Legal policy pages
- Domain registration details
- SSL/TLS configuration
- DNS records
- Website security indicators
- Public business presence
- Marketing claims
- Technical website characteristics
The information is often distributed or scattered across multiple sources and requires manual investigation.
MerchantGuard automates this process by collecting publicly available information, correlating it with external intelligence, and evaluating it using transparent rule-based logic.
MerchantGuard is not simply a web scraper.
The crawler is only the first stage of a larger evidence-based risk evaluation pipeline.
The project was designed around the idea that publicly observable signals can be transformed into structured evidence and evaluated through explainable business rules.
Instead of producing raw scraped data, MerchantGuard answers questions such as:
- Does the website provide sufficient business identity information?
- Is the merchant using secure infrastructure?
- Are legal policy pages available?
- Does the domain registration raise additional verification requirements?
- Are there inconsistencies between the website and publicly available information?
- Which observations should increase the merchant's overall risk?
Every triggered finding includes:
- Detected evidence
- Rule triggered
- Risk category
- Severity
- Rationale
- Overall contribution to the final assessment
This makes the generated report transparent and easy to understand.
- Automatic website crawling
- Multi-page discovery (About, Contact, Privacy, Terms, Refund, Shipping)
- JavaScript rendering fallback using Playwright
- HTML parsing and structured content extraction
- Business identity
- Contact information
- Email addresses
- Phone numbers
- Business addresses
- Legal pages
- Marketing claims
- Payment indicators
- Social media links
- Downloads
- External links
- Embedded scripts
- Mixed content detection
- WHOIS analysis
- DNS record validation
- SSL/TLS certificate inspection
- Domain registration analysis
- Public business search correlation
- Configurable rule catalog
- Explainable findings
- Severity classification
- Risk categorisation
- Evidence-backed observations
- Structured JSON output
- Human-readable console report
- Explainable merchant findings
- Overall merchant risk score
- Risk rating
- Recommendation summary
| Category | Technologies |
|---|---|
| Language | Python 3 |
| Web Crawling | Requests |
| HTML Parsing | BeautifulSoup4 |
| Browser Automation | Playwright |
| Domain Intelligence | python-whois, dnspython, tldextract |
| Data Models | Python Dataclasses |
| Reporting | JSON, Rich |
| Core Utilities | pathlib, logging, regex, typing, ssl, socket |
Merchant Website
│
▼
Website Extraction Layer
│
▼
Structured Website Evidence
│
▼
External Intelligence Correlation
(WHOIS • DNS • SSL • Public Search)
│
▼
Rule Evaluation Engine
│
▼
Risk Scoring & Categorisation
│
▼
Structured Merchant Risk Report
MerchantGuard follows a modular pipeline architecture where each component performs a single responsibility. Evidence flows in one direction through the system, making the evaluation process easier to understand, maintain, and extend.
Input URL
│
▼
Website Crawling
│
▼
Evidence Extraction
│
▼
External Correlation
│
▼
Rule Evaluation
│
▼
Risk Scoring
│
▼
JSON + Console Report
Each stage enriches the previous one until a final merchant risk assessment is produced.
Before running MerchantGuard, ensure the following software is installed on your system.
Python 3.11+ Playwright (JavaScript website rendering)
Git (optional) to Clone the repository
git clone https://github.com/sarbeshmallick/merchantguard.git
cd merchantguard-risk-engineCreating a virtual environment isolates project dependencies from other Python projects installed on your machine.
python -m venv .venvActivate the environment.
.\.venv\Scripts\Activate.venv\Scripts\activate.batsource .venv/bin/activatepython -m pip install -r requirements.txtMerchantGuard uses Playwright as a fallback for websites that require JavaScript rendering.
Install the supported browsers using:
playwright installThis step downloads Chromium, Firefox, and WebKit engines used during browser automation.
Run the evaluator by providing a website URL.
python risk_evaluator.py https://example.comExample
python risk_evaluator.py https://manifestwaresoftware.com/web/
python risk_evaluator.py https://zylotechindia.online/MerchantGuard will automatically
- Crawl the website
- Discover important pages
- Extract merchant information
- Perform infrastructure correlation
- Execute the rule engine
- Calculate a merchant risk score
- Produce a structured report
MerchantGuard/
│
├── risk_evaluator.py # Application entry point
│
├── extractor.py # Website crawling & evidence extraction
│
├── correlator.py # WHOIS, DNS, SSL & external intelligence
│
├── rules.py # Rule catalogue & evaluation engine
│
├── scorer.py # Risk score calculation
│
├── report.py # Console & JSON reporting
│
├── models.py # Shared data models
│
├── requirements.txt
│
├── README.md
│
└── outputs/
├── manifest.json
├── zylotech.json
├── amazon.json
└── realdsa.json
The project follows a layered architecture where each module has a single responsibility.
The evaluation process follows a fixed pipeline.
Website URL
│
▼
Website Extraction
│
▼
Evidence Collection
│
▼
External Correlation
│
▼
Rule Engine
│
▼
Risk Scoring
│
▼
Merchant Risk Report
MerchantGuard begins by crawling the supplied website and identifying pages that commonly contain merchant information.
Examples include:
- Home
- About
- Contact
- Privacy Policy
- Terms & Conditions
- Refund Policy
- Shipping Policy
The extracted information is converted into structured evidence.
The extracted website evidence is enriched using publicly available infrastructure information.
Examples include:
- WHOIS
- DNS Records
- SSL/TLS Certificates
- Domain Registration
- Public Business Presence
The evidence is evaluated against a configurable catalogue of business rules.
Each rule determines whether additional verification or increased risk should be assigned.
Every finding contains:
- Detected evidence
- Rule triggered
- Risk category
- Severity
- Explanation
- Supporting rationale
The evaluation process is deterministic and fully explainable.
Triggered findings contribute towards an overall merchant risk score.
The scoring engine:
- Aggregates rule results
- Applies severity weighting
- Groups findings by category
- Produces an overall merchant risk rating
MerchantGuard produces both:
- Human-readable console output
- Structured JSON output
The JSON report is designed to be machine-readable and can be integrated into downstream workflows.
Each finding generated by MerchantGuard includes:
| Field | Description |
|---|---|
| Detected Element | Website or infrastructure observation |
| Rule Triggered | Rule responsible for the finding |
| Risk Category | Category of observation |
| Severity | Low / Medium / High |
| Rationale | Why the rule triggered |
| Supporting Evidence | Website or external correlation evidence |
Finally, the engine produces:
- Overall Risk Score
- Overall Risk Rating
- Summary Recommendation
rather than a collection of isolated observations.
Below is an example of MerchantGuard generating a structured merchant risk assessment from a live website.
┌────────────────────────────────────────────────────────────┐
│ │
│ 
│
│ │
└────────────────────────────────────────────────────────────┘
Every execution produces a structured JSON report. To store each results in their respective files just type --output example.com json
Example
example.com json
The reports contain:
- Extracted website evidence
- External intelligence
- Triggered findings
- Risk categories
- Severity
- Supporting rationale
- Overall merchant assessment
The output is intentionally structured so that it can be consumed by both humans and downstream applications.
MerchantGuard currently evaluates multiple categories of merchant risk.
Examples include:
- Merchant Identity
- Contact Information
- Website Content
- Legal & Compliance
- Domain Intelligence
- Infrastructure Security
- SSL/TLS Configuration
- DNS Configuration
- Public Presence
- Website Behaviour
- External Correlation
The rule engine has been designed to be easily extensible, allowing additional business rules to be added without changing the overall system architecture.
Merchant verification is an important component of modern payment ecosystems.
Before a business is allowed to process online payments, payment service providers (PSPs), acquiring banks, marketplaces, and financial institutions typically perform various verification checks to understand whether additional due diligence is required.
Some of these checks include verifying:
- Business identity
- Website legitimacy
- Contact information
- Legal compliance
- Domain ownership
- Infrastructure security
- Public business presence
- Website consistency
- Risk indicators
Many of these observations are publicly available but are distributed across multiple sources.
MerchantGuard demonstrates how these publicly observable signals can be collected, correlated, and evaluated using a transparent rule-based approach.
Although simplified for educational purposes, the overall workflow reflects the architecture commonly found in modern merchant verification systems.
MerchantGuard sits at the intersection of several software engineering domains.
| Domain | Role within MerchantGuard |
|---|---|
| Backend Development | Overall application architecture |
| Web Crawling | Website discovery and retrieval |
| HTML Parsing | Structured information extraction |
| Browser Automation | JavaScript website rendering |
| Security Automation | SSL, DNS and infrastructure inspection |
| Risk Analytics | Merchant evaluation |
| Rule Engine | Business decision logic |
| OSINT (Open Source Intelligence) | External public information correlation |
Rather than focusing on a single discipline, MerchantGuard combines several independent components into a single explainable evaluation pipeline.
A typical scraper usually follows this workflow:
Website
│
▼
Collect HTML
│
▼
Save Data
MerchantGuard extends that process considerably.
Website
│
▼
Website Crawling
│
▼
Evidence Extraction
│
▼
Infrastructure Correlation
│
▼
Rule Evaluation
│
▼
Risk Scoring
│
▼
Merchant Risk Assessment
The website crawler is only the first stage of a larger evidence-driven evaluation pipeline.
The primary objective of MerchantGuard is not data collection, but risk assessment.
One of the primary goals of MerchantGuard is transparency.
Instead of producing a single numerical score without explanation, every observation is linked to:
- Supporting evidence
- Triggered rule
- Risk category
- Severity
- Technical rationale
This makes the generated assessment significantly easier to understand, review, and extend.
Several excellent open-source platforms solve problems adjacent to merchant risk assessment.
| Project | Primary Focus |
|---|---|
| urlscan.io | Website analysis, screenshots, infrastructure intelligence |
| SpiderFoot | Automated OSINT collection |
| OpenCTI | Cyber Threat Intelligence platform |
| Wappalyzer | Website technology fingerprinting |
| BuiltWith | Website technology detection |
| OWASP ZAP | Web application security testing |
| Nuclei | Template-based vulnerability scanning |
These projects are widely used within cybersecurity, threat intelligence, and infrastructure analysis.
MerchantGuard differs in its objective.
Rather than focusing on vulnerability discovery, technology detection, or threat intelligence, MerchantGuard focuses specifically on merchant website evaluation through explainable business rules.
Merchant underwriting systems are typically proprietary.
Payment providers treat their internal risk models as valuable intellectual property because those systems directly influence:
- Fraud prevention
- Merchant onboarding
- Compliance
- Financial exposure
- Operational risk
Publishing the complete rule logic would make it significantly easier for fraudulent actors to understand how automated evaluations are performed and potentially adapt their behaviour to bypass them.
For this reason, publicly available merchant underwriting engines are uncommon compared with vulnerability scanners, OSINT platforms, or technology fingerprinting tools.
MerchantGuard is therefore intended as an educational implementation inspired by publicly observable aspects of merchant verification rather than a recreation of proprietary commercial systems.
Many payment companies perform merchant verification during onboarding.
Examples include:
- Stripe
- Visa
- Mastercard
- Razorpay
- PayPal
- Adyen
- Square
Commercial implementations are substantially more sophisticated than MerchantGuard and often incorporate additional information that is not publicly available, such as:
- Historical transaction behaviour
- Chargeback trends
- Fraud intelligence
- Device fingerprinting
- Behavioural analytics
- Customer disputes
- Machine learning models
- Internal merchant history
MerchantGuard intentionally focuses only on publicly available information and transparent rule-based evaluation.
Although MerchantGuard focuses on merchant websites, the underlying architecture can be adapted to many other domains that require explainable evidence-based decision making.
Examples include:
- Merchant onboarding
- Marketplace seller verification
- Vendor risk assessment
- Third-party supplier evaluation
- Compliance automation
- Cybersecurity posture assessment
- Insurance underwriting
- Loan pre-screening
- Know Your Customer (KYC) support
- Anti-Money Laundering (AML) investigations
The architectural pattern remains the same:
Evidence Collection
│
▼
Evidence Correlation
│
▼
Rule Evaluation
│
▼
Risk Scoring
│
▼
Explainable Decision
Only the evidence sources and business rules change.
MerchantGuard can be useful for anyone interested in understanding or experimenting with explainable risk evaluation workflows.
Potential users include:
- Students exploring web crawling and rule engines
- Software engineers learning backend architecture
- Security enthusiasts interested in OSINT automation
- Researchers studying explainable risk assessment
- Businesses performing preliminary website reviews
- Developers building merchant onboarding workflows
While MerchantGuard is not intended to replace commercial underwriting systems, it provides a practical foundation for understanding how publicly available website intelligence can be transformed into structured risk observations.
MerchantGuard was developed around a few core design principles.
Each module performs a single responsibility and can evolve independently.
Every risk finding is backed by supporting evidence rather than opaque scoring.
New extraction techniques, external intelligence sources, and business rules can be added without redesigning the overall architecture.
The same evidence will always produce the same result under the current rule configuration.
The project is intentionally structured to demonstrate how multiple engineering concepts can be combined into a complete end-to-end system rather than focusing on a single technology.
MerchantGuard is intentionally designed as an educational implementation of a merchant website risk assessment engine. While it demonstrates many real-world concepts, it also has several limitations.
Current limitations include:
- Evaluations are based solely on publicly available information.
- The rule engine is deterministic and does not currently incorporate machine learning.
- Public search results depend on the availability of indexed information.
- Dynamic websites may require additional browser automation depending on their implementation.
- Merchant identity cannot always be independently verified using only website content.
- Risk scores are configurable but intentionally simplified for demonstration purposes.
- Commercial payment providers combine significantly larger datasets that are outside the scope of this project.
MerchantGuard should therefore be viewed as a foundation for understanding explainable merchant risk evaluation rather than a replacement for proprietary commercial underwriting platforms.
MerchantGuard has been intentionally designed to be modular so that additional capabilities can be introduced without changing the overall architecture.
Some planned improvements include:
- Additional merchant identity extraction
- Company registration verification
- Business registry integration
- Google Maps and business listing correlation
- Social media verification
- Expanded legal policy analysis
- Reputation feeds
- Certificate transparency logs
- IP reputation analysis
- ASN enrichment
- Domain history
- Technology fingerprinting
- Infrastructure relationship mapping
- YAML-based configurable rule packs so any non-tech person do modifications
- User-defined custom rules
- Rule dependency management
- Rule confidence scoring
- Versioned rule catalogue
- Configurable weighting
- Category-specific scoring models
- Historical comparisons
- Risk trend monitoring
- Configurable organisation policies
- Interactive HTML reports
- PDF report generation
- REST API
- Dashboard
- Web interface
- Export formats
- Scheduled evaluations
- Unit testing
- Integration testing
- Docker support
- CI/CD pipelines
- Configuration management
- Plugin architecture
- Performance optimisation
The long-term goal is to evolve MerchantGuard into a flexible research platform for experimenting with explainable merchant website risk assessment.
Contributions, suggestions, bug reports, and feature requests are always welcome.
Possible contribution areas include:
- New extraction techniques
- Additional business rules
- Infrastructure intelligence sources
- Documentation improvements
- Performance optimisation
- Bug fixes
- Test coverage
- Reporting enhancements
If you would like to contribute, please feel free to open an issue or submit a pull request.
MerchantGuard was developed as an opportunity to explore how multiple software engineering concepts can be combined into a complete end-to-end application.
The project brings together concepts from:
- Python Development
- Backend Engineering
- Web Crawling
- Browser Automation
- HTML Parsing
- Rule Engine Design
- Security Automation
- Risk Analytics
- Open Source Intelligence (OSINT)
- Explainable Decision Systems
Beyond the implementation itself, the project also served as an exploration of modular software architecture, separation of concerns, evidence modelling, and explainable decision making.
This project draws inspiration from publicly documented concepts in:
- Merchant onboarding workflows
- Risk technology (RiskTech)
- Compliance automation
- Fraud detection
- Open Source Intelligence (OSINT)
- Explainable rule-based systems
The architecture is intended for educational purposes and reflects publicly observable concepts rather than proprietary commercial implementations.
This project is released under the MIT License.
You are free to use, modify, and distribute the project in accordance with the terms of the license.
MerchantGuard began as an exploration into website analysis but gradually evolved into something much broader—a modular, explainable merchant risk assessment engine.
Rather than focusing solely on collecting website data, the project demonstrates how publicly available information can be transformed into structured evidence, correlated with external intelligence, evaluated through transparent business rules, and presented as an explainable risk assessment.
While intentionally simplified compared with commercial merchant underwriting platforms, MerchantGuard aims to provide a practical foundation for understanding the engineering principles behind evidence collection, rule-based decision systems, and explainable risk evaluation.
The project is intended to be continuously improved as new techniques, data sources, and evaluation strategies are explored.
If you found this project interesting, have suggestions for improvement, or would like to collaborate, feel free to open an issue or contribute to the repository.
Every contribution, whether it is a bug fix, a new rule, improved documentation, or an architectural suggestion, is greatly appreciated.