WebTester

An authorized web security testing tool focused on safe checks, false-positive reduction, and clear JSON/HTML reports.

Use WebTester only on websites and APIs you own or have written permission to test. The project is report-only: it does not perform exploitation, brute force, credential dumping, reverse shells, shell upload, payload execution, or destructive HTTP methods.

Installation

pip install -r requirements.txt

Quick Start (Interactive)

The fastest way to get started:

# Step 1: Run the setup wizard (creates config.yaml and .env)
python main.py setup

# Step 2: Add your auth token (saved to .env, never to config.yaml)
python main.py auth add normal_user_1 --bearer "<your-token>"

# Step 3: Verify authentication works
python main.py --check-auth

# Step 4: Run a scan
python main.py scan --profile standard

Credential Management

# Add credentials for different profiles
python main.py auth add normal_user_1 --bearer "<token>"
python main.py auth add admin_user --bearer "<admin-token>"
python main.py auth add normal_user_1 --cookie "sessionid=abc; csrftoken=xyz"

# List configured profiles (secrets are masked)
python main.py auth list

# Remove credentials
python main.py auth remove normal_user_1

# Preview resolved endpoint URLs
python main.py endpoints preview

# Validate everything before scanning
python main.py --check-config

Windows PowerShell users: You do not need to use $env: or export to set tokens. Use python main.py auth add instead — it saves tokens directly to .env which is loaded automatically on every run.

Quick Start (Manual)

# Passive checks only
python main.py --url https://example.com --profile passive

# Passive checks with optional common backend subdomain discovery
python main.py --url https://example.com --profile passive --discover-subdomains

# Include external API-like domains that are directly linked from the frontend
python main.py --url https://example.com --profile passive --include-external-linked

# Default low-risk profile
python main.py --url https://example.com --profile standard

# Probe-based checks require explicit authorization
python main.py --url https://example.com --profile aggressive --i-have-permission

# Run specific scanners only
python main.py --scanners headers,cookies,cors,https

# List all available scanners
python main.py --list-scanners

# Validate target/auth/path readiness before scanning
python main.py --check-config

# Verify the selected auth profile before scanning
python main.py --check-auth

# Choose output format
python main.py --output html
python main.py --output json
python main.py --output both

# Safely re-run evidence checks from a previous report
python main.py --config config.yaml --verify-report reports/report.json
python main.py --config config.yaml --verify-report reports/report.json --verify-index 0

# Run the local frontend/backend code scanner
python main.py --config config.yaml --scanners frontend_code --code-path .

Scan Profiles

WebTester groups scanners by risk:

• passive: observes public pages, headers, linked assets, and low-risk metadata/artifact paths.
• standard: default profile; adds low-risk active checks such as OPTIONS-only method detection, CORS/header validation, open redirect checks, and error-leakage checks.
• aggressive: probe-based checks such as XSS reflection, injection indicators, CRLF, request smuggling, directory traversal awareness, and rate-limit probing. This profile will not run unless --i-have-permission is passed.

Aggressive scanners are disabled by default. PUT, PATCH, and DELETE are not executed; risky methods are detected from OPTIONS headers only. Backend/API discovery is enabled by default and can be disabled with --no-auto-discover-backend.

Scanner Modules (40 total)

Scanner	Category	Checks
https	Transport Security	HTTPS redirect, TLS cert expiry, protocol version
headers	Security Headers	CSP, HSTS, X-Frame-Options, 10+ headers
cookies	Cookie Security	HttpOnly, Secure, SameSite flags
cors	CORS	Wildcard, reflected origin, null origin
methods	HTTP Methods	TRACE, PUT, DELETE detection
http_put_detection	HTTP Method Misconfiguration	OPTIONS-only risky method detection
backend_discovery	Backend/API Discovery	HTML, JavaScript, headers, robots, sitemaps, OpenAPI, and optional small subdomain candidate discovery
exposed_files	Exposure	.env, .git, Docker, backups, admin panels, 50+ paths
auth	Authentication	No-token access, fake token, empty auth
authorization_matrix	Authorization	Configured per-profile access-control matrix with response diffing
jwt	JWT Analysis	Algorithm, expiry, sensitive claims
xss	XSS	Reflected payload detection
injection	Injection	SQL/NoSQL error signatures
ssti	Template Injection	SSTI expression evaluation
error_leakage	Error Handling	Stack traces, verbose errors
js_secrets	Frontend Secrets	API keys, JWTs, AWS keys in JS bundles
sourcemaps	Info Disclosure	JS/CSS .map file exposure
rate_limit	Rate Limiting	Burst request throttling
api_discovery	API Security	GraphQL introspection, method tampering
open_redirect	Redirect	URL parameter redirect to external
host_header	Header Injection	Host/X-Forwarded-Host reflection
crlf	CRLF Injection	Header injection via CRLF
subdomain	Reconnaissance	Common subdomain DNS resolution
clickjacking	Clickjacking	Frame protection headers
mixed_content	Mixed Content	HTTP resources on HTTPS pages
directory_traversal	Path Traversal	../ payload detection
request_smuggling	Smuggling	CL-TE header mismatch
security_txt	Compliance	RFC 9116 security.txt
content_type	Content Security	MIME type, charset checks
wordpress	CMS Security	WP login, XML-RPC, user enum
wordpress_backup_plugins	WordPress Plugin Exposure	Backup/migration plugin detection and visible risky versions
wordpress_content_injection	WordPress Core Version Risk	WordPress 4.7.0/4.7.1 REST API content injection version-only check
pretalx_file_read	Technology Detection	pretalx exposure and configured CVE-2023-28459 version indicators
telerik_report_server_exposure	Exposed Management Interface	Telerik Report Server exposure and configured version indicators
backup_exposure	Backup Exposure	ZIP/SQL/listing signatures for public backup artifacts
grafana_exposure	Technology Exposure	Grafana fingerprints and visible risky versions
spring_cloud_config_exposure	Configuration Exposure	Spring Cloud Config and Actuator endpoint fingerprints
legacy_framework_exposure	Legacy Framework Exposure	Axis2 and Cisco ASA/FTD portal fingerprints
nginx_legacy_source_disclosure	Outdated Server Version	Legacy nginx version header checks
frontend_code	Static Code Review	Frontend secrets, API URLs, admin routes, token storage, DOM sinks, payment endpoints, CORS/CSP code patterns
external_tools	Tool Integration	Optional allowlisted external tool inventory/integration

Learning-Based Safe Checks

Several safe checks were inspired by studying Apache Axis2 LFI, Cisco ASA/FTD directory traversal, Grafana plugin traversal, Spring Cloud Config traversal, WordPress Duplicator file read, WordPress Simple Backup file read, WordPress Total Upkeep backup exposure, and nginx source disclosure.

WebTester does not exploit these vulnerabilities. It performs safe detection of exposed technologies, risky visible versions, public backup artifacts, and misconfiguration indicators. Evidence is based on product-specific fingerprints, version hints, magic bytes, SQL/listing markers, and false-positive reduction against homepage, SPA fallback, and soft-404 baselines.

Safe Checks for PUT, WordPress, pretalx, and Telerik

WebTester includes defensive checks inspired by HTTP PUT misconfiguration, historical WordPress REST API content injection, pretalx file-read risk, and Telerik Report Server authentication-bypass advisories.

• http_put_detection sends OPTIONS only and reports risky methods advertised in Allow or Access-Control-Allow-Methods. It does not upload files or execute PUT/DELETE/PATCH.
• wordpress_content_injection detects WordPress and flags WordPress 4.7.0 or 4.7.1 by visible version only. It does not modify posts, pages, or REST API content.
• pretalx_file_read detects pretalx fingerprints and optional configured vulnerable versions for CVE-2023-28459. It does not perform file-read or traversal exploitation.
• telerik_report_server_exposure detects Telerik Report Server login/admin surfaces and optional configured vulnerable versions. It does not attempt authentication bypass, create users, use default credentials, or call admin modification APIs.

These checks are defensive and non-exploitative. Use only on systems you own or have written permission to test.

Automatic Backend/API Discovery

WebTester can now start from only the main website URL. It discovers likely backend/API targets by parsing public frontend evidence:

• HTML links, forms, meta tags, inline JavaScript, and script references.
• Same-origin JavaScript bundles for API_URL, BACKEND_URL, NEXT_PUBLIC_API_URL, VITE_API_URL, axios baseURL, fetch, XHR, Apollo/GraphQL URIs, and relative API paths.
• Framework indicators such as WordPress REST API links, Next.js/Vite/React/Angular bundle references, Netlify functions, and Vercel-style /api paths.
• HTTP Link, redirect, CORS, server, and powered-by headers.
• robots.txt, sitemap.xml, and safe public OpenAPI/Swagger paths such as /openapi.json, /swagger.json, /api/docs, and /swagger-ui.
• Optional small same-domain backend subdomain guesses (api, backend, server, app, admin, auth) when --discover-subdomains is passed.

By default, WebTester keeps scanning in scope: same host, same registrable domain, subdomains, and clearly linked API-like backend domains found in frontend code or metadata. Common third-party analytics, payment, CDN, font, and monitoring domains are treated as external dependencies and are not scanned. Use --include-external-linked only when you have permission to test those linked external API domains.

Discovered API endpoints are checked with safe GET requests only. 401 and 403 responses are reported as protected, not vulnerable. A 200 JSON response is only flagged when it contains sensitive-looking field names such as email, token, password, secret, accessToken, refreshToken, role, or isAdmin; values are not stored in reports.

Reports include a discovered_targets section with URL, type, source, confidence, evidence, and in-scope status.

Project Structure

WebTester/
├── main.py                     # CLI entry point
├── config.yaml                 # Target & scanner configuration
├── requirements.txt            # Python dependencies
├── core/
│   ├── finding.py              # Finding data model (CWE/OWASP)
│   ├── http_client.py          # Rate-limited HTTP client
│   ├── response_classifier.py  # JSON/API/SPA/soft-404 classifier
│   ├── reporter.py             # JSON + HTML report generator
│   ├── verifier.py             # Safe finding verification mode
│   ├── discovered_target.py    # Data model for discovered backend targets
│   ├── scan_context.py         # Shared scan context and baselines
│   ├── scanner_registry.py     # Scanner metadata and profile resolution
│   ├── severity_engine.py      # Severity normalization helpers
│   ├── baseline.py             # Baseline 404/homepage response builder
│   ├── evidence_store.py       # Evidence redaction utilities
│   └── utils.py                # Utility functions
├── scanners/
│   ├── backend_discovery.py    # Automatic backend/API discovery (HTML, JS, headers, OpenAPI)
│   ├── exposed_files.py        # 50+ sensitive path checks
│   ├── headers.py              # 10+ security header checks
│   ├── https.py                # TLS/SSL certificate analysis
│   ├── cookies.py              # Cookie flag analysis
│   ├── cors.py                 # CORS misconfiguration
│   ├── methods.py              # HTTP method checks
│   ├── auth.py                 # Auth bypass detection
│   ├── jwt_check.py            # JWT structure analysis
│   ├── xss_reflection.py       # Reflected XSS indicators
│   ├── injection_check.py      # SQL/NoSQL injection indicators
│   ├── ssti_check.py           # Template injection indicators
│   ├── error_leakage.py        # Stack trace detection
│   ├── js_secrets.py           # Secret scanning in JS bundles
│   ├── sourcemaps.py           # Source map exposure
│   ├── rate_limit.py           # Rate limit testing
│   ├── api_discovery.py        # API/GraphQL discovery
│   ├── open_redirect.py        # Open redirect checks
│   ├── host_header.py          # Host header injection
│   ├── crlf_injection.py       # CRLF injection
│   ├── subdomain_enum.py       # Subdomain enumeration
│   ├── clickjacking.py         # Clickjacking protection
│   ├── mixed_content.py        # Mixed content detection
│   ├── directory_traversal.py  # Path traversal checks
│   ├── request_smuggling.py    # HTTP smuggling indicators
│   ├── security_txt.py         # security.txt compliance
│   ├── content_type.py         # MIME type security
│   ├── wordpress.py            # WordPress-specific checks
│   └── frontend_code.py        # Static frontend/backend code review
├── utils/
│   └── url_extractor.py        # URL extraction, scope filtering, and API path detection
└── reports/
    ├── report.json             # Generated JSON report
    └── report.html             # Generated HTML report

Configuration

Edit config.yaml to set your frontend URL. A separate backend API base URL is optional because WebTester can discover likely backend/API targets automatically:

target:
  frontend_url: "https://www.example.com"
  api_base_url: ""  # optional; auto-discovery can find linked API bases
  base_url: "https://www.example.com"  # backward-compatible fallback

scanner:
  timeout: 10
  delay_seconds: 0.5
  user_agent: "WebTester/2.0"
  rate_limit_attempts: 20
  safe_delay_seconds: 0.2
  max_requests_per_endpoint: 25
  xss_playwright: false

backend_discovery:
  enabled: true
  max_js_files: 20
  max_discovered_targets: 30

codebase:
  enabled: false
  path: "."
  max_file_bytes: 2000000
  include_low_confidence: false
  skip_dirs:
    - "node_modules"
    - ".git"
    - "reports"

Frontend scanners use frontend_url. API, auth, injection, error-leakage, rate-limit, and traversal scanners use api_base_url when configured, or a high-confidence discovered API base when available. API findings still require confirmed JSON API evidence and safe request behavior.

The frontend_code scanner is read-only and reports local file/line metadata in JSON and HTML reports. Set codebase.path, pass --code-path, or remove frontend_code from a profile if you only want remote HTTP checks.

Frontend Code Scanner

code scanner checks for:

• Exposed high-confidence secrets and API keys, including Stripe secret keys, private keys, AWS access key IDs, JWT secrets, and generic API secrets.
• Frontend-visible backend URLs, admin routes, payment/subscription endpoints, paid feature endpoints, and source map references.
• Browser-accessible token storage in localStorage or sessionStorage.
• Dangerous DOM and code execution sinks such as dangerouslySetInnerHTML, innerHTML, insertAdjacentHTML, document.write, eval, new Function, and string-based timers.
• CORS/CSP code patterns such as wildcard origins, credentials-enabled CORS, unsafe-inline, and unsafe-eval.
• Compound risks, for example token storage plus dangerous DOM sinks in the same file.

The scanner skips common dependency/build/report directories and ignores Markdown/test/example matches to reduce noise.

Reliability Model

Every finding includes status, confidence, response_type, manual_verification_required, and false_positive_risk.

Critical findings require status=confirmed, confidence=High, and evidence beyond Status: 200. HTML app shells, login pages, custom 404 pages, and SPA fallbacks are suppressed or downgraded for auth/API/exposure tests.

Example suppressed result:

[Info] API-like path returned HTML fallback
Status: false_positive_suppressed
Response Type: spa_fallback
Evidence: Content-Type text/html, body contains <div id="root">, no JSON API data
Recommendation: Use the actual backend API base URL for API tests.

Example confirmed result:

[Critical] Authentication bypass confirmed
Status: confirmed
Response Type: json_api
Evidence: Missing-auth request returned 200 JSON containing id, email, profile fields
Recommendation: Require authentication middleware and validate tokens on every protected route.

Authorization Matrix

Configure authorization_matrix.endpoints with API paths relative to target.api_base_url. For example, if api_base_url is https://example.com/api/v1, use /auth/me, not /api/v1/auth/me.

The scanner sends each endpoint as anonymous, normal_user_1, normal_user_2, and admin_user, then reports Endpoint | Method | Anonymous | Normal User 1 | Normal User 2 | Admin | Finding with each cell showing status_code response_type.

Set credentials through environment variables, not in config.yaml:

$env:NORMAL_USER_1_AUTH_HEADER="Bearer USER_1_TOKEN"
$env:NORMAL_USER_2_AUTH_HEADER="Bearer USER_2_TOKEN"
$env:ADMIN_USER_AUTH_HEADER="Bearer ADMIN_TOKEN"
$env:NORMAL_USER_1_ID="user_1_id_here"
$env:NORMAL_USER_2_ID="user_2_id_here"

Confirmed Critical authorization findings require a 200 JSON API response with private-data indicators where access was not expected. SPA fallbacks, login pages, and soft 404s are suppressed as false positives.

Reports

Reports are saved to the reports/ directory:

• JSON — machine-readable findings with CWE/OWASP mappings
• HTML — styled dark-mode report with severity badges and recommendations

Report summaries include confirmed critical/high counts, suspected findings, informational observations, false positives suppressed, soft-404 path counts, and SPA fallback path counts. Suppressed false positives are not counted as vulnerabilities.