From dd1fc4e74c19ecd8cead43597950599d04cd3c1c Mon Sep 17 00:00:00 2001 From: GovindarajanL Date: Tue, 4 Mar 2025 18:35:54 -0600 Subject: [PATCH 1/4] docs: Add comprehensive framework documentation for contributors and users - Add FRAMEWORK_OVERVIEW.md explaining capabilities and architecture - Add ARCHITECTURE.md detailing component design and interfaces These documents provide clear guidance for new contributors on project structure, implementation details, and future direction while helping users understand how to effectively utilize the framework. --- README.md | 8 ++ docs/ARCHITECTURE.md | 183 +++++++++++++++++++++++++++++++++++++ docs/FRAMEWORK_OVERVIEW.md | 106 +++++++++++++++++++++ 3 files changed, 297 insertions(+) create mode 100644 docs/ARCHITECTURE.md create mode 100644 docs/FRAMEWORK_OVERVIEW.md diff --git a/README.md b/README.md index e5bc360..2dcc231 100644 --- a/README.md +++ b/README.md @@ -67,6 +67,14 @@ api-security-testing-framework/ For more detailed information, please refer to our [Documentation](docs/README.md). +## Framework Overview + +For detailed understand on the framework, please refer to our [Framework Overview](docs/FRAMEWORK_OVERVIEW.md). + +## Architecture + +Please refer to our [Architecture](docs/ARCHITECTURE.md). + ## Contributing We welcome contributions from the community! Please see our [Contributing Guidelines](CONTRIBUTING.md) for more information on how to get involved. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..5613f45 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,183 @@ +# Framework Architecture + +## Overview + +The OWASP API Security Testing Framework is designed with a modular architecture to allow for extensibility and maintainability. This document outlines the high-level architecture and key components of the framework. + +## Component Architecture + +``` ++----------------------------+ +| CLI Layer | ++----------------------------+ + | ++----------------------------+ +| Scanner Core | ++----------------------------+ + / | \ + / | \ ++--------+ +--------+ +--------+ +| HTTP | | Test | | Report | +| Client | | Cases | | Engine | ++--------+ +--------+ +--------+ +``` + +### Key Components + +1. **CLI Layer** (`org.owasp.astf.cli`) + - Parses command-line arguments + - Configures and initializes the scanner + - Handles user interaction + - Manages input/output + +2. **Scanner Core** (`org.owasp.astf.core`) + - Orchestrates the scanning process + - Manages test case execution + - Handles multi-threading and concurrency + - Collects and processes results + +3. **HTTP Client** (`org.owasp.astf.core.http`) + - Manages API communications + - Handles authentication + - Processes requests and responses + - Supports various HTTP methods and content types + +4. **Test Cases** (`org.owasp.astf.testcases`) + - Individual security test implementations + - Each test case targets specific vulnerability types + - Implements the TestCase interface + - Registered and managed by TestCaseRegistry + +5. **Report Engine** (`org.owasp.astf.reporting`) + - Generates reports in various formats (JSON, HTML, XML, SARIF) + - Formats findings with appropriate details + - Supports different output destinations + +6. **Integrations** (`org.owasp.astf.integrations`) + - CI/CD integration components + - External tool connectors + - Notification systems + +## Data Flow + +1. User invokes the CLI with scan parameters +2. CLI configures the scanner with appropriate settings +3. Scanner discovers or loads target endpoints +4. For each endpoint, applicable test cases are executed +5. Test cases use the HTTP client to make API requests +6. Findings are collected by the scanner +7. Report engine generates the requested output format +8. Results are returned to the user + +## Key Interfaces + +### TestCase Interface + +```java +public interface TestCase { + String getId(); + String getName(); + String getDescription(); + List execute(EndpointInfo endpoint, HttpClient httpClient) throws IOException; +} +``` + +All test cases implement this interface, allowing the scanner to execute them uniformly. + +### EndpointInfo Class + +```java +public class EndpointInfo { + private String path; + private String method; + private String contentType; + private String requestBody; + private boolean requiresAuthentication; + + // Constructors, getters, etc. +} +``` + +Represents an API endpoint to be tested, including path, method, and metadata. + +### Finding Class + +```java +public class Finding { + private String id; + private String title; + private String description; + private Severity severity; + private String testCaseId; + private String endpoint; + private String requestDetails; + private String responseDetails; + private String remediation; + private String evidence; + + // Constructors, getters, etc. +} +``` + +Represents a security finding with all relevant details. + +## Design Principles + +1. **Modularity**: Components are designed with clear boundaries +2. **Extensibility**: Easy to add new test cases and functionality +3. **Testability**: Components can be tested in isolation +4. **Performance**: Efficient execution for large API surfaces +5. **Usability**: Clear interfaces and documentation + +## Thread Model + +The scanner uses a thread pool to execute test cases concurrently: + +1. One thread per endpoint-testcase combination +2. Configurable thread count via `--threads` option +3. Uses Java 21 virtual threads for efficiency +4. Results are synchronized to prevent race conditions + +## Adding New Test Cases + +To add a new test case: + +1. Create a class implementing the `TestCase` interface +2. Implement the required methods +3. Register the test case in `TestCaseRegistry.registerDefaultTestCases()` +4. Add unit tests for the new test case + +Example: + +```java +public class NewVulnerabilityTestCase implements TestCase { + @Override + public String getId() { + return "ASTF-API11-2023"; + } + + @Override + public String getName() { + return "New Vulnerability"; + } + + @Override + public String getDescription() { + return "Tests for a new type of vulnerability"; + } + + @Override + public List execute(EndpointInfo endpoint, HttpClient httpClient) throws IOException { + // Implement vulnerability detection logic + // Return list of findings (or empty list if none found) + } +} +``` + +## Future Architecture Enhancements + +1. Plugin system for custom test cases +2. Distributed scanning capabilities +3. Real-time reporting and notification +4. Machine learning-based detection improvements +5. Integration with vulnerability management platforms \ No newline at end of file diff --git a/docs/FRAMEWORK_OVERVIEW.md b/docs/FRAMEWORK_OVERVIEW.md new file mode 100644 index 0000000..d7bb972 --- /dev/null +++ b/docs/FRAMEWORK_OVERVIEW.md @@ -0,0 +1,106 @@ +# OWASP API Security Testing Framework Overview + +## What This Framework Does + +The OWASP API Security Testing Framework (ASTF) is a comprehensive tool designed to identify security vulnerabilities in APIs based on the OWASP API Security Top 10. Unlike traditional security tools, ASTF specifically focuses on API-specific vulnerabilities that are often missed by general-purpose scanners. + +## How It Works + +The framework operates using a black-box testing approach: + +1. **Endpoint Discovery**: Automatically discovers API endpoints through: + - OpenAPI/Swagger specification parsing + - Common endpoint pattern testing + - Intelligent path traversal + - Manual endpoint specification + +2. **Security Testing**: Executes a comprehensive suite of tests targeting: + - API1:2023 - Broken Object Level Authorization + - API2:2023 - Broken Authentication + - API3:2023 - Excessive Data Exposure + - API4:2023 - Lack of Resources & Rate Limiting + - API5:2023 - Broken Function Level Authorization + - (Additional test cases in future releases) + +3. **Vulnerability Reporting**: Provides detailed findings including: + - Severity classification + - Vulnerability details + - Evidence capture + - Remediation guidance + - References to OWASP standards + +## Key Capabilities + +### Dynamic API Testing + +* Tests live API endpoints without needing source code +* Detects vulnerabilities through intelligent request manipulation +* Identifies security issues that affect APIs specifically + +### Authentication & Authorization Testing + +* Tests for improper access controls +* Detects weak authentication mechanisms +* Identifies JWT vulnerabilities +* Tests for privilege escalation + +### Data Protection Analysis + +* Identifies sensitive data exposure +* Detects missing encryption +* Finds excessive data in responses + +### Resource Protection + +* Tests for missing rate limiting +* Identifies DoS vulnerabilities +* Detects resource consumption issues + +### Integration Capabilities + +* CI/CD pipeline integration +* SARIF output for security dashboards +* HTML reports for stakeholders +* Command-line interface for scripting + +## Use Cases + +### Development Teams + +* Test APIs during development +* Integrate security testing into CI/CD pipelines +* Validate security controls before deployment + +### Security Teams + +* Assess API security posture +* Validate vendor API security +* Perform regular security assessments + +### DevSecOps + +* Automate API security testing +* Generate compliance evidence +* Track security improvements over time + +## Technical Architecture + +The framework is built on a modular Java architecture: + +* **Core Engine**: Manages test execution and coordination +* **HTTP Client**: Handles API communications and request manipulation +* **Test Cases**: Modular, extensible security tests +* **Reporting Engine**: Generates findings in multiple formats +* **CLI Interface**: Provides user interaction and configuration + +## Intended Audience + +The ASTF is designed for: + +* Security engineers +* API developers +* DevOps engineers +* Security consultants +* Quality assurance testers + +No deep security expertise is required to run basic scans, but security knowledge helps interpret results and implement fixes. \ No newline at end of file From e71f89bc02eaf5520899d9f41226a24d05a3b1e9 Mon Sep 17 00:00:00 2001 From: GovindarajanL Date: Fri, 7 Mar 2025 14:23:01 -0600 Subject: [PATCH 2/4] docs: Add Apache License 2.0 Add Apache License 2.0 to formalize the project's open source status. This license allows users to freely use, modify, distribute, and contribute to the OWASP API Security Testing Framework while requiring attribution and providing basic liability protection. --- LICENSE.md | 177 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 176 insertions(+), 1 deletion(-) diff --git a/LICENSE.md b/LICENSE.md index 480cb22..5b66d9f 100644 --- a/LICENSE.md +++ b/LICENSE.md @@ -1 +1,176 @@ -# TODO: Please update this file with the license of your project \ No newline at end of file +Apache License +Version 2.0, January 2004 +http://www.apache.org/licenses/ + +TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + +1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + +2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + +3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + +4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + +5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + +6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + +7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + +8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + +9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + +END OF TERMS AND CONDITIONS \ No newline at end of file From 25703f3db5a713f491ad56483f5d24da3741db20 Mon Sep 17 00:00:00 2001 From: GovindarajanL Date: Sat, 8 Mar 2025 11:52:30 -0600 Subject: [PATCH 3/4] feat(core): Implement advanced configuration and discovery components Add comprehensive framework components: - ConfigLoader for flexible configuration from multiple sources - EndpointDiscoveryService for automatic API endpoint discovery - Enhanced HttpClient with advanced authentication and proxy support - Expanded ScanConfig with complete configuration options - Improved Scanner with optimized thread management These components provide the foundation for robust API security testing with support for various authentication methods, automatic endpoint detection, and flexible configuration options. --- pom.xml | 12 + .../java/org/owasp/astf/core/Scanner.java | 225 +++++--- .../owasp/astf/core/config/ConfigLoader.java | 414 ++++++++++++++ .../owasp/astf/core/config/ScanConfig.java | 528 +++++++++++++++++- .../discovery/EndpointDiscoveryService.java | 470 ++++++++++++++++ .../org/owasp/astf/core/http/HttpClient.java | 356 ++++++++++-- 6 files changed, 1836 insertions(+), 169 deletions(-) create mode 100644 src/main/java/org/owasp/astf/core/config/ConfigLoader.java create mode 100644 src/main/java/org/owasp/astf/core/discovery/EndpointDiscoveryService.java diff --git a/pom.xml b/pom.xml index e901e5d..0ed2774 100644 --- a/pom.xml +++ b/pom.xml @@ -96,6 +96,18 @@ ${junit.version} test + + + com.fasterxml.jackson.dataformat + jackson-dataformat-yaml + ${jackson.version} + + + + com.fasterxml.jackson.core + jackson-databind + ${jackson.version} + diff --git a/src/main/java/org/owasp/astf/core/Scanner.java b/src/main/java/org/owasp/astf/core/Scanner.java index 7ecad3d..3703b89 100644 --- a/src/main/java/org/owasp/astf/core/Scanner.java +++ b/src/main/java/org/owasp/astf/core/Scanner.java @@ -1,15 +1,20 @@ package org.owasp.astf.core; +import java.time.LocalDateTime; import java.util.ArrayList; import java.util.List; import java.util.Map; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicInteger; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; import org.owasp.astf.core.config.ScanConfig; +import org.owasp.astf.core.discovery.EndpointDiscoveryService; import org.owasp.astf.core.http.HttpClient; import org.owasp.astf.core.result.Finding; import org.owasp.astf.core.result.ScanResult; @@ -19,6 +24,14 @@ /** * The main scanner engine that orchestrates the API security testing process. + * This class is responsible for: + *
    + *
  • Initializing and executing the scan based on configuration
    • + *
  • Managing endpoint discovery or using provided endpoints
    • + *
  • Coordinating test case execution across endpoints
    • + *
  • Collecting and aggregating findings
    • + *
  • Providing progress updates and metrics
    • + *
*/ public class Scanner { private static final Logger logger = LogManager.getLogger(Scanner.class); @@ -26,119 +39,169 @@ public class Scanner { private final ScanConfig config; private final HttpClient httpClient; private final TestCaseRegistry testCaseRegistry; + private final EndpointDiscoveryService discoveryService; + // Scan metrics and tracking + private final AtomicInteger completedTasks = new AtomicInteger(0); + private final AtomicInteger totalTasks = new AtomicInteger(0); + private final Map findingsBySeverity = new ConcurrentHashMap<>(); + private LocalDateTime scanStartTime; + private LocalDateTime scanEndTime; + + /** + * Creates a new scanner with the specified configuration. + * + * @param config The scan configuration + */ public Scanner(ScanConfig config) { this.config = config; this.httpClient = new HttpClient(config); this.testCaseRegistry = new TestCaseRegistry(); + this.discoveryService = new EndpointDiscoveryService(config, httpClient); + + // Initialize severity counters + for (Severity severity : Severity.values()) { + findingsBySeverity.put(severity, new AtomicInteger(0)); + } } /** * Executes a full scan based on the provided configuration. * - * @return The scan results containing all findings. + * @return The scan results containing all findings */ public ScanResult scan() { - logger.info("Starting API security scan for target: {}", config.getTargetUrl()); - + scanStartTime = LocalDateTime.now(); List findings = new ArrayList<>(); - // Determine if we need to discover endpoints or use provided ones - List endpoints = new ArrayList<>(); - if (config.isDiscoveryEnabled() && config.getEndpoints().isEmpty()) { - endpoints = discoverEndpoints(); - } else { - endpoints = config.getEndpoints(); - } + try { + logger.info("Starting API security scan for target: {}", config.getTargetUrl()); + + // Determine if we need to discover endpoints or use provided ones + List endpoints = new ArrayList<>(); + if (config.isDiscoveryEnabled() && config.getEndpoints().isEmpty()) { + logger.info("No endpoints provided. Attempting endpoint discovery..."); + endpoints = discoverEndpoints(); + } else { + endpoints = config.getEndpoints(); + logger.info("Using {} provided endpoints", endpoints.size()); + } + + if (endpoints.isEmpty()) { + logger.warn("No endpoints found to scan. Check target URL or provide endpoints manually."); + return createEmptyScanResult(); + } - logger.info("Found {} endpoints to scan", endpoints.size()); - - // Get applicable test cases - List testCases = testCaseRegistry.getEnabledTestCases(config); - logger.info("Running {} test cases", testCases.size()); - - // Run test cases against endpoints - ExecutorService executor = Executors.newFixedThreadPool(config.getThreads()); - - for (EndpointInfo endpoint : endpoints) { - for (TestCase testCase : testCases) { - executor.submit(() -> { - try { - List testFindings = testCase.execute(endpoint, httpClient); - synchronized (findings) { - findings.addAll(testFindings); - } - } catch (Exception e) { - logger.error("Error executing test case {} on endpoint {}: {}", - testCase.getId(), endpoint.getPath(), e.getMessage()); + // Get applicable test cases + List testCases = testCaseRegistry.getEnabledTestCases(config); + logger.info("Running {} test cases against {} endpoints", testCases.size(), endpoints.size()); + + // Calculate total tasks for progress tracking + totalTasks.set(endpoints.size() * testCases.size()); + + // Run test cases against endpoints using virtual threads (Java 21) + try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) { + List> futures = new ArrayList<>(); + + for (EndpointInfo endpoint : endpoints) { + for (TestCase testCase : testCases) { + CompletableFuture future = CompletableFuture.runAsync(() -> { + try { + logger.debug("Executing {} on {}", testCase.getId(), endpoint); + List testFindings = testCase.execute(endpoint, httpClient); + + if (!testFindings.isEmpty()) { + synchronized (findings) { + findings.addAll(testFindings); + + // Update severity counters + for (Finding finding : testFindings) { + findingsBySeverity.get(finding.getSeverity()).incrementAndGet(); + } + } + + logger.debug("Found {} issues with {} on {}", + testFindings.size(), testCase.getId(), endpoint); + } + } catch (Exception e) { + logger.error("Error executing test case {} on endpoint {}: {}", + testCase.getId(), endpoint.getPath(), e.getMessage()); + logger.debug("Exception details:", e); + } finally { + // Update progress + int completed = completedTasks.incrementAndGet(); + if (completed % 10 == 0 || completed == totalTasks.get()) { + logProgress(); + } + } + }, executor); + + futures.add(future); } - }); + } + + // Wait for all tasks to complete or timeout + CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])) + .orTimeout(config.getTimeoutMinutes(), TimeUnit.MINUTES) + .exceptionally(ex -> { + logger.warn("Scan interrupted or timed out before completion: {}", ex.getMessage()); + return null; + }) + .join(); } - } - executor.shutdown(); - try { - executor.awaitTermination(config.getTimeoutMinutes(), TimeUnit.MINUTES); - } catch (InterruptedException e) { - logger.warn("Scan interrupted before completion"); - Thread.currentThread().interrupt(); + logger.info("Scan completed. Found {} issues: {} critical, {} high, {} medium, {} low, {} info", + findings.size(), + findingsBySeverity.get(Severity.CRITICAL).get(), + findingsBySeverity.get(Severity.HIGH).get(), + findingsBySeverity.get(Severity.MEDIUM).get(), + findingsBySeverity.get(Severity.LOW).get(), + findingsBySeverity.get(Severity.INFO).get()); + + } catch (Exception e) { + logger.error("Unhandled exception during scan: {}", e.getMessage()); + logger.debug("Exception details:", e); } + scanEndTime = LocalDateTime.now(); ScanResult result = new ScanResult(config.getTargetUrl(), findings); - logger.info("Scan completed. Found {} issues: {} high, {} medium, {} low severity", - findings.size(), - findings.stream().filter(f -> f.getSeverity() == Severity.HIGH).count(), - findings.stream().filter(f -> f.getSeverity() == Severity.MEDIUM).count(), - findings.stream().filter(f -> f.getSeverity() == Severity.LOW).count()); + result.setScanStartTime(scanStartTime); + result.setScanEndTime(scanEndTime); return result; } /** * Attempts to discover API endpoints for the target. - * This is a basic implementation that uses common paths and OpenAPI detection. * * @return A list of discovered endpoints */ private List discoverEndpoints() { - logger.info("Attempting to discover API endpoints"); - List endpoints = new ArrayList<>(); - - // Try to find OpenAPI/Swagger specification - List specPaths = List.of( - "/swagger/v1/swagger.json", - "/swagger.json", - "/api-docs", - "/v2/api-docs", - "/v3/api-docs", - "/openapi.json" - ); - - for (String path : specPaths) { - try { - String url = config.getTargetUrl() + path; - String response = httpClient.get(url, Map.of()); - - if (response != null && !response.isEmpty()) { - logger.info("Found potential API specification at: {}", url); - // TODO: Parse OpenAPI spec and extract endpoints - break; - } - } catch (Exception e) { - // Continue with next path - } - } + return discoveryService.discoverEndpoints(); + } - // If no endpoints were found through specifications, return some common ones for testing - if (endpoints.isEmpty()) { - logger.info("No API spec found, using common paths for testing"); - endpoints.add(new EndpointInfo("/api/v1/users", "GET")); - endpoints.add(new EndpointInfo("/api/v1/users", "POST")); - endpoints.add(new EndpointInfo("/api/v1/users/{id}", "GET")); - endpoints.add(new EndpointInfo("/api/v1/auth/login", "POST")); - endpoints.add(new EndpointInfo("/api/v1/products", "GET")); - } + /** + * Logs the current progress of the scan. + */ + private void logProgress() { + int completed = completedTasks.get(); + int total = totalTasks.get(); + double percentComplete = (double) completed / total * 100; + + logger.info("Scan progress: {}% ({}/{} tasks completed)", + String.format("%.1f", percentComplete), completed, total); + } - return endpoints; + /** + * Creates an empty scan result when no endpoints are found. + * + * @return An empty scan result + */ + private ScanResult createEmptyScanResult() { + scanEndTime = LocalDateTime.now(); + ScanResult result = new ScanResult(config.getTargetUrl(), List.of()); + result.setScanStartTime(scanStartTime); + result.setScanEndTime(scanEndTime); + return result; } } \ No newline at end of file diff --git a/src/main/java/org/owasp/astf/core/config/ConfigLoader.java b/src/main/java/org/owasp/astf/core/config/ConfigLoader.java new file mode 100644 index 0000000..10ac41f --- /dev/null +++ b/src/main/java/org/owasp/astf/core/config/ConfigLoader.java @@ -0,0 +1,414 @@ +package org.owasp.astf.core.config; + +import java.io.File; +import java.io.FileInputStream; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Paths; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.Properties; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.owasp.astf.core.EndpointInfo; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.dataformat.yaml.YAMLFactory; + +/** + * Loads configuration from various sources such as files, environment variables, + * and command-line arguments. + *

+ * This class provides methods to: + *

    + *
  • Load configuration from YAML/JSON files
    • + *
  • Load configuration from properties files
    • + *
  • Load configuration from environment variables
    • + *
  • Load endpoints from text files
    • + *
  • Merge configurations from multiple sources
    • + *
+ *

+ */ +public class ConfigLoader { + private static final Logger logger = LogManager.getLogger(ConfigLoader.class); + + private final ObjectMapper jsonMapper; + private final ObjectMapper yamlMapper; + + /** + * Creates a new configuration loader. + */ + public ConfigLoader() { + this.jsonMapper = new ObjectMapper(); + this.yamlMapper = new ObjectMapper(new YAMLFactory()); + } + + /** + * Loads a configuration from a YAML or JSON file. + * + * @param filePath The path to the configuration file + * @return The scan configuration + * @throws IOException If the file cannot be read or parsed + */ + public ScanConfig loadFromFile(String filePath) throws IOException { + File file = new File(filePath); + if (!file.exists() || !file.isFile()) { + throw new IOException("Configuration file not found: " + filePath); + } + + ScanConfig config = new ScanConfig(); + + // Determine file type by extension + if (filePath.endsWith(".yaml") || filePath.endsWith(".yml")) { + JsonNode root = yamlMapper.readTree(file); + parseJsonConfig(root, config); + } else if (filePath.endsWith(".json")) { + JsonNode root = jsonMapper.readTree(file); + parseJsonConfig(root, config); + } else if (filePath.endsWith(".properties")) { + Properties props = new Properties(); + try (FileInputStream fis = new FileInputStream(file)) { + props.load(fis); + } + parsePropertiesConfig(props, config); + } else { + throw new IOException("Unsupported configuration file format: " + filePath); + } + + logger.info("Loaded configuration from file: {}", filePath); + return config; + } + + /** + * Loads endpoints from a text file. + * + * @param filePath The path to the endpoints file + * @return A list of endpoints + * @throws IOException If the file cannot be read + */ + public List loadEndpointsFromFile(String filePath) throws IOException { + File file = new File(filePath); + if (!file.exists() || !file.isFile()) { + throw new IOException("Endpoints file not found: " + filePath); + } + + List endpoints = new ArrayList<>(); + List lines = Files.readAllLines(Paths.get(filePath)); + + for (String line : lines) { + // Skip comments and empty lines + line = line.trim(); + if (line.isEmpty() || line.startsWith("#")) { + continue; + } + + String[] parts = line.split("\\s+", 2); + if (parts.length == 2) { + String method = parts[0].trim(); + String path = parts[1].trim(); + endpoints.add(new EndpointInfo(path, method)); + } else { + logger.warn("Invalid endpoint format: {}. Expected 'METHOD PATH'", line); + } + } + + logger.info("Loaded {} endpoints from file: {}", endpoints.size(), filePath); + return endpoints; + } + + /** + * Loads configuration from environment variables. + * + * @param prefix The prefix for environment variables (e.g., "ASTF_") + * @param config The configuration to update + */ + public void loadFromEnvironment(String prefix, ScanConfig config) { + Map env = System.getenv(); + + // Process environment variables with the specified prefix + for (Map.Entry entry : env.entrySet()) { + String key = entry.getKey(); + String value = entry.getValue(); + + if (key.startsWith(prefix)) { + String configKey = key.substring(prefix.length()).toLowerCase(); + applyConfigValue(config, configKey, value); + } + } + + logger.info("Loaded configuration from environment variables with prefix: {}", prefix); + } + + /** + * Loads configuration from system properties. + * + * @param prefix The prefix for system properties (e.g., "astf.") + * @param config The configuration to update + */ + public void loadFromSystemProperties(String prefix, ScanConfig config) { + Properties sysProps = System.getProperties(); + + // Process system properties with the specified prefix + for (String key : sysProps.stringPropertyNames()) { + if (key.startsWith(prefix)) { + String configKey = key.substring(prefix.length()).toLowerCase(); + String value = sysProps.getProperty(key); + applyConfigValue(config, configKey, value); + } + } + + logger.info("Loaded configuration from system properties with prefix: {}", prefix); + } + + /** + * Parses a JSON/YAML configuration. + * + * @param root The root JSON node + * @param config The configuration to update + */ + private void parseJsonConfig(JsonNode root, ScanConfig config) { + // Basic settings + if (root.has("targetUrl")) { + config.setTargetUrl(root.get("targetUrl").asText()); + } + + if (root.has("outputFile")) { + config.setOutputFile(root.get("outputFile").asText()); + } + + if (root.has("outputFormat")) { + String format = root.get("outputFormat").asText().toUpperCase(); + try { + config.setOutputFormat(ScanConfig.OutputFormat.valueOf(format)); + } catch (IllegalArgumentException e) { + logger.warn("Invalid output format: {}. Using default: {}", format, config.getOutputFormat()); + } + } + + if (root.has("threads")) { + config.setThreads(root.get("threads").asInt()); + } + + if (root.has("timeoutMinutes")) { + config.setTimeoutMinutes(root.get("timeoutMinutes").asInt()); + } + + if (root.has("discoveryEnabled")) { + config.setDiscoveryEnabled(root.get("discoveryEnabled").asBoolean()); + } + + if (root.has("verbose")) { + config.setVerbose(root.get("verbose").asBoolean()); + } + + // Headers + if (root.has("headers") && root.get("headers").isObject()) { + JsonNode headers = root.get("headers"); + headers.fields().forEachRemaining(entry -> { + config.addHeader(entry.getKey(), entry.getValue().asText()); + }); + } + + // Proxy settings + if (root.has("proxy")) { + JsonNode proxy = root.get("proxy"); + if (proxy.has("host")) { + config.setProxyHost(proxy.get("host").asText()); + } + if (proxy.has("port")) { + config.setProxyPort(proxy.get("port").asInt()); + } + if (proxy.has("username")) { + config.setProxyUsername(proxy.get("username").asText()); + } + if (proxy.has("password")) { + config.setProxyPassword(proxy.get("password").asText()); + } + } + + // Basic auth + if (root.has("basicAuth")) { + JsonNode basicAuth = root.get("basicAuth"); + if (basicAuth.has("username")) { + config.setBasicAuthUsername(basicAuth.get("username").asText()); + } + if (basicAuth.has("password")) { + config.setBasicAuthPassword(basicAuth.get("password").asText()); + } + } + + // Test case configuration + if (root.has("enableTestCases") && root.get("enableTestCases").isArray()) { + JsonNode enableTests = root.get("enableTestCases"); + List enabledTestCaseIds = new ArrayList<>(); + enableTests.forEach(node -> enabledTestCaseIds.add(node.asText())); + config.setEnabledTestCaseIds(enabledTestCaseIds); + } + + if (root.has("disableTestCases") && root.get("disableTestCases").isArray()) { + JsonNode disableTests = root.get("disableTestCases"); + List disabledTestCaseIds = new ArrayList<>(); + disableTests.forEach(node -> disabledTestCaseIds.add(node.asText())); + config.setDisabledTestCaseIds(disabledTestCaseIds); + } + } + + /** + * Parses a properties configuration. + * + * @param props The properties + * @param config The configuration to update + */ + private void parsePropertiesConfig(Properties props, ScanConfig config) { + // Basic settings + if (props.containsKey("targetUrl")) { + config.setTargetUrl(props.getProperty("targetUrl")); + } + + if (props.containsKey("outputFile")) { + config.setOutputFile(props.getProperty("outputFile")); + } + + if (props.containsKey("outputFormat")) { + String format = props.getProperty("outputFormat").toUpperCase(); + try { + config.setOutputFormat(ScanConfig.OutputFormat.valueOf(format)); + } catch (IllegalArgumentException e) { + logger.warn("Invalid output format: {}. Using default: {}", format, config.getOutputFormat()); + } + } + + if (props.containsKey("threads")) { + config.setThreads(Integer.parseInt(props.getProperty("threads"))); + } + + if (props.containsKey("timeoutMinutes")) { + config.setTimeoutMinutes(Integer.parseInt(props.getProperty("timeoutMinutes"))); + } + + if (props.containsKey("discoveryEnabled")) { + config.setDiscoveryEnabled(Boolean.parseBoolean(props.getProperty("discoveryEnabled"))); + } + + if (props.containsKey("verbose")) { + config.setVerbose(Boolean.parseBoolean(props.getProperty("verbose"))); + } + + // Headers + for (String key : props.stringPropertyNames()) { + if (key.startsWith("header.")) { + String headerName = key.substring("header.".length()); + config.addHeader(headerName, props.getProperty(key)); + } + } + + // Proxy settings + if (props.containsKey("proxy.host")) { + config.setProxyHost(props.getProperty("proxy.host")); + } + if (props.containsKey("proxy.port")) { + config.setProxyPort(Integer.parseInt(props.getProperty("proxy.port"))); + } + if (props.containsKey("proxy.username")) { + config.setProxyUsername(props.getProperty("proxy.username")); + } + if (props.containsKey("proxy.password")) { + config.setProxyPassword(props.getProperty("proxy.password")); + } + + // Basic auth + if (props.containsKey("basicAuth.username")) { + config.setBasicAuthUsername(props.getProperty("basicAuth.username")); + } + if (props.containsKey("basicAuth.password")) { + config.setBasicAuthPassword(props.getProperty("basicAuth.password")); + } + + // Test case configuration + if (props.containsKey("enableTestCases")) { + String enableTests = props.getProperty("enableTestCases"); + String[] testIds = enableTests.split(","); + List enabledTestCaseIds = new ArrayList<>(); + for (String id : testIds) { + id = id.trim(); + if (!id.isEmpty()) { + enabledTestCaseIds.add(id); + } + } + config.setEnabledTestCaseIds(enabledTestCaseIds); + } + + if (props.containsKey("disableTestCases")) { + String disableTests = props.getProperty("disableTestCases"); + String[] testIds = disableTests.split(","); + List disabledTestCaseIds = new ArrayList<>(); + for (String id : testIds) { + id = id.trim(); + if (!id.isEmpty()) { + disabledTestCaseIds.add(id); + } + } + config.setDisabledTestCaseIds(disabledTestCaseIds); + } + } + + /** + * Applies a configuration value to the specified configuration. + * + * @param config The configuration to update + * @param key The configuration key + * @param value The configuration value + */ + private void applyConfigValue(ScanConfig config, String key, String value) { + switch (key) { + case "targeturl" -> config.setTargetUrl(value); + case "outputfile" -> config.setOutputFile(value); + case "outputformat" -> { + try { + config.setOutputFormat(ScanConfig.OutputFormat.valueOf(value.toUpperCase())); + } catch (IllegalArgumentException e) { + logger.warn("Invalid output format: {}. Using default: {}", value, config.getOutputFormat()); + } + } + case "threads" -> config.setThreads(Integer.parseInt(value)); + case "timeoutminutes" -> config.setTimeoutMinutes(Integer.parseInt(value)); + case "discoveryenabled" -> config.setDiscoveryEnabled(Boolean.parseBoolean(value)); + case "verbose" -> config.setVerbose(Boolean.parseBoolean(value)); + case "proxy_host" -> config.setProxyHost(value); + case "proxy_port" -> config.setProxyPort(Integer.parseInt(value)); + case "proxy_username" -> config.setProxyUsername(value); + case "proxy_password" -> config.setProxyPassword(value); + case "basicauth_username" -> config.setBasicAuthUsername(value); + case "basicauth_password" -> config.setBasicAuthPassword(value); + default -> { + if (key.startsWith("header_")) { + String headerName = key.substring("header_".length()); + config.addHeader(headerName, value); + } else if (key.equals("enabletestcases")) { + String[] testIds = value.split(","); + List enabledTestCaseIds = new ArrayList<>(); + for (String id : testIds) { + id = id.trim(); + if (!id.isEmpty()) { + enabledTestCaseIds.add(id); + } + } + config.setEnabledTestCaseIds(enabledTestCaseIds); + } else if (key.equals("disabletestcases")) { + String[] testIds = value.split(","); + List disabledTestCaseIds = new ArrayList<>(); + for (String id : testIds) { + id = id.trim(); + if (!id.isEmpty()) { + disabledTestCaseIds.add(id); + } + } + config.setDisabledTestCaseIds(disabledTestCaseIds); + } + } + } + } +} \ No newline at end of file diff --git a/src/main/java/org/owasp/astf/core/config/ScanConfig.java b/src/main/java/org/owasp/astf/core/config/ScanConfig.java index 6e40819..7e0820a 100644 --- a/src/main/java/org/owasp/astf/core/config/ScanConfig.java +++ b/src/main/java/org/owasp/astf/core/config/ScanConfig.java @@ -8,37 +8,81 @@ import org.owasp.astf.core.EndpointInfo; /** - * Configuration for a security scan. + * Configuration for an API security scan. + *

+ * This class contains all configuration parameters for executing a security scan, + * including target information, authentication settings, test case selection, + * threading options, output formats, and more. + *

*/ public class ScanConfig { + // Target and scope configuration private String targetUrl; - private Map headers; private List endpoints; - private int threads; - private int timeoutMinutes; - private boolean discoveryEnabled; + private boolean discoveryEnabled = true; + private List excludePatterns; + private Map headers; + + // Authentication settings + private String basicAuthUsername; + private String basicAuthPassword; + private String apiKey; + private String apiKeyHeader = "X-API-Key"; + private String bearerToken; + + // Proxy settings + private String proxyHost; + private int proxyPort; + private String proxyUsername; + private String proxyPassword; + + // Test case configuration private List enabledTestCaseIds; private List disabledTestCaseIds; - private OutputFormat outputFormat; + + // Execution settings + private int threads = 10; + private int timeoutMinutes = 30; + private int requestDelayMs = 0; + private int maxRequestsPerSecond = 0; + private boolean followRedirects = true; + private boolean validateCertificates = true; + + // Output settings + private OutputFormat outputFormat = OutputFormat.JSON; private String outputFile; - private boolean verbose; + private boolean verbose = false; + private int maxFindings = 0; + private List excludeSeverities; + /** + * Creates a new scan configuration with default settings. + */ public ScanConfig() { this.headers = new HashMap<>(); this.endpoints = new ArrayList<>(); - this.threads = 10; - this.timeoutMinutes = 30; - this.discoveryEnabled = true; this.enabledTestCaseIds = new ArrayList<>(); this.disabledTestCaseIds = new ArrayList<>(); - this.outputFormat = OutputFormat.JSON; - this.verbose = false; + this.excludePatterns = new ArrayList<>(); + this.excludeSeverities = new ArrayList<>(); } + // Target and scope getters/setters + + /** + * Gets the target URL for the API scan. + * + * @return The target URL + */ public String getTargetUrl() { return targetUrl; } + /** + * Sets the target URL for the API scan. + * + * @param targetUrl The target URL + */ public void setTargetUrl(String targetUrl) { // Ensure target URL ends with a trailing slash if (targetUrl != null && !targetUrl.endsWith("/")) { @@ -48,98 +92,520 @@ public void setTargetUrl(String targetUrl) { } } + /** + * Gets the list of specific endpoints to scan. + * + * @return The endpoints to scan + */ + public List getEndpoints() { + return endpoints; + } + + /** + * Sets the list of specific endpoints to scan. + * + * @param endpoints The endpoints to scan + */ + public void setEndpoints(List endpoints) { + this.endpoints = endpoints; + } + + /** + * Adds an endpoint to scan. + * + * @param endpoint The endpoint to add + */ + public void addEndpoint(EndpointInfo endpoint) { + this.endpoints.add(endpoint); + } + + /** + * Checks if endpoint discovery is enabled. + * + * @return true if endpoint discovery is enabled + */ + public boolean isDiscoveryEnabled() { + return discoveryEnabled; + } + + /** + * Sets whether endpoint discovery is enabled. + * + * @param discoveryEnabled true to enable endpoint discovery + */ + public void setDiscoveryEnabled(boolean discoveryEnabled) { + this.discoveryEnabled = discoveryEnabled; + } + + /** + * Gets patterns to exclude from scanning. + * + * @return The exclude patterns + */ + public List getExcludePatterns() { + return excludePatterns; + } + + /** + * Sets patterns to exclude from scanning. + * + * @param excludePatterns The exclude patterns + */ + public void setExcludePatterns(List excludePatterns) { + this.excludePatterns = excludePatterns; + } + + /** + * Gets HTTP headers to include in requests. + * + * @return The HTTP headers + */ public Map getHeaders() { return headers; } + /** + * Sets HTTP headers to include in requests. + * + * @param headers The HTTP headers + */ public void setHeaders(Map headers) { this.headers = headers; } + /** + * Adds an HTTP header. + * + * @param name The header name + * @param value The header value + */ public void addHeader(String name, String value) { this.headers.put(name, value); } - public List getEndpoints() { - return endpoints; + // Authentication getters/setters + + /** + * Gets the username for basic authentication. + * + * @return The basic auth username + */ + public String getBasicAuthUsername() { + return basicAuthUsername; } - public void setEndpoints(List endpoints) { - this.endpoints = endpoints; + /** + * Sets the username for basic authentication. + * + * @param basicAuthUsername The basic auth username + */ + public void setBasicAuthUsername(String basicAuthUsername) { + this.basicAuthUsername = basicAuthUsername; } - public void addEndpoint(EndpointInfo endpoint) { - this.endpoints.add(endpoint); + /** + * Gets the password for basic authentication. + * + * @return The basic auth password + */ + public String getBasicAuthPassword() { + return basicAuthPassword; } - public int getThreads() { - return threads; + /** + * Sets the password for basic authentication. + * + * @param basicAuthPassword The basic auth password + */ + public void setBasicAuthPassword(String basicAuthPassword) { + this.basicAuthPassword = basicAuthPassword; } - public void setThreads(int threads) { - this.threads = threads; + /** + * Gets the API key for authentication. + * + * @return The API key + */ + public String getApiKey() { + return apiKey; } - public int getTimeoutMinutes() { - return timeoutMinutes; + /** + * Sets the API key for authentication. + * + * @param apiKey The API key + */ + public void setApiKey(String apiKey) { + this.apiKey = apiKey; } - public void setTimeoutMinutes(int timeoutMinutes) { - this.timeoutMinutes = timeoutMinutes; + /** + * Gets the header name for the API key. + * + * @return The API key header name + */ + public String getApiKeyHeader() { + return apiKeyHeader; } - public boolean isDiscoveryEnabled() { - return discoveryEnabled; + /** + * Sets the header name for the API key. + * + * @param apiKeyHeader The API key header name + */ + public void setApiKeyHeader(String apiKeyHeader) { + this.apiKeyHeader = apiKeyHeader; } - public void setDiscoveryEnabled(boolean discoveryEnabled) { - this.discoveryEnabled = discoveryEnabled; + /** + * Gets the bearer token for authentication. + * + * @return The bearer token + */ + public String getBearerToken() { + return bearerToken; + } + + /** + * Sets the bearer token for authentication. + * + * @param bearerToken The bearer token + */ + public void setBearerToken(String bearerToken) { + this.bearerToken = bearerToken; + + // Automatically add the Authorization header if not present + if (bearerToken != null && !bearerToken.isEmpty() && + !headers.containsKey("Authorization")) { + headers.put("Authorization", "Bearer " + bearerToken); + } + } + + // Proxy getters/setters + + /** + * Gets the proxy host. + * + * @return The proxy host + */ + public String getProxyHost() { + return proxyHost; } + /** + * Sets the proxy host. + * + * @param proxyHost The proxy host + */ + public void setProxyHost(String proxyHost) { + this.proxyHost = proxyHost; + } + + /** + * Gets the proxy port. + * + * @return The proxy port + */ + public int getProxyPort() { + return proxyPort; + } + + /** + * Sets the proxy port. + * + * @param proxyPort The proxy port + */ + public void setProxyPort(int proxyPort) { + this.proxyPort = proxyPort; + } + + /** + * Gets the proxy username for authentication. + * + * @return The proxy username + */ + public String getProxyUsername() { + return proxyUsername; + } + + /** + * Sets the proxy username for authentication. + * + * @param proxyUsername The proxy username + */ + public void setProxyUsername(String proxyUsername) { + this.proxyUsername = proxyUsername; + } + + /** + * Gets the proxy password for authentication. + * + * @return The proxy password + */ + public String getProxyPassword() { + return proxyPassword; + } + + /** + * Sets the proxy password for authentication. + * + * @param proxyPassword The proxy password + */ + public void setProxyPassword(String proxyPassword) { + this.proxyPassword = proxyPassword; + } + + // Test case configuration getters/setters + + /** + * Gets the IDs of test cases to enable. + * + * @return The enabled test case IDs + */ public List getEnabledTestCaseIds() { return enabledTestCaseIds; } + /** + * Sets the IDs of test cases to enable. + * + * @param enabledTestCaseIds The enabled test case IDs + */ public void setEnabledTestCaseIds(List enabledTestCaseIds) { this.enabledTestCaseIds = enabledTestCaseIds; } + /** + * Gets the IDs of test cases to disable. + * + * @return The disabled test case IDs + */ public List getDisabledTestCaseIds() { return disabledTestCaseIds; } + /** + * Sets the IDs of test cases to disable. + * + * @param disabledTestCaseIds The disabled test case IDs + */ public void setDisabledTestCaseIds(List disabledTestCaseIds) { this.disabledTestCaseIds = disabledTestCaseIds; } + // Execution settings getters/setters + + /** + * Gets the number of threads to use for scanning. + * + * @return The thread count + */ + public int getThreads() { + return threads; + } + + /** + * Sets the number of threads to use for scanning. + * + * @param threads The thread count + */ + public void setThreads(int threads) { + this.threads = threads; + } + + /** + * Gets the timeout for the scan in minutes. + * + * @return The timeout in minutes + */ + public int getTimeoutMinutes() { + return timeoutMinutes; + } + + /** + * Sets the timeout for the scan in minutes. + * + * @param timeoutMinutes The timeout in minutes + */ + public void setTimeoutMinutes(int timeoutMinutes) { + this.timeoutMinutes = timeoutMinutes; + } + + /** + * Gets the delay between requests in milliseconds. + * + * @return The request delay in milliseconds + */ + public int getRequestDelayMs() { + return requestDelayMs; + } + + /** + * Sets the delay between requests in milliseconds. + * + * @param requestDelayMs The request delay in milliseconds + */ + public void setRequestDelayMs(int requestDelayMs) { + this.requestDelayMs = requestDelayMs; + } + + /** + * Gets the maximum number of requests per second. + * + * @return The maximum requests per second + */ + public int getMaxRequestsPerSecond() { + return maxRequestsPerSecond; + } + + /** + * Sets the maximum number of requests per second. + * + * @param maxRequestsPerSecond The maximum requests per second + */ + public void setMaxRequestsPerSecond(int maxRequestsPerSecond) { + this.maxRequestsPerSecond = maxRequestsPerSecond; + } + + /** + * Checks if the client should follow redirects. + * + * @return true if redirects should be followed + */ + public boolean isFollowRedirects() { + return followRedirects; + } + + /** + * Sets whether the client should follow redirects. + * + * @param followRedirects true to follow redirects + */ + public void setFollowRedirects(boolean followRedirects) { + this.followRedirects = followRedirects; + } + + /** + * Checks if SSL certificates should be validated. + * + * @return true if certificates should be validated + */ + public boolean isValidateCertificates() { + return validateCertificates; + } + + /** + * Sets whether SSL certificates should be validated. + * + * @param validateCertificates true to validate certificates + */ + public void setValidateCertificates(boolean validateCertificates) { + this.validateCertificates = validateCertificates; + } + + // Output settings getters/setters + + /** + * Gets the output format for the scan results. + * + * @return The output format + */ public OutputFormat getOutputFormat() { return outputFormat; } + /** + * Sets the output format for the scan results. + * + * @param outputFormat The output format + */ public void setOutputFormat(OutputFormat outputFormat) { this.outputFormat = outputFormat; } + /** + * Gets the output file path for the scan results. + * + * @return The output file path + */ public String getOutputFile() { return outputFile; } + /** + * Sets the output file path for the scan results. + * + * @param outputFile The output file path + */ public void setOutputFile(String outputFile) { this.outputFile = outputFile; } + /** + * Checks if verbose output is enabled. + * + * @return true if verbose output is enabled + */ public boolean isVerbose() { return verbose; } + /** + * Sets whether verbose output is enabled. + * + * @param verbose true to enable verbose output + */ public void setVerbose(boolean verbose) { this.verbose = verbose; } + /** + * Gets the maximum number of findings to include in the results. + * + * @return The maximum number of findings + */ + public int getMaxFindings() { + return maxFindings; + } + + /** + * Sets the maximum number of findings to include in the results. + * + * @param maxFindings The maximum number of findings + */ + public void setMaxFindings(int maxFindings) { + this.maxFindings = maxFindings; + } + + /** + * Gets the severities to exclude from the results. + * + * @return The excluded severities + */ + public List getExcludeSeverities() { + return excludeSeverities; + } + + /** + * Sets the severities to exclude from the results. + * + * @param excludeSeverities The excluded severities + */ + public void setExcludeSeverities(List excludeSeverities) { + this.excludeSeverities = excludeSeverities; + } + + /** + * Enumeration of supported output formats. + */ public enum OutputFormat { + /** JSON output format */ JSON, + /** XML output format */ XML, + /** HTML output format */ HTML, + /** SARIF output format for tool integration */ SARIF } -} +} \ No newline at end of file diff --git a/src/main/java/org/owasp/astf/core/discovery/EndpointDiscoveryService.java b/src/main/java/org/owasp/astf/core/discovery/EndpointDiscoveryService.java new file mode 100644 index 0000000..1becef1 --- /dev/null +++ b/src/main/java/org/owasp/astf/core/discovery/EndpointDiscoveryService.java @@ -0,0 +1,470 @@ +package org.owasp.astf.core.discovery; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.owasp.astf.core.EndpointInfo; +import org.owasp.astf.core.config.ScanConfig; +import org.owasp.astf.core.http.HttpClient; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; + +/** + * Service responsible for discovering API endpoints through various methods. + *

+ * This class implements multiple discovery strategies: + *

    + *
  • OpenAPI/Swagger specification parsing
    • + *
  • Common endpoint pattern testing
    • + *
  • API root path exploration
    • + *
  • Fallback to common endpoints for testing
    • + *
+ *

+ */ +public class EndpointDiscoveryService { + private static final Logger logger = LogManager.getLogger(EndpointDiscoveryService.class); + + private final ScanConfig config; + private final HttpClient httpClient; + private final ObjectMapper objectMapper; + + // Paths where API specifications are commonly found + private static final List SPEC_PATHS = List.of( + "/swagger/v1/swagger.json", + "/swagger.json", + "/api-docs", + "/v2/api-docs", + "/v3/api-docs", + "/openapi.json", + "/swagger/index.html", + "/.well-known/openapi.json", + "/openapi/v3/api-docs", + "/api/swagger.json" + ); + + // Common API root paths to check + private static final List COMMON_API_ROOTS = List.of( + "/api", + "/api/v1", + "/api/v2", + "/rest", + "/service", + "/services", + "/v1", + "/v2" + ); + + // Common resource patterns for APIs + private static final List COMMON_RESOURCES = List.of( + "users", + "accounts", + "customers", + "products", + "orders", + "items", + "transactions", + "auth", + "login", + "register", + "profile", + "settings", + "admin", + "search", + "comments", + "posts", + "articles", + "categories", + "tags" + ); + + // Default HTTP methods to test + private static final List COMMON_METHODS = List.of( + "GET", + "POST", + "PUT", + "DELETE" + ); + + /** + * Creates a new endpoint discovery service. + * + * @param config The scan configuration + * @param httpClient The HTTP client for making requests + */ + public EndpointDiscoveryService(ScanConfig config, HttpClient httpClient) { + this.config = config; + this.httpClient = httpClient; + this.objectMapper = new ObjectMapper(); + } + + /** + * Discovers API endpoints using multiple strategies. + * + * @return A list of discovered endpoints + */ + public List discoverEndpoints() { + logger.info("Starting API endpoint discovery for: {}", config.getTargetUrl()); + Set discoveredEndpoints = new HashSet<>(); + + try { + // Strategy 1: Try to find and parse OpenAPI/Swagger specifications + logger.debug("Attempting to discover OpenAPI/Swagger specifications"); + List specEndpoints = discoverFromSpecifications(); + if (!specEndpoints.isEmpty()) { + logger.info("Found {} endpoints from API specifications", specEndpoints.size()); + discoveredEndpoints.addAll(specEndpoints); + return new ArrayList<>(discoveredEndpoints); + } + + // Strategy 2: Explore common API roots + logger.debug("Exploring common API root paths"); + List rootEndpoints = exploreApiRoots(); + discoveredEndpoints.addAll(rootEndpoints); + + // Strategy 3: Test common resource patterns + logger.debug("Testing common API resource patterns"); + List resourceEndpoints = testCommonResourcePatterns(); + discoveredEndpoints.addAll(resourceEndpoints); + + if (discoveredEndpoints.isEmpty()) { + logger.warn("No endpoints discovered through automatic methods"); + // Fallback strategy: Use common endpoints for testing + logger.info("Using fallback common endpoints for testing"); + discoveredEndpoints.addAll(getFallbackEndpoints()); + } else { + logger.info("Discovered {} unique endpoints", discoveredEndpoints.size()); + } + + } catch (Exception e) { + logger.error("Error during endpoint discovery: {}", e.getMessage()); + logger.debug("Exception details:", e); + // Fallback to common endpoints on error + discoveredEndpoints.addAll(getFallbackEndpoints()); + } + + return new ArrayList<>(discoveredEndpoints); + } + + /** + * Attempts to discover endpoints by finding and parsing OpenAPI/Swagger specifications. + * + * @return List of endpoints discovered from specifications + */ + private List discoverFromSpecifications() { + List endpoints = new ArrayList<>(); + + for (String specPath : SPEC_PATHS) { + try { + String url = config.getTargetUrl() + specPath; + logger.debug("Checking for API specification at: {}", url); + + String response = httpClient.get(url, Map.of()); + + if (response != null && !response.isEmpty()) { + logger.info("Found potential API specification at: {}", url); + + // Check if it's a valid JSON response that might be an OpenAPI spec + if (isValidJson(response) && (response.contains("\"swagger\"") || + response.contains("\"openapi\""))) { + + // Parse the specification and extract endpoints + endpoints.addAll(parseOpenApiSpec(response)); + + if (!endpoints.isEmpty()) { + return endpoints; + } + } else if (response.contains(" parseOpenApiSpec(String specJson) { + List endpoints = new ArrayList<>(); + + try { + JsonNode root = objectMapper.readTree(specJson); + + // Determine if this is Swagger 2.0 or OpenAPI 3.x + boolean isSwagger2 = root.has("swagger") && root.get("swagger").asText().startsWith("2"); + + if (isSwagger2) { + // Parse Swagger 2.0 + JsonNode paths = root.path("paths"); + if (paths.isObject()) { + paths.fields().forEachRemaining(entry -> { + String path = entry.getKey(); + JsonNode pathItem = entry.getValue(); + + pathItem.fields().forEachRemaining(methodEntry -> { + String method = methodEntry.getKey().toUpperCase(); + // Skip non-HTTP method fields + if (Arrays.asList("GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS") + .contains(method)) { + + EndpointInfo endpoint = new EndpointInfo(path, method); + endpoints.add(endpoint); + } + }); + }); + } + } else { + // Parse OpenAPI 3.x + JsonNode paths = root.path("paths"); + if (paths.isObject()) { + paths.fields().forEachRemaining(entry -> { + String path = entry.getKey(); + JsonNode pathItem = entry.getValue(); + + pathItem.fields().forEachRemaining(methodEntry -> { + String method = methodEntry.getKey().toUpperCase(); + // Skip non-HTTP method fields + if (Arrays.asList("GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS") + .contains(method)) { + + boolean requiresAuth = false; + + // Check for security requirements + JsonNode security = methodEntry.getValue().path("security"); + if (!security.isMissingNode() && security.isArray() && security.size() > 0) { + requiresAuth = true; + } + + EndpointInfo endpoint = new EndpointInfo( + path, method, "application/json", null, requiresAuth); + endpoints.add(endpoint); + } + }); + }); + } + } + + logger.info("Extracted {} endpoints from OpenAPI specification", endpoints.size()); + + } catch (Exception e) { + logger.error("Error parsing OpenAPI specification: {}", e.getMessage()); + } + + return endpoints; + } + + /** + * Extracts the specification URL from Swagger UI HTML. + * + * @param html The Swagger UI HTML + * @return The specification URL or null if not found + */ + private String extractSpecUrlFromSwaggerUi(String html) { + // Look for the spec URL in the Swagger UI HTML + Pattern pattern = Pattern.compile("url:\\s*['\"]([^'\"]+)['\"]"); + Matcher matcher = pattern.matcher(html); + + if (matcher.find()) { + return matcher.group(1); + } + + return null; + } + + /** + * Extracts the base URL from a full URL. + * + * @param url The full URL + * @return The base URL + */ + private String extractBaseUrl(String url) { + int pathStart = url.indexOf("/", 8); // Skip "http://" or "https://" + if (pathStart != -1) { + return url.substring(0, pathStart); + } + return url; + } + + /** + * Explores common API root paths to find valid API endpoints. + * + * @return List of endpoints discovered from root paths + */ + private List exploreApiRoots() { + List endpoints = new ArrayList<>(); + + for (String rootPath : COMMON_API_ROOTS) { + try { + String url = config.getTargetUrl() + rootPath; + logger.debug("Testing API root path: {}", url); + + String response = httpClient.get(url, Map.of()); + + if (response != null && !response.isEmpty()) { + logger.info("Found potential API root at: {}", url); + + // If we get a valid response, add the root path + endpoints.add(new EndpointInfo(rootPath, "GET")); + + // If it's JSON, it might be an API that returns available endpoints + if (isValidJson(response)) { + try { + // Try to extract endpoints from the response + JsonNode root = objectMapper.readTree(response); + if (root.isArray() || (root.isObject() && root.size() > 0)) { + // This might be a listing of resources or endpoints + logger.debug("Root path returned JSON data, might contain API information"); + + // Add some child resource paths to test based on common patterns + for (String resource : COMMON_RESOURCES) { + endpoints.add(new EndpointInfo(rootPath + "/" + resource, "GET")); + } + } + } catch (Exception e) { + logger.debug("Error parsing root path response: {}", e.getMessage()); + } + } + } + } catch (Exception e) { + logger.debug("Error testing root path {}: {}", rootPath, e.getMessage()); + // Continue with next path + } + } + + return endpoints; + } + + /** + * Tests common API resource patterns to find valid endpoints. + * + * @return List of endpoints discovered from common patterns + */ + private List testCommonResourcePatterns() { + List endpoints = new ArrayList<>(); + + // Combine API roots with common resources + for (String rootPath : COMMON_API_ROOTS) { + for (String resource : COMMON_RESOURCES) { + String resourcePath = rootPath + "/" + resource; + + try { + String url = config.getTargetUrl() + resourcePath; + logger.debug("Testing resource path: {}", url); + + String response = httpClient.get(url, Map.of()); + + if (response != null && !response.isEmpty() && !response.contains("error") && + !response.contains("not found") && !response.contains("404")) { + logger.info("Found potential resource endpoint: {}", url); + + // Add the base resource endpoint + endpoints.add(new EndpointInfo(resourcePath, "GET")); + + // Add the resource with ID parameter for common methods + for (String method : COMMON_METHODS) { + endpoints.add(new EndpointInfo(resourcePath + "/{id}", method)); + } + + // If we found users, add some common user-related endpoints + if (resource.equals("users") || resource.equals("accounts")) { + endpoints.add(new EndpointInfo(resourcePath + "/me", "GET")); + endpoints.add(new EndpointInfo(resourcePath + "/{id}/profile", "GET")); + } + + // If we found products, add some common product-related endpoints + if (resource.equals("products") || resource.equals("items")) { + endpoints.add(new EndpointInfo(resourcePath + "/search", "GET")); + endpoints.add(new EndpointInfo(resourcePath + "/categories", "GET")); + } + } + } catch (Exception e) { + logger.debug("Error testing resource path {}: {}", resourcePath, e.getMessage()); + // Continue with next path + } + } + } + + return endpoints; + } + + /** + * Gets a list of common fallback endpoints to use when discovery fails. + * + * @return List of common API endpoints + */ + private List getFallbackEndpoints() { + List endpoints = new ArrayList<>(); + + // Default base path + String basePath = "/api/v1"; + + // Add common endpoints for testing + endpoints.add(new EndpointInfo(basePath + "/users", "GET")); + endpoints.add(new EndpointInfo(basePath + "/users/{id}", "GET")); + endpoints.add(new EndpointInfo(basePath + "/users", "POST")); + endpoints.add(new EndpointInfo(basePath + "/users/{id}", "PUT")); + endpoints.add(new EndpointInfo(basePath + "/users/{id}", "DELETE")); + + endpoints.add(new EndpointInfo(basePath + "/auth/login", "POST")); + endpoints.add(new EndpointInfo(basePath + "/auth/logout", "POST")); + + endpoints.add(new EndpointInfo(basePath + "/products", "GET")); + endpoints.add(new EndpointInfo(basePath + "/products/{id}", "GET")); + + endpoints.add(new EndpointInfo(basePath + "/orders", "GET")); + endpoints.add(new EndpointInfo(basePath + "/orders/{id}", "GET")); + endpoints.add(new EndpointInfo(basePath + "/orders", "POST")); + + logger.info("Using {} fallback endpoints for testing", endpoints.size()); + return endpoints; + } + + /** + * Checks if a string is valid JSON. + * + * @param json The string to check + * @return true if the string is valid JSON + */ + private boolean isValidJson(String json) { + try { + objectMapper.readTree(json); + return true; + } catch (IOException e) { + return false; + } + } +} \ No newline at end of file diff --git a/src/main/java/org/owasp/astf/core/http/HttpClient.java b/src/main/java/org/owasp/astf/core/http/HttpClient.java index f52de26..85b5913 100644 --- a/src/main/java/org/owasp/astf/core/http/HttpClient.java +++ b/src/main/java/org/owasp/astf/core/http/HttpClient.java @@ -1,6 +1,12 @@ package org.owasp.astf.core.http; import java.io.IOException; +import java.net.InetSocketAddress; +import java.net.Proxy; +import java.time.Duration; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; import java.util.Map; import java.util.concurrent.TimeUnit; @@ -8,28 +14,70 @@ import org.apache.logging.log4j.Logger; import org.owasp.astf.core.config.ScanConfig; +import okhttp3.Authenticator; +import okhttp3.Call; +import okhttp3.Callback; +import okhttp3.ConnectionPool; +import okhttp3.Cookie; +import okhttp3.CookieJar; +import okhttp3.Credentials; +import okhttp3.HttpUrl; import okhttp3.MediaType; import okhttp3.OkHttpClient; import okhttp3.Request; import okhttp3.RequestBody; import okhttp3.Response; +import okhttp3.ResponseBody; /** * HTTP client wrapper for making API requests. + *

+ * This class provides a robust HTTP client implementation that supports: + *

    + *
  • All common HTTP methods (GET, POST, PUT, DELETE, etc.)
    • + *
  • Various authentication methods
    • + *
  • Cookie handling
    • + *
  • Proxy configuration
    • + *
  • Connection pooling and timeout management
    • + *
  • Response processing with headers
    • + *
+ *

*/ public class HttpClient { private static final Logger logger = LogManager.getLogger(HttpClient.class); private final OkHttpClient client; private final ScanConfig config; + private final Map defaultHeaders; + private final Map> cookieStore = new HashMap<>(); + /** + * Creates a new HTTP client with the specified configuration. + * + * @param config The scan configuration + */ public HttpClient(ScanConfig config) { this.config = config; + this.defaultHeaders = new HashMap<>(config.getHeaders()); OkHttpClient.Builder builder = new OkHttpClient.Builder() - .connectTimeout(30, TimeUnit.SECONDS) - .readTimeout(30, TimeUnit.SECONDS) - .writeTimeout(30, TimeUnit.SECONDS); + .connectTimeout(Duration.ofSeconds(30)) + .readTimeout(Duration.ofSeconds(30)) + .writeTimeout(Duration.ofSeconds(30)) + .connectionPool(new ConnectionPool(20, 5, TimeUnit.MINUTES)) + .cookieJar(new InMemoryCookieJar()) + .followRedirects(true) + .followSslRedirects(true); + + // Configure proxy if specified + if (config.getProxyHost() != null && !config.getProxyHost().isEmpty()) { + configureProxy(builder); + } + + // Configure basic authentication if specified + if (config.getBasicAuthUsername() != null && !config.getBasicAuthUsername().isEmpty()) { + configureBasicAuth(builder); + } this.client = builder.build(); } @@ -43,19 +91,7 @@ public HttpClient(ScanConfig config) { * @throws IOException If the request fails */ public String get(String url, Map headers) throws IOException { - Request.Builder requestBuilder = new Request.Builder() - .url(url) - .get(); - - addHeaders(requestBuilder, headers); - - Request request = requestBuilder.build(); - try (Response response = client.newCall(request).execute()) { - if (response.body() != null) { - return response.body().string(); - } - return ""; - } + return executeRequest(createRequest(url, "GET", headers, null, null)); } /** @@ -71,20 +107,7 @@ public String get(String url, Map headers) throws IOException { public String post(String url, Map headers, String contentType, String body) throws IOException { MediaType mediaType = MediaType.parse(contentType); RequestBody requestBody = RequestBody.create(body, mediaType); - - Request.Builder requestBuilder = new Request.Builder() - .url(url) - .post(requestBody); - - addHeaders(requestBuilder, headers); - - Request request = requestBuilder.build(); - try (Response response = client.newCall(request).execute()) { - if (response.body() != null) { - return response.body().string(); - } - return ""; - } + return executeRequest(createRequest(url, "POST", headers, mediaType, requestBody)); } /** @@ -100,20 +123,7 @@ public String post(String url, Map headers, String contentType, public String put(String url, Map headers, String contentType, String body) throws IOException { MediaType mediaType = MediaType.parse(contentType); RequestBody requestBody = RequestBody.create(body, mediaType); - - Request.Builder requestBuilder = new Request.Builder() - .url(url) - .put(requestBody); - - addHeaders(requestBuilder, headers); - - Request request = requestBuilder.build(); - try (Response response = client.newCall(request).execute()) { - if (response.body() != null) { - return response.body().string(); - } - return ""; - } + return executeRequest(createRequest(url, "PUT", headers, mediaType, requestBody)); } /** @@ -125,30 +135,120 @@ public String put(String url, Map headers, String contentType, S * @throws IOException If the request fails */ public String delete(String url, Map headers) throws IOException { - Request.Builder requestBuilder = new Request.Builder() - .url(url) - .delete(); + return executeRequest(createRequest(url, "DELETE", headers, null, null)); + } - addHeaders(requestBuilder, headers); + /** + * Makes a PATCH request to the specified URL. + * + * @param url The target URL + * @param headers Additional headers to include + * @param contentType The content type of the request + * @param body The request body + * @return The response body as a string + * @throws IOException If the request fails + */ + public String patch(String url, Map headers, String contentType, String body) throws IOException { + MediaType mediaType = MediaType.parse(contentType); + RequestBody requestBody = RequestBody.create(body, mediaType); + return executeRequest(createRequest(url, "PATCH", headers, mediaType, requestBody)); + } - Request request = requestBuilder.build(); - try (Response response = client.newCall(request).execute()) { - if (response.body() != null) { - return response.body().string(); + /** + * Makes a HEAD request to the specified URL. + * + * @param url The target URL + * @param headers Additional headers to include + * @return The response headers + * @throws IOException If the request fails + */ + public Map> head(String url, Map headers) throws IOException { + Response response = client.newCall(createRequest(url, "HEAD", headers, null, null)).execute(); + try { + return extractHeaders(response); + } finally { + response.close(); + } + } + + /** + * Makes an asynchronous request to the specified URL. + * + * @param url The target URL + * @param method The HTTP method + * @param headers Additional headers to include + * @param contentType The content type of the request (null for GET, HEAD, DELETE) + * @param body The request body (null for GET, HEAD, DELETE) + * @param callback The callback to handle the response + */ + public void asyncRequest(String url, String method, Map headers, + String contentType, String body, HttpResponseCallback callback) { + try { + MediaType mediaType = contentType != null ? MediaType.parse(contentType) : null; + RequestBody requestBody = null; + + if (body != null && mediaType != null) { + requestBody = RequestBody.create(body, mediaType); } - return ""; + + Request request = createRequest(url, method, headers, mediaType, requestBody); + + client.newCall(request).enqueue(new Callback() { + @Override + public void onFailure(Call call, IOException e) { + callback.onFailure(e); + } + + @Override + public void onResponse(Call call, Response response) throws IOException { + try (ResponseBody responseBody = response.body()) { + String body = responseBody != null ? responseBody.string() : ""; + Map> headers = extractHeaders(response); + int statusCode = response.code(); + + callback.onSuccess(statusCode, headers, body); + } + } + }); + } catch (Exception e) { + callback.onFailure(e); } } /** - * Adds headers to the request builder. + * Creates an HTTP request with the specified parameters. * - * @param requestBuilder The request builder + * @param url The target URL + * @param method The HTTP method * @param additionalHeaders Additional headers to include + * @param mediaType The media type of the request (null for GET, HEAD, DELETE) + * @param body The request body (null for GET, HEAD, DELETE) + * @return The HTTP request */ - private void addHeaders(Request.Builder requestBuilder, Map additionalHeaders) { + private Request createRequest(String url, String method, Map additionalHeaders, + MediaType mediaType, RequestBody body) { + Request.Builder requestBuilder = new Request.Builder() + .url(url); + + // Set the appropriate method and body + switch (method.toUpperCase()) { + case "GET" -> requestBuilder.get(); + case "HEAD" -> requestBuilder.head(); + case "DELETE" -> requestBuilder.delete(); + case "POST" -> requestBuilder.post(body); + case "PUT" -> requestBuilder.put(body); + case "PATCH" -> requestBuilder.patch(body); + default -> { + if (body != null) { + requestBuilder.method(method, body); + } else { + requestBuilder.method(method, null); + } + } + } + // Add default headers from config - for (Map.Entry entry : config.getHeaders().entrySet()) { + for (Map.Entry entry : defaultHeaders.entrySet()) { requestBuilder.header(entry.getKey(), entry.getValue()); } @@ -158,5 +258,147 @@ private void addHeaders(Request.Builder requestBuilder, Map addi requestBuilder.header(entry.getKey(), entry.getValue()); } } + + return requestBuilder.build(); + } + + /** + * Executes a request and returns the response body as a string. + * + * @param request The HTTP request to execute + * @return The response body as a string + * @throws IOException If the request fails + */ + private String executeRequest(Request request) throws IOException { + try (Response response = client.newCall(request).execute()) { + if (response.body() != null) { + return response.body().string(); + } + return ""; + } + } + + /** + * Extracts headers from a response. + * + * @param response The HTTP response + * @return A map of header names to values + */ + private Map> extractHeaders(Response response) { + Map> headers = new HashMap<>(); + + for (String name : response.headers().names()) { + headers.put(name, response.headers(name)); + } + + return headers; + } + + /** + * Configures proxy settings for the HTTP client. + * + * @param builder The OkHttpClient builder + */ + private void configureProxy(OkHttpClient.Builder builder) { + Proxy proxy = new Proxy( + Proxy.Type.HTTP, + new InetSocketAddress(config.getProxyHost(), config.getProxyPort()) + ); + + builder.proxy(proxy); + + // Configure proxy authentication if needed + if (config.getProxyUsername() != null && !config.getProxyUsername().isEmpty()) { + Authenticator proxyAuthenticator = (route, response) -> { + String credential = Credentials.basic(config.getProxyUsername(), config.getProxyPassword()); + return response.request().newBuilder() + .header("Proxy-Authorization", credential) + .build(); + }; + + builder.proxyAuthenticator(proxyAuthenticator); + } + } + + /** + * Configures basic authentication for the HTTP client. + * + * @param builder The OkHttpClient builder + */ + private void configureBasicAuth(OkHttpClient.Builder builder) { + Authenticator authenticator = (route, response) -> { + String credential = Credentials.basic(config.getBasicAuthUsername(), config.getBasicAuthPassword()); + return response.request().newBuilder() + .header("Authorization", credential) + .build(); + }; + + builder.authenticator(authenticator); + } + + /** + * In-memory cookie jar implementation for cookie management. + */ + private class InMemoryCookieJar implements CookieJar { + @Override + public void saveFromResponse(HttpUrl url, List cookies) { + String domain = url.host(); + + if (!cookieStore.containsKey(domain)) { + cookieStore.put(domain, new ArrayList<>()); + } + + // Replace existing cookies with the same name + List domainCookies = cookieStore.get(domain); + for (Cookie cookie : cookies) { + // Remove existing cookie with same name if present + domainCookies.removeIf(existingCookie -> existingCookie.name().equals(cookie.name())); + + // Add the new cookie + domainCookies.add(cookie); + } + + logger.debug("Cookies for {}: {}", domain, domainCookies.size()); + } + + @Override + public List loadForRequest(HttpUrl url) { + String domain = url.host(); + List validCookies = new ArrayList<>(); + + if (cookieStore.containsKey(domain)) { + List domainCookies = cookieStore.get(domain); + for (Cookie cookie : domainCookies) { + if (cookie.matches(url)) { + validCookies.add(cookie); + } + } + } + + return validCookies; + } + } + + /** + * Callback interface for asynchronous HTTP requests. + */ + public interface HttpResponseCallback { + /** + * Called when the request is successful. + * + * @param statusCode The HTTP status code + * @param headers The response headers + * @param body The response body + */ + void onSuccess(int statusCode, Map> headers, String body); + + /** + * Called when the request fails. + * + * @param e The exception that caused the failure + */ + void onFailure(IOException e); + + void onFailure(Exception e); } } \ No newline at end of file From d3dfbdd601b9260c6cd2f2181d578ba45959b23b Mon Sep 17 00:00:00 2001 From: GovindarajanL Date: Mon, 10 Mar 2025 17:13:05 -0500 Subject: [PATCH 4/4] feat: Add reporting classes This commit introduces the following reporting classes: - Finding: Represents a security finding discovered during a scan. - ScanResult: Contains the complete results of a security scan. - Severity: Represents the severity level of a security finding. --- .../astf/reporting/HtmlReportGenerator.java | 264 ++++++++++++++++++ .../astf/reporting/JsonReportGenerator.java | 105 +++++++ .../owasp/astf/reporting/ReportGenerator.java | 40 +++ .../reporting/ReportGeneratorFactory.java | 68 +++++ .../astf/reporting/SarifReportGenerator.java | 239 ++++++++++++++++ .../astf/reporting/XmlReportGenerator.java | 130 +++++++++ 6 files changed, 846 insertions(+) create mode 100644 src/main/java/org/owasp/astf/reporting/HtmlReportGenerator.java create mode 100644 src/main/java/org/owasp/astf/reporting/JsonReportGenerator.java create mode 100644 src/main/java/org/owasp/astf/reporting/ReportGenerator.java create mode 100644 src/main/java/org/owasp/astf/reporting/ReportGeneratorFactory.java create mode 100644 src/main/java/org/owasp/astf/reporting/SarifReportGenerator.java create mode 100644 src/main/java/org/owasp/astf/reporting/XmlReportGenerator.java diff --git a/src/main/java/org/owasp/astf/reporting/HtmlReportGenerator.java b/src/main/java/org/owasp/astf/reporting/HtmlReportGenerator.java new file mode 100644 index 0000000..25a7aaf --- /dev/null +++ b/src/main/java/org/owasp/astf/reporting/HtmlReportGenerator.java @@ -0,0 +1,264 @@ +package org.owasp.astf.reporting; + +import java.io.FileWriter; +import java.io.IOException; +import java.time.format.DateTimeFormatter; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.owasp.astf.core.result.Finding; +import org.owasp.astf.core.result.ScanResult; +import org.owasp.astf.core.result.Severity; + +/** + * Generates HTML reports from scan results. + *

+ * This report generator creates human-readable HTML files with interactive + * features for exploring scan findings. The HTML report includes: + *

    + *
  • Executive summary with vulnerability statistics
    • + *
  • Interactive finding details
    • + *
  • Evidence and remediation guidance
    • + *
  • Responsive design for various devices
    • + *
+ *

+ */ +public class HtmlReportGenerator implements ReportGenerator { + private static final Logger logger = LogManager.getLogger(HtmlReportGenerator.class); + + /** + * Creates a new HTML report generator with default settings. + */ + public HtmlReportGenerator() { + // Default constructor + } + + @Override + public void generateReport(ScanResult result, String outputPath) throws IOException { + logger.info("Generating HTML report at {}", outputPath); + + StringBuilder html = new StringBuilder(); + + // Generate HTML header and styles + generateHtmlHeader(html); + + // Generate summary section + generateSummarySection(html, result); + + // Generate findings section + generateFindingsSection(html, result); + + // Close HTML tags + html.append("\n"); + + // Write to the output file + try (FileWriter writer = new FileWriter(outputPath)) { + writer.write(html.toString()); + } + + logger.info("HTML report generated successfully with {} findings", result.getTotalFindingsCount()); + } + + /** + * Generates the HTML header and CSS styles. + * + * @param html The StringBuilder to append to + */ + private void generateHtmlHeader(StringBuilder html) { + html.append("\n") + .append("\n") + .append("\n") + .append(" \n") + .append(" \n") + .append(" OWASP API Security Scan Report\n") + .append(" \n") + .append("\n") + .append("\n") + .append("

OWASP API Security Testing Framework - Scan Report

\n"); + } + + /** + * Generates the summary section of the report. + * + * @param html The StringBuilder to append to + * @param result The scan result + */ + private void generateSummarySection(StringBuilder html, ScanResult result) { + Map severitySummary = result.getSeveritySummary(); + + html.append("
\n") + .append("

Scan Summary

\n") + .append("

Target: ").append(result.getTargetUrl()).append("

\n") + .append("

Scan Start: ").append(result.getScanStartTime().format(DateTimeFormatter.ISO_DATE_TIME)).append("

\n") + .append("

Scan End: ").append(result.getScanEndTime().format(DateTimeFormatter.ISO_DATE_TIME)).append("

\n") + .append("

Total Findings: ").append(result.getTotalFindingsCount()).append("

\n") + .append(" \n") + .append(" \n"); + + html.append(" \n") + .append(" \n") + .append(" \n") + .append(" \n") + .append(" \n") + .append("
SeverityCount
Critical").append(severitySummary.getOrDefault(Severity.CRITICAL, 0L)).append("
High").append(severitySummary.getOrDefault(Severity.HIGH, 0L)).append("
Medium").append(severitySummary.getOrDefault(Severity.MEDIUM, 0L)).append("
Low").append(severitySummary.getOrDefault(Severity.LOW, 0L)).append("
Info").append(severitySummary.getOrDefault(Severity.INFO, 0L)).append("
\n") + .append("
\n"); + } + + /** + * Generates the findings section of the report. + * + * @param html The StringBuilder to append to + * @param result The scan result + */ + private void generateFindingsSection(StringBuilder html, ScanResult result) { + if (result.getTotalFindingsCount() == 0) { + html.append("
\n") + .append("

Findings

\n") + .append("

No security findings were detected during the scan.

\n") + .append("
\n"); + return; + } + + html.append("
\n") + .append("

Findings

\n"); + + // Group findings by severity for better organization + Map> findingsBySeverity = result.getFindings().stream() + .collect(Collectors.groupingBy(Finding::getSeverity)); + + // Process findings in order of severity + processFindingsBySeverity(html, findingsBySeverity, Severity.CRITICAL); + processFindingsBySeverity(html, findingsBySeverity, Severity.HIGH); + processFindingsBySeverity(html, findingsBySeverity, Severity.MEDIUM); + processFindingsBySeverity(html, findingsBySeverity, Severity.LOW); + processFindingsBySeverity(html, findingsBySeverity, Severity.INFO); + + html.append("
\n"); + + // Add JavaScript for collapsible sections + html.append("\n"); + } + + /** + * Processes findings for a specific severity level. + * + * @param html The StringBuilder to append to + * @param findingsBySeverity Map of findings grouped by severity + * @param severity The severity level to process + */ + private void processFindingsBySeverity(StringBuilder html, Map> findingsBySeverity, Severity severity) { + List findings = findingsBySeverity.get(severity); + if (findings == null || findings.isEmpty()) { + return; + } + + String severityClass = severity.name().toLowerCase(); + + html.append("

").append(severity).append(" Severity Findings (").append(findings.size()).append(")

\n"); + + for (Finding finding : findings) { + html.append("
\n") + .append("

") + .append(finding.getTitle()) + .append(" ").append(severity).append("

\n") + .append("

Endpoint: ").append(finding.getEndpoint()).append("

\n") + .append("

Test Case: ").append(finding.getTestCaseId()).append("

\n") + .append("
\n") + .append("

").append(finding.getDescription()).append("

\n"); + + // Add evidence if available + if (finding.getEvidence() != null && !finding.getEvidence().isEmpty()) { + html.append("

Evidence: ").append(finding.getEvidence()).append("

\n"); + } + + // Add request/response details if available + if (finding.getRequestDetails() != null || finding.getResponseDetails() != null) { + html.append("
Request/Response Details
\n") + .append("
\n"); + + if (finding.getRequestDetails() != null) { + html.append("

Request:

").append(escapeHtml(finding.getRequestDetails())).append("

\n"); + } + + if (finding.getResponseDetails() != null) { + html.append("

Response:

").append(escapeHtml(finding.getResponseDetails())).append("

\n"); + } + + html.append("
\n"); + } + + // Add remediation guidance + html.append("
Remediation
\n") + .append("
\n") + .append("

").append(finding.getRemediation()).append("

\n") + .append("
\n") + .append("
\n") + .append("
\n"); + } + } + + /** + * Escapes HTML special characters in a string. + * + * @param input The input string + * @return The escaped string + */ + private String escapeHtml(String input) { + if (input == null) { + return ""; + } + + return input.replace("&", "&") + .replace("<", "<") + .replace(">", ">") + .replace("\"", """) + .replace("'", "'"); + } + + @Override + public String getName() { + return "HTML Report Generator"; + } + + @Override + public String getFileExtension() { + return "html"; + } +} \ No newline at end of file diff --git a/src/main/java/org/owasp/astf/reporting/JsonReportGenerator.java b/src/main/java/org/owasp/astf/reporting/JsonReportGenerator.java new file mode 100644 index 0000000..6aae0a1 --- /dev/null +++ b/src/main/java/org/owasp/astf/reporting/JsonReportGenerator.java @@ -0,0 +1,105 @@ +package org.owasp.astf.reporting; + +import java.io.File; +import java.io.IOException; +import java.time.format.DateTimeFormatter; +import java.util.Map; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.owasp.astf.core.result.Finding; +import org.owasp.astf.core.result.ScanResult; +import org.owasp.astf.core.result.Severity; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.SerializationFeature; +import com.fasterxml.jackson.databind.node.ArrayNode; +import com.fasterxml.jackson.databind.node.ObjectNode; + +/** + * Generates JSON reports from scan results. + *

+ * This report generator creates JSON files containing all scan details and findings. + * The JSON format is useful for automated processing, integration with other tools, + * and creating custom visualizations. + *

+ */ +public class JsonReportGenerator implements ReportGenerator { + private static final Logger logger = LogManager.getLogger(JsonReportGenerator.class); + + private final ObjectMapper objectMapper; + + /** + * Creates a new JSON report generator with default settings. + */ + public JsonReportGenerator() { + this.objectMapper = new ObjectMapper(); + this.objectMapper.enable(SerializationFeature.INDENT_OUTPUT); + } + + @Override + public void generateReport(ScanResult result, String outputPath) throws IOException { + logger.info("Generating JSON report at {}", outputPath); + + ObjectNode rootNode = objectMapper.createObjectNode(); + + // Add report metadata + rootNode.put("targetUrl", result.getTargetUrl()); + rootNode.put("scanStartTime", result.getScanStartTime().format(DateTimeFormatter.ISO_DATE_TIME)); + rootNode.put("scanEndTime", result.getScanEndTime().format(DateTimeFormatter.ISO_DATE_TIME)); + + // Add summary information + ObjectNode summaryNode = rootNode.putObject("summary"); + summaryNode.put("totalFindings", result.getTotalFindingsCount()); + + // Add severity breakdown + ObjectNode severityNode = summaryNode.putObject("bySeverity"); + Map severitySummary = result.getSeveritySummary(); + severityNode.put("critical", severitySummary.getOrDefault(Severity.CRITICAL, 0L)); + severityNode.put("high", severitySummary.getOrDefault(Severity.HIGH, 0L)); + severityNode.put("medium", severitySummary.getOrDefault(Severity.MEDIUM, 0L)); + severityNode.put("low", severitySummary.getOrDefault(Severity.LOW, 0L)); + severityNode.put("info", severitySummary.getOrDefault(Severity.INFO, 0L)); + + // Add all findings + ArrayNode findingsNode = rootNode.putArray("findings"); + for (Finding finding : result.getFindings()) { + ObjectNode findingNode = findingsNode.addObject(); + findingNode.put("id", finding.getId()); + findingNode.put("title", finding.getTitle()); + findingNode.put("description", finding.getDescription()); + findingNode.put("severity", finding.getSeverity().name()); + findingNode.put("testCaseId", finding.getTestCaseId()); + findingNode.put("endpoint", finding.getEndpoint()); + + // Add optional fields if present + if (finding.getRequestDetails() != null) { + findingNode.put("requestDetails", finding.getRequestDetails()); + } + + if (finding.getResponseDetails() != null) { + findingNode.put("responseDetails", finding.getResponseDetails()); + } + + findingNode.put("remediation", finding.getRemediation()); + + if (finding.getEvidence() != null) { + findingNode.put("evidence", finding.getEvidence()); + } + } + + // Write to the output file + objectMapper.writeValue(new File(outputPath), rootNode); + logger.info("JSON report generated successfully with {} findings", result.getTotalFindingsCount()); + } + + @Override + public String getName() { + return "JSON Report Generator"; + } + + @Override + public String getFileExtension() { + return "json"; + } +} \ No newline at end of file diff --git a/src/main/java/org/owasp/astf/reporting/ReportGenerator.java b/src/main/java/org/owasp/astf/reporting/ReportGenerator.java new file mode 100644 index 0000000..2caf610 --- /dev/null +++ b/src/main/java/org/owasp/astf/reporting/ReportGenerator.java @@ -0,0 +1,40 @@ +package org.owasp.astf.reporting; + +import java.io.IOException; + +import org.owasp.astf.core.result.ScanResult; + +/** + * Interface for report generators that produce different output formats. + *

+ * The report generator is responsible for transforming scan results into + * a specific output format such as JSON, HTML, XML, or SARIF. Each implementation + * of this interface handles a specific format and provides appropriate formatting + * and styling for that format. + *

+ */ +public interface ReportGenerator { + + /** + * Generates a report from scan results and writes it to the specified path. + * + * @param result The scan result containing findings and metadata + * @param outputPath The file path where the report should be written + * @throws IOException If an error occurs while writing the report + */ + void generateReport(ScanResult result, String outputPath) throws IOException; + + /** + * Gets the name of this report generator. + * + * @return A descriptive name for this report generator + */ + String getName(); + + /** + * Gets the file extension for reports generated by this generator. + * + * @return The file extension (without the dot) for generated reports + */ + String getFileExtension(); +} \ No newline at end of file diff --git a/src/main/java/org/owasp/astf/reporting/ReportGeneratorFactory.java b/src/main/java/org/owasp/astf/reporting/ReportGeneratorFactory.java new file mode 100644 index 0000000..9b5bc16 --- /dev/null +++ b/src/main/java/org/owasp/astf/reporting/ReportGeneratorFactory.java @@ -0,0 +1,68 @@ +package org.owasp.astf.reporting; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.owasp.astf.core.config.ScanConfig; + +/** + * Factory for creating report generators based on the desired output format. + *

+ * This factory creates the appropriate report generator based on the output + * format specified in the scan configuration. It supports all standard output + * formats such as JSON, HTML, XML, and SARIF. + *

+ */ +public class ReportGeneratorFactory { + private static final Logger logger = LogManager.getLogger(ReportGeneratorFactory.class); + + /** + * Private constructor to prevent instantiation. + */ + private ReportGeneratorFactory() { + // Utility class should not be instantiated + } + + /** + * Creates a report generator for the specified output format. + * + * @param format The desired output format + * @return A report generator for the specified format + * @throws IllegalArgumentException If the format is not supported + */ + public static ReportGenerator createGenerator(ScanConfig.OutputFormat format) { + logger.debug("Creating report generator for format: {}", format); + + return switch (format) { + case JSON -> new JsonReportGenerator(); + case HTML -> new HtmlReportGenerator(); + case XML -> new XmlReportGenerator(); + case SARIF -> new SarifReportGenerator(); + default -> throw new IllegalArgumentException("Unsupported output format: " + format); + }; + } + + /** + * Creates a report generator based on the file extension. + * + * @param filePath The output file path + * @return A report generator for the specified file extension + * @throws IllegalArgumentException If the file extension is not supported + */ + public static ReportGenerator createGeneratorFromFilePath(String filePath) { + logger.debug("Creating report generator for file: {}", filePath); + + String lowerPath = filePath.toLowerCase(); + + if (lowerPath.endsWith(".json")) { + return new JsonReportGenerator(); + } else if (lowerPath.endsWith(".html") || lowerPath.endsWith(".htm")) { + return new HtmlReportGenerator(); + } else if (lowerPath.endsWith(".xml")) { + return new XmlReportGenerator(); + } else if (lowerPath.endsWith(".sarif")) { + return new SarifReportGenerator(); + } else { + throw new IllegalArgumentException("Unsupported file extension: " + filePath); + } + } +} \ No newline at end of file diff --git a/src/main/java/org/owasp/astf/reporting/SarifReportGenerator.java b/src/main/java/org/owasp/astf/reporting/SarifReportGenerator.java new file mode 100644 index 0000000..283ee63 --- /dev/null +++ b/src/main/java/org/owasp/astf/reporting/SarifReportGenerator.java @@ -0,0 +1,239 @@ +package org.owasp.astf.reporting; + +import java.io.File; +import java.io.IOException; +import java.util.HashMap; +import java.util.Map; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.owasp.astf.core.result.Finding; +import org.owasp.astf.core.result.ScanResult; +import org.owasp.astf.core.result.Severity; + +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.node.ArrayNode; +import com.fasterxml.jackson.databind.node.ObjectNode; + +/** + * Generates SARIF (Static Analysis Results Interchange Format) reports. + *

+ * This report generator creates reports in the SARIF format, which is a standard + * for static analysis tool results. This format is supported by many development tools + * and platforms, including GitHub Code Scanning and Azure DevOps. + *

+ *

+ * The SARIF format allows for detailed information about findings, including: + *

    + *
  • Rule definitions with metadata
    • + *
  • Result locations with file paths
    • + *
  • Severity levels with standardized mapping
    • + *
  • Fix suggestions with code snippets
    • + *
  • Detailed contextual information
    • + *
+ *

+ * + * @see SARIF Specification + */ +public class SarifReportGenerator implements ReportGenerator { + private static final Logger logger = LogManager.getLogger(SarifReportGenerator.class); + + private final ObjectMapper objectMapper; + + /** + * Creates a new SARIF report generator. + */ + public SarifReportGenerator() { + this.objectMapper = new ObjectMapper(); + } + + @Override + public void generateReport(ScanResult result, String outputPath) throws IOException { + logger.info("Generating SARIF report at {}", outputPath); + + // Create SARIF report structure + ObjectNode sarifNode = objectMapper.createObjectNode(); + sarifNode.put("$schema", "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json"); + sarifNode.put("version", "2.1.0"); + + ArrayNode runsArray = sarifNode.putArray("runs"); + ObjectNode runNode = runsArray.addObject(); + + // Tool information + ObjectNode toolNode = runNode.putObject("tool"); + ObjectNode toolDriverNode = toolNode.putObject("driver"); + toolDriverNode.put("name", "OWASP API Security Testing Framework"); + toolDriverNode.put("informationUri", "https://github.com/OWASP/api-security-testing-framework"); + toolDriverNode.put("semanticVersion", "1.0.0"); + toolDriverNode.put("version", "1.0.0"); + + // Rules + ArrayNode rulesArray = toolDriverNode.putArray("rules"); + + // Create a map of test case IDs to rule nodes + Map ruleMap = new HashMap<>(); + + // Results + ArrayNode resultsArray = runNode.putArray("results"); + + // Generate rules and results + for (Finding finding : result.getFindings()) { + String testCaseId = finding.getTestCaseId(); + + // Create rule if not exists + if (!ruleMap.containsKey(testCaseId)) { + ObjectNode ruleNode = rulesArray.addObject(); + ruleNode.put("id", testCaseId); + + // Short description + ObjectNode shortDescNode = objectMapper.createObjectNode(); + shortDescNode.put("text", finding.getTitle()); + ruleNode.set("shortDescription", shortDescNode); + + // Full description + ObjectNode fullDescNode = objectMapper.createObjectNode(); + fullDescNode.put("text", finding.getDescription()); + ruleNode.set("fullDescription", fullDescNode); + + // Help text (remediation) + ObjectNode helpNode = ruleNode.putObject("help"); + helpNode.put("text", finding.getRemediation()); + + // Properties + ObjectNode propertiesNode = ruleNode.putObject("properties"); + propertiesNode.put("security-severity", getSarifSeverityNumber(finding.getSeverity())); + + // Tags + ArrayNode tagsNode = objectMapper.createArrayNode(); + tagsNode.add("security"); + tagsNode.add("api"); + + // Map OWASP categories + if (testCaseId.contains("API1")) { + tagsNode.add("broken-object-level-authorization"); + propertiesNode.put("category", "broken-object-level-authorization"); + } else if (testCaseId.contains("API2")) { + tagsNode.add("broken-authentication"); + propertiesNode.put("category", "broken-authentication"); + } else if (testCaseId.contains("API3")) { + tagsNode.add("excessive-data-exposure"); + propertiesNode.put("category", "excessive-data-exposure"); + } else if (testCaseId.contains("API4")) { + tagsNode.add("lack-of-resources-and-rate-limiting"); + propertiesNode.put("category", "lack-of-resources-and-rate-limiting"); + } else if (testCaseId.contains("API5")) { + tagsNode.add("broken-function-level-authorization"); + propertiesNode.put("category", "broken-function-level-authorization"); + } + + propertiesNode.set("tags", tagsNode); + + ruleMap.put(testCaseId, ruleNode); + } + + // Create result + ObjectNode resultNode = resultsArray.addObject(); + + // Rule ID reference + resultNode.put("ruleId", finding.getTestCaseId()); + + // Set level based on severity + String level = switch (finding.getSeverity()) { + case CRITICAL, HIGH -> "error"; + case MEDIUM -> "warning"; + case LOW, INFO -> "note"; + }; + resultNode.put("level", level); + + // Message + ObjectNode messageNode = resultNode.putObject("message"); + messageNode.put("text", finding.getTitle() + ": " + finding.getDescription()); + + // Locations + ArrayNode locationsArray = resultNode.putArray("locations"); + ObjectNode locationNode = locationsArray.addObject(); + + ObjectNode physicalLocationNode = locationNode.putObject("physicalLocation"); + ObjectNode artifactLocationNode = physicalLocationNode.putObject("artifactLocation"); + + // Extract endpoint path for URI + String endpointPath = finding.getEndpoint(); + if (endpointPath != null && endpointPath.contains(" ")) { + endpointPath = endpointPath.split("\\s+")[1]; + } + + artifactLocationNode.put("uri", result.getTargetUrl() + (endpointPath != null ? endpointPath : "")); + artifactLocationNode.put("uriBaseId", "%SRCROOT%"); + + // Add evidence if available + if (finding.getEvidence() != null && !finding.getEvidence().isEmpty()) { + ObjectNode regionNode = physicalLocationNode.putObject("region"); + regionNode.put("startLine", 1); + regionNode.put("startColumn", 1); + + // Snippet + ObjectNode snippetNode = objectMapper.createObjectNode(); + snippetNode.put("text", finding.getEvidence()); + regionNode.set("snippet", snippetNode); + } + + // Add request/response as attachments if available + if (finding.getRequestDetails() != null || finding.getResponseDetails() != null) { + ArrayNode attachmentsArray = resultNode.putArray("attachments"); + + if (finding.getRequestDetails() != null) { + ObjectNode attachmentNode = attachmentsArray.addObject(); + attachmentNode.put("description", "HTTP Request"); + + ObjectNode contentNode = objectMapper.createObjectNode(); + contentNode.put("text", finding.getRequestDetails()); + attachmentNode.set("content", contentNode); + } + + if (finding.getResponseDetails() != null) { + ObjectNode attachmentNode = attachmentsArray.addObject(); + attachmentNode.put("description", "HTTP Response"); + + ObjectNode contentNode = objectMapper.createObjectNode(); + contentNode.put("text", finding.getResponseDetails()); + attachmentNode.set("content", contentNode); + } + } + } + + // Write to file + objectMapper.writerWithDefaultPrettyPrinter().writeValue(new File(outputPath), sarifNode); + + logger.info("SARIF report generated successfully with {} findings", result.getTotalFindingsCount()); + } + + /** + * Converts an ASTF severity level to a SARIF security-severity number. + *

+ * SARIF uses numbers between 0.0 and 10.0 for security severity, + * similar to CVSS scores. + *

+ * + * @param severity The ASTF severity level + * @return The SARIF security-severity number + */ + private double getSarifSeverityNumber(Severity severity) { + return switch (severity) { + case CRITICAL -> 9.8; // Critical: Similar to CVSS 9.8 + case HIGH -> 8.0; // High: Similar to CVSS 8.0 + case MEDIUM -> 5.0; // Medium: Similar to CVSS 5.0 + case LOW -> 3.0; // Low: Similar to CVSS 3.0 + case INFO -> 0.0; // Info: Informational only + }; + } + + @Override + public String getName() { + return "SARIF Report Generator"; + } + + @Override + public String getFileExtension() { + return "sarif"; + } +} \ No newline at end of file diff --git a/src/main/java/org/owasp/astf/reporting/XmlReportGenerator.java b/src/main/java/org/owasp/astf/reporting/XmlReportGenerator.java new file mode 100644 index 0000000..154b45e --- /dev/null +++ b/src/main/java/org/owasp/astf/reporting/XmlReportGenerator.java @@ -0,0 +1,130 @@ +package org.owasp.astf.reporting; + +import java.io.FileWriter; +import java.io.IOException; +import java.time.format.DateTimeFormatter; +import java.util.Map; + +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.owasp.astf.core.result.Finding; +import org.owasp.astf.core.result.ScanResult; +import org.owasp.astf.core.result.Severity; + +/** + * Generates XML reports from scan results. + *

+ * This report generator creates XML files with scan findings and metadata. + * The XML format is useful for integration with other tools and systems + * that can parse XML data. + *

+ */ +public class XmlReportGenerator implements ReportGenerator { + private static final Logger logger = LogManager.getLogger(XmlReportGenerator.class); + + /** + * Creates a new XML report generator. + */ + public XmlReportGenerator() { + // Default constructor + } + + @Override + public void generateReport(ScanResult result, String outputPath) throws IOException { + logger.info("Generating XML report at {}", outputPath); + + StringBuilder xml = new StringBuilder(); + + // XML header + xml.append("\n"); + xml.append("\n"); + + // Scan metadata + xml.append(" ").append(escapeXml(result.getTargetUrl())).append("\n"); + xml.append(" ").append(result.getScanStartTime().format(DateTimeFormatter.ISO_DATE_TIME)).append("\n"); + xml.append(" ").append(result.getScanEndTime().format(DateTimeFormatter.ISO_DATE_TIME)).append("\n"); + + // Summary + xml.append(" \n"); + xml.append(" ").append(result.getTotalFindingsCount()).append("\n"); + + // Severity breakdown + xml.append(" \n"); + Map severitySummary = result.getSeveritySummary(); + xml.append(" ").append(severitySummary.getOrDefault(Severity.CRITICAL, 0L)).append("\n"); + xml.append(" ").append(severitySummary.getOrDefault(Severity.HIGH, 0L)).append("\n"); + xml.append(" ").append(severitySummary.getOrDefault(Severity.MEDIUM, 0L)).append("\n"); + xml.append(" ").append(severitySummary.getOrDefault(Severity.LOW, 0L)).append("\n"); + xml.append(" ").append(severitySummary.getOrDefault(Severity.INFO, 0L)).append("\n"); + xml.append(" \n"); + xml.append(" \n"); + + // Findings + xml.append(" \n"); + + for (Finding finding : result.getFindings()) { + xml.append(" \n"); + xml.append(" ").append(escapeXml(finding.getId())).append("\n"); + xml.append(" ").append(escapeXml(finding.getTitle())).append("\n"); + xml.append(" ").append(escapeXml(finding.getDescription())).append("\n"); + xml.append(" ").append(finding.getSeverity()).append("\n"); + xml.append(" ").append(escapeXml(finding.getTestCaseId())).append("\n"); + xml.append(" ").append(escapeXml(finding.getEndpoint())).append("\n"); + + // Optional fields + if (finding.getRequestDetails() != null) { + xml.append(" ").append(escapeXml(finding.getRequestDetails())).append("\n"); + } + + if (finding.getResponseDetails() != null) { + xml.append(" ").append(escapeXml(finding.getResponseDetails())).append("\n"); + } + + xml.append(" ").append(escapeXml(finding.getRemediation())).append("\n"); + + if (finding.getEvidence() != null) { + xml.append(" ").append(escapeXml(finding.getEvidence())).append("\n"); + } + + xml.append(" \n"); + } + + xml.append(" \n"); + xml.append(""); + + // Write to file + try (FileWriter writer = new FileWriter(outputPath)) { + writer.write(xml.toString()); + } + + logger.info("XML report generated successfully with {} findings", result.getTotalFindingsCount()); + } + + /** + * Escapes XML special characters in a string. + * + * @param input The input string + * @return The escaped string + */ + private String escapeXml(String input) { + if (input == null) { + return ""; + } + + return input.replace("&", "&") + .replace("<", "<") + .replace(">", ">") + .replace("\"", """) + .replace("'", "'"); + } + + @Override + public String getName() { + return "XML Report Generator"; + } + + @Override + public String getFileExtension() { + return "xml"; + } +} \ No newline at end of file