Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 34 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,7 @@ The fuzzing engine orchestrates the testing process and manages test execution.

- `tool_fuzzer.py`: Tests individual tools with various argument combinations
- `protocol_fuzzer.py`: Tests MCP protocol types with various message structures
- `invariants.py`: Implements property-based invariants and checks for fuzz testing
- `executor.py`: Provides asynchronous execution framework with concurrency control and retry mechanisms

**Fuzzing Process:**
Expand All @@ -252,7 +253,8 @@ The fuzzing engine orchestrates the testing process and manages test execution.
2. **Strategy Selection**: Choose appropriate fuzzing strategy (realistic vs aggressive)
3. **Data Generation**: Generate test data using Hypothesis and custom strategies
4. **Execution**: Execute tests with controlled concurrency via AsyncFuzzExecutor
5. **Analysis**: Analyze results and generate reports
5. **Invariant Verification**: Verify responses against property-based invariants
6. **Analysis**: Analyze results and generate reports

### 4. Strategy System

Expand All @@ -263,13 +265,43 @@ The strategy system generates test data using different approaches.
- `realistic/`: Generates valid, realistic data for functionality testing
- `aggressive/`: Generates malicious/malformed data for security testing
- `strategy_manager.py`: Orchestrates strategy selection and execution
- `schema_parser.py`: Parses JSON Schema definitions to generate appropriate test data

**Strategy Types:**

- **Realistic Strategies**: Generate valid Base64, UUIDs, timestamps, semantic versions
- **Aggressive Strategies**: Generate SQL injection, XSS, path traversal, buffer overflow attempts

### 5. Safety System
**Schema Parser:**

The schema parser provides comprehensive support for parsing JSON Schema definitions and generating appropriate test data based on schema specifications. It handles:

- Basic types: string, number, integer, boolean, array, object, null
- String constraints: minLength, maxLength, pattern, format
- Number/Integer constraints: minimum, maximum, exclusiveMinimum, exclusiveMaximum, multipleOf
- Array constraints: minItems, maxItems, uniqueItems
- Object constraints: required properties, minProperties, maxProperties
- Schema combinations: oneOf, anyOf, allOf
- Enums and constants

The module supports both "realistic" and "aggressive" fuzzing strategies, where realistic mode generates valid data conforming to the schema, while aggressive mode intentionally generates edge cases and invalid data to test error handling.

### 5. Invariants System

The invariants system provides property-based testing capabilities to verify response validity, error type correctness, and prevention of unintended crashes or unexpected states during fuzzing.

**Key Components:**

- `check_response_validity`: Ensures responses follow JSON-RPC 2.0 specification
- `check_error_type_correctness`: Verifies error responses have correct structure and codes
- `check_response_schema_conformity`: Validates responses against JSON schema definitions
- `verify_response_invariants`: Orchestrates multiple invariant checks on a single response
- `verify_batch_responses`: Applies invariant checks to batches of responses
- `check_state_consistency`: Ensures server state remains consistent during fuzzing

These invariants serve as runtime assertions that validate the behavior of the server being tested, helping to identify potential issues that might not be caught by simple error checking.

### 6. Safety System

The safety system provides multiple layers of protection against dangerous operations.

Expand Down
70 changes: 70 additions & 0 deletions docs/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,13 +113,15 @@ mcp_fuzzer/
protocol_fuzzer.py # Orchestrates protocol-type fuzzing
tool_fuzzer.py # Orchestrates tool fuzzing
strategy/
schema_parser.py # JSON Schema parser for test data generation
strategy_manager.py # Selects strategies per phase/type
realistic/
tool_strategy.py
protocol_type_strategy.py
aggressive/
tool_strategy.py
protocol_type_strategy.py
invariants.py # Property-based invariants and checks
runtime/
manager.py # Async ProcessManager (start/stop, signals)
watchdog.py # ProcessWatchdog (hang detection)
Expand All @@ -143,6 +145,74 @@ mcp_fuzzer/
client.py # UnifiedMCPFuzzerClient orchestrator
```

## Schema Parser

The schema parser module (`mcp_fuzzer.fuzz_engine.strategy.schema_parser`) provides comprehensive support for parsing JSON Schema definitions and generating appropriate test data based on schema specifications.

### Features

- **Basic Types**: Handles string, number, integer, boolean, array, object, and null types
- **String Constraints**: Supports minLength, maxLength, pattern, and format validations
- **Number/Integer Constraints**: Handles minimum, maximum, exclusiveMinimum, exclusiveMaximum, multipleOf
- **Array Constraints**: Supports minItems, maxItems, uniqueItems
- **Object Constraints**: Handles required properties, minProperties, maxProperties
- **Schema Combinations**: Processes oneOf, anyOf, allOf schema combinations
- **Enums and Constants**: Supports enum values and constants
- **Fuzzing Phases**: Supports both "realistic" (valid) and "aggressive" (edge cases) modes

### Example Usage

```python
from mcp_fuzzer.fuzz_engine.strategy.schema_parser import make_fuzz_strategy_from_jsonschema

# Define a JSON schema
schema = {
"type": "object",
"properties": {
"name": {"type": "string", "minLength": 3, "maxLength": 50},
"age": {"type": "integer", "minimum": 18, "maximum": 120},
"email": {"type": "string", "format": "email"}
},
"required": ["name", "age"]
}

# Generate realistic data
realistic_data = make_fuzz_strategy_from_jsonschema(schema, phase="realistic")

# Generate aggressive data for security testing
aggressive_data = make_fuzz_strategy_from_jsonschema(schema, phase="aggressive")
```

## Invariants System

The invariants module (`mcp_fuzzer.fuzz_engine.invariants`) provides property-based testing capabilities to verify response validity, error type correctness, and prevention of unintended crashes or unexpected states during fuzzing.

### Features

- **Response Validity**: Ensures responses follow JSON-RPC 2.0 specification
- **Error Type Correctness**: Verifies error responses have correct structure and codes
- **Schema Conformity**: Validates responses against JSON schema definitions
- **Batch Verification**: Applies invariant checks to batches of responses
- **State Consistency**: Ensures server state remains consistent during fuzzing

### Example Usage

```python
from mcp_fuzzer.fuzz_engine.invariants import verify_response_invariants, InvariantViolation

# Verify a response against invariants
try:
verify_response_invariants(
response={"jsonrpc": "2.0", "id": 1, "result": "success"},
expected_error_codes=[400, 404, 500],
schema={"type": "object", "properties": {"result": {"type": "string"}}}
)
# Response is valid
except InvariantViolation as e:
# Invariant violation detected
print(f"Violation: {e}")
```

- Strategy: Generates inputs for tools and protocol types in two phases:
- realistic (valid/spec-conformant), aggressive (malformed/attack vectors).
- Fuzzer: Runs strategies, sends envelopes via a transport, and records results.
Expand Down
43 changes: 43 additions & 0 deletions mcp_fuzzer/fuzz_engine/fuzzer/protocol_fuzzer.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@

from ..executor import AsyncFuzzExecutor
from ..strategy import ProtocolStrategies
from ..invariants import (
verify_response_invariants,
InvariantViolation,
verify_batch_responses,
)


class ProtocolFuzzer:
Expand Down Expand Up @@ -212,11 +217,48 @@ async def _fuzz_protocol_type_single_run(
protocol_type, fuzz_data, generate_only
)

# Verify invariants if we have a server response
invariant_violations = []
if server_response is not None and not generate_only:
try:
if isinstance(server_response, list):
try:
# Handle batch responses with timeout to prevent hanging
batch = await asyncio.wait_for(
asyncio.create_task(
verify_batch_responses(server_response)
),
timeout=5.0, # 5 second timeout for batch validation
)
viols = [str(v) for k, v in batch.items() if v is not True]
invariant_violations.extend(viols)
except asyncio.TimeoutError:
invariant_violations.append("Batch validation timed out")
self._logger.warning(
"Batch validation timeout in %s run %s",
protocol_type,
run_index + 1,
)
else:
verify_response_invariants(server_response)
except InvariantViolation as e:
invariant_violations.append(str(e))
self._logger.warning(
"Invariant violation in %s run %s: %s",
protocol_type,
run_index + 1,
e,
)

# Create the result
result = self._create_fuzz_result(
protocol_type, run_index, fuzz_data, server_response, server_error
)

# Add invariant violations to the result
if invariant_violations:
result["invariant_violations"] = invariant_violations

self._logger.debug(f"Fuzzed {protocol_type} run {run_index + 1}")
return result

Expand Down Expand Up @@ -326,6 +368,7 @@ def _create_fuzz_result(
"server_response": server_response,
"server_error": server_error,
"server_rejected_input": server_error is not None,
"invariant_violations": [], # Will be populated if violations occur
}

async def fuzz_protocol_type_both_phases(
Expand Down
Loading