Contributing

Contributing Guide

---

Development Setup

git clone <repo>
cd pentestai

# Install dev dependencies
pip install -r requirements-dev.txt

# Run tests in parallel (required on this host; serial runs can stall)
python -m pytest tests/ -n 4 --timeout=60 -q

# With coverage
python -m coverage run -m pytest tests/ -n 4 --timeout=60 -q
coverage report

---

Cassettes & Fixtures

Offline tests use VCR cassettes (tests/cassettes/) and tool output fixtures (tests/fixtures/tool_outputs/). Record new cassettes only with your own key:

VCR_RECORD_MODE=all pytest tests/test_llm_vcr.py -m network -q

Secrets are filtered automatically by tests/conftest.py. Do not commit cassettes that still contain real auth headers.

Test Standards

Required for every PR:

1. Assertions must be meaningful — not assert result is not None

   # ❌ Wrong
   result = store.search("sql injection")
   assert isinstance(result, list)
   
   # ✅ Right
   result = store.search("sql injection")
   assert len(result) >= 1
   assert "sql" in result[0].document.text.lower()
   assert result[0].score > 0.5

2. Test isolation — use tmp_path and unique IDs

   # ❌ Wrong — hardcoded session ID can collide
   a = AuditLogger("session1", "test")
   
   # ✅ Right
   import uuid
   sid = f"test_{uuid.uuid4().hex[:8]}"
   a = AuditLogger(sid, "test")

3. Verify coverage — if a test claims to cover specific lines

   # Verify test hits target line
   coverage run --include=module.py \
     -m pytest tests/test_file.py::TestClass::test_name -q
   coverage report --include=module.py --show-missing
   # Make sure the target line does not appear under "Missing"

4. Read the source before writing a test

   cat agents/killchain.py | head -100
   # Understand exactly what the target line does

---

Test Structure

tests/
├── test_api.py              # REST API endpoints
├── test_core.py             # core/ modules
├── test_shell.py            # CLI shell + make_shell fixture
├── test_functional.py       # functional workflows
├── test_e2e.py              # E2E (embedded server)
├── test_phase*.py           # Coverage per component
├── test_meaningful.py       # High-quality behavior tests
├── test_precise_coverage.py # Precision coverage tests
└── test_gap_fixes.py        # Coverage gap fixes

make_shell Fixture

from tests.test_shell import make_shell

def test_something(tmp_path, monkeypatch):
    shell = make_shell(tmp_path, monkeypatch=monkeypatch)
    shell._do_recon([])
    assert len(shell.session.services) >= 0

---

CI Gates

A PR will be rejected if:

• There is a failing test
• Coverage < 80% (current floor; target 85%)
• Trivial test ratio > 25%
• Spot-check tests do not hit target lines

---

Adding a New LLM Provider

1. Edit core/llm.py, add it to PROVIDER_SPECS
2. Implement _init_client_<provider>()
3. Add the env var in from_env()
4. Test: llm add → choose the new provider
5. Write a test in test_core.py

---

Adding a New Tool

1. Create a class in tools/integrations.py inheriting from BaseTool
2. Implement run(target, **kwargs) -> ToolResult
3. Add a parser in tools/parsers.py
4. Register it in ToolRegistry.__init__()
5. Test with a real tool output sample

class HydraToolTest(BaseTool):
    name = "hydra"
    description = "Credential brute-forcing"

    def run(self, target: str, **kwargs) -> ToolResult:
        cmd = ["hydra", "-L", kwargs.get("userlist", "users.txt"), ...]
        result = subprocess.run(cmd, ...)
        return ToolResult(
            tool="hydra",
            target=target,
            command=" ".join(cmd),
            raw_output=result.stdout,
            success=result.returncode == 0,
        )