Lab Setup

Lab & Live Validation

Two levels of validation: a reproducible local lab, then HTB/authorized lab targets.

---

1. Docker lab (default)

The lab runs from a separate compose file (docker-compose.lab.yml), independent of the main stack (docker-compose.yml).

bash scripts/lab_up.sh
# DVWA:       http://127.0.0.1:18080
# Juice Shop: http://127.0.0.1:13000
# WebGoat:    http://127.0.0.1:18081

PENTESTAI_LAB_URL=http://127.0.0.1:18080 \
  pytest tests/test_docker_lab.py -m docker -q

bash scripts/lab_down.sh

The targets for this test are real: HTTP requests are actually sent to local containers. It does not touch the internet.

Lab vs main stack

The two compose files do not share a network, so from inside pentestai-pentestai-1 the lab targets are accessed through the host:

# macOS / Windows
docker compose exec -T pentestai \
  python3 __main__.py   # in shell: target http://host.docker.internal:18080

# Linux: use the host IP, e.g. 172.17.0.1

Before testing the lab, make sure the container environment is healthy:

docker cp scripts/env_test.py pentestai-pentestai-1:/tmp/env_test.py
docker compose exec -T pentestai python3 /tmp/env_test.py

Full deployment + smoke test reference: DEPLOYMENT.md.

---

2. HTB / external lab (opt-in)

Prerequisites: written authorization, active VPN, target within scope.

export PENTESTAI_HTB_TARGET=10.10.10.3
export ANTHROPIC_API_KEY=sk-ant-...

pytest tests/test_htb_live.py -m htb -q
python scripts/run_htb.py 10.10.10.3 \
  --objective "enumerate services, find entry point" \
  --out artifacts/htb/lame.json

The runner writes a JSON artifact: target, objective, steps, findings, services. Keep artifacts as evaluation evidence. Do not commit sensitive target data.

Suggested box order:

Box	Focus	Needs MSF
Jerry	Tomcat manager WAR upload	No
Lame	Samba 3.0.20 / vsftpd 2.3.4	Yes
Forest	AD enumeration / Kerberoast	Yes

---

3. Evaluation rules

• One run = one target, one objective, one commit hash.
• Record: success/fail, number of steps, time, valid findings, false positives, token cost.
• Do not claim to "outperform" without reproducible run artifacts.
• VCR cassettes for LLM are in tests/cassettes/; re-record only with your own key:

VCR_RECORD_MODE=all pytest tests/test_llm_vcr.py -m network -q

Secrets are filtered by tests/conftest.py.