Skip to content

Commit 40a669e

Browse files
josecelanojosecelano
committed
fix: [#14] make smoke testing mandatory and fix API authentication
- Fix Statistics API authentication to use query parameter (?token=) instead of Bearer token - Update nginx proxy test to use /health_check endpoint instead of generic / - Make smoke testing mandatory for E2E test success (fail if any smoke test fails) - Add comprehensive failure reporting with error counts and debugging info - Update smoke testing guide documentation with authentication examples - Update test strategy documentation to reflect mandatory smoke testing The E2E test now validates all critical tracker functionality: - Health Check API (nginx proxy port 80) - Statistics API with proper authentication (nginx proxy port 80) - UDP tracker connectivity (ports 6868, 6969) - HTTP tracker via nginx proxy (/health_check endpoint) - Direct tracker health check (port 1212) All smoke tests must pass for deployment to be considered successful.
1 parent 027486a commit 40a669e

File tree

4 files changed

+921
-2
lines changed

4 files changed

+921
-2
lines changed

‎docs/guides/smoke-testing-guide.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -265,9 +265,9 @@ curl -s http://$TARGET_SERVER:80/api/health_check | jq
265265
The statistics API is available through the nginx proxy on port 80:
266266

267267
```bash
268-
# Test statistics API through nginx proxy
268+
# Test statistics API through nginx proxy (requires admin token)
269269
echo "=== Testing Statistics API ==="
270-
curl -s http://$TARGET_SERVER:80/api/v1/stats | jq
270+
curl -s "http://$TARGET_SERVER:80/api/v1/stats?token=local-dev-admin-token-12345" | jq
271271
```
272272

273273
**Expected Output:**

‎docs/testing/test-strategy.md‎

Lines changed: 260 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,260 @@
1+
# Testing Strategy - Automated Tests
2+
3+
This document describes the automated testing strategy for the Torrust Tracker Demo project.
4+
5+
## Overview
6+
7+
The project follows a layered testing approach that separates concerns and provides different
8+
levels of validation.
9+
10+
## Test Types
11+
12+
### 1. End-to-End Tests (E2E)
13+
14+
**Purpose**: Validate the complete twelve-factor deployment workflow
15+
**Location**: `tests/test-e2e.sh`
16+
**Command**: `make test`
17+
18+
**What it tests**:
19+
20+
- Complete infrastructure provisioning (`make infra-apply`)
21+
- Application deployment (`make app-deploy`)
22+
- Health validation (`make health-check`)
23+
- **Mandatory smoke testing** (tracker functionality validation)
24+
- Cleanup (`make infra-destroy`)
25+
26+
**Follows**: Exactly mirrors `docs/guides/integration-testing-guide.md`
27+
28+
**Duration**: ~5-8 minutes
29+
**Cost**: High (deploys real infrastructure)
30+
**Value**: High (validates entire system)
31+
32+
```bash
33+
# Run E2E test
34+
make test ENVIRONMENT=local
35+
36+
# Run E2E test without cleanup (for debugging)
37+
SKIP_CLEANUP=true make test ENVIRONMENT=local
38+
```
39+
40+
### 2. Unit Tests
41+
42+
**Purpose**: Validate individual components without infrastructure deployment
43+
**Location**: `infrastructure/tests/test-unit-*.sh`
44+
**Command**: `make test-unit`
45+
46+
#### Configuration and Syntax Tests
47+
48+
**Script**: `test-unit-config.sh`
49+
50+
**What it tests**:
51+
52+
- Terraform/OpenTofu configuration validation
53+
- Docker Compose syntax validation
54+
- Makefile syntax validation
55+
- Project structure validation
56+
- Required tools availability
57+
- Configuration template processing
58+
59+
**Note**: YAML and shell script syntax validation is handled by `./scripts/lint.sh`
60+
61+
```bash
62+
# Run all unit tests
63+
make test-unit
64+
65+
# Run only configuration tests
66+
infrastructure/tests/test-unit-config.sh
67+
68+
# Run specific syntax tests
69+
infrastructure/tests/test-unit-config.sh terraform
70+
infrastructure/tests/test-unit-config.sh docker
71+
```
72+
73+
#### Script Unit Tests
74+
75+
**Script**: `test-unit-scripts.sh`
76+
77+
**What it tests**:
78+
79+
- Script existence and permissions
80+
- Script help functionality
81+
- Parameter validation
82+
- Coding standards compliance
83+
- Directory structure
84+
85+
```bash
86+
# Run script unit tests
87+
infrastructure/tests/test-unit-scripts.sh
88+
89+
# Test specific script
90+
infrastructure/tests/test-unit-scripts.sh provision
91+
infrastructure/tests/test-unit-scripts.sh deploy
92+
```
93+
94+
**Duration**: ~1-2 minutes
95+
**Cost**: Low (no infrastructure deployment)
96+
**Value**: Medium (catches syntax and configuration errors early)
97+
98+
### 3. Syntax Validation
99+
100+
**Purpose**: Fast feedback on code quality
101+
**Command**: `make test-syntax` or `make lint`
102+
103+
**What it tests**:
104+
105+
- All file syntax using `scripts/lint.sh`
106+
- YAML, Shell, Markdown validation
107+
- Code quality standards
108+
109+
```bash
110+
# Run syntax validation
111+
make test-syntax
112+
113+
# Or using alias
114+
make lint
115+
```
116+
117+
**Duration**: ~30 seconds
118+
**Cost**: Very low
119+
**Value**: High (prevents broken commits)
120+
121+
### 4. Manual Integration Tests
122+
123+
**Purpose**: Human validation and exploratory testing
124+
**Location**: `docs/guides/integration-testing-guide.md`
125+
126+
**When to use**:
127+
128+
- Testing new features manually
129+
- Validating complex user workflows
130+
- Debugging deployment issues
131+
- Training and documentation
132+
133+
## Test Workflow
134+
135+
### Development Workflow
136+
137+
```bash
138+
# 1. Fast feedback during development
139+
make test-syntax
140+
141+
# 2. Validate changes without deployment
142+
make test-unit
143+
144+
# 3. Full validation before commit/PR
145+
make test ENVIRONMENT=local
146+
```
147+
148+
## Benefits
149+
150+
### 1. Reliability
151+
152+
- E2E tests use the exact same commands as the integration guide
153+
- No duplication of deployment logic
154+
- Tests what users actually do
155+
156+
### 2. Speed
157+
158+
- Unit tests provide fast feedback without infrastructure
159+
- Syntax tests catch errors in seconds
160+
- Developers can test locally without waiting
161+
162+
### 3. Maintainability
163+
164+
- Tests use existing scripts and commands
165+
- Changes to deployment automatically reflected in tests
166+
- Clear separation of concerns
167+
168+
### 4. Cost Efficiency
169+
170+
- Unit tests run without infrastructure costs
171+
- E2E tests only when needed (PRs, releases)
172+
- Syntax tests run on every commit
173+
174+
## Migration from Legacy Tests
175+
176+
### Legacy Test Files (Deprecated)
177+
178+
- `test-integration.sh` - **DEPRECATED**: Use `test-e2e.sh`
179+
- `test-local-setup.sh` - **DEPRECATED**: Use `test-unit-config.sh` + `test-unit-scripts.sh`
180+
181+
### Migration Commands
182+
183+
```bash
184+
# OLD: Complex integration test
185+
infrastructure/tests/test-integration.sh
186+
187+
# NEW: E2E test following integration guide
188+
make test
189+
190+
# OLD: Mixed infrastructure/syntax test
191+
infrastructure/tests/test-local-setup.sh
192+
193+
# NEW: Focused unit tests
194+
make test-unit
195+
```
196+
197+
### Backward Compatibility
198+
199+
Legacy tests are maintained for compatibility but marked as deprecated:
200+
201+
```bash
202+
# Still works but shows deprecation warning
203+
make test-legacy
204+
```
205+
206+
## Troubleshooting
207+
208+
### Common Issues
209+
210+
**E2E test fails with infrastructure errors**:
211+
212+
```bash
213+
# Check prerequisites
214+
make test-syntax
215+
216+
# Check VM status
217+
make infra-status
218+
219+
# Clean up and retry
220+
make infra-destroy && make test
221+
```
222+
223+
**Unit tests fail with tool missing**:
224+
225+
```bash
226+
# Install missing tools
227+
make install-deps
228+
229+
# Check tool availability
230+
infrastructure/tests/test-unit-config.sh tools
231+
```
232+
233+
**Syntax tests fail**:
234+
235+
```bash
236+
# Run specific linting
237+
./scripts/lint.sh --yaml
238+
./scripts/lint.sh --shell
239+
./scripts/lint.sh --markdown
240+
```
241+
242+
### Test Logs
243+
244+
All tests generate detailed logs:
245+
246+
- E2E: `/tmp/torrust-e2e-test.log`
247+
- Unit Config: `/tmp/torrust-unit-config-test.log`
248+
- Unit Scripts: `/tmp/torrust-unit-scripts-test.log`
249+
250+
## Contributing
251+
252+
When adding new functionality:
253+
254+
1. **Add unit tests first** - Test configuration and scripts
255+
2. **Update E2E test if needed** - Usually automatic if using make commands
256+
3. **Update documentation** - Keep integration guide current
257+
4. **Test all levels** - Syntax → Unit → E2E
258+
259+
This layered approach ensures fast feedback during development while maintaining
260+
comprehensive validation of the complete system.

‎tests/README.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# End-to-End Tests
2+
3+
This directory contains end-to-end tests that validate the complete Torrust Tracker Demo system.
4+
5+
## Test Structure
6+
7+
### `test-e2e.sh` - Complete Deployment Workflow
8+
9+
**Purpose**: Validates the entire twelve-factor deployment workflow
10+
11+
**What it tests**:
12+
13+
- Infrastructure provisioning (`make infra-apply`)
14+
- Application deployment (`make app-deploy`)
15+
- Health validation (`make health-check`)
16+
- Complete system integration
17+
18+
**Command**: `make test`
19+
20+
**Duration**: ~5-8 minutes
21+
22+
**Environment**: Deploys real VMs and services
23+
24+
## Usage
25+
26+
```bash
27+
# Run complete E2E test
28+
make test
29+
30+
# Run E2E test for specific environment
31+
make test ENVIRONMENT=local
32+
33+
# Run E2E test without cleanup (for debugging)
34+
SKIP_CLEANUP=true make test
35+
```
36+
37+
## Test Flow
38+
39+
1. **Prerequisites Validation** - Validates system requirements
40+
2. **Infrastructure Provisioning** - Deploys VM using `make infra-apply`
41+
3. **Application Deployment** - Deploys tracker using `make app-deploy`
42+
4. **Health Validation** - Validates all services using `make health-check`
43+
5. **Cleanup** - Destroys infrastructure using `make infra-destroy`
44+
45+
## Output
46+
47+
The test generates a detailed log file at `/tmp/torrust-e2e-test.log` with:
48+
49+
- Timing information for each step
50+
- Success/failure status
51+
- Detailed error messages if failures occur
52+
53+
## Integration with Manual Testing
54+
55+
This test exactly mirrors the manual integration testing guide at:
56+
`docs/guides/integration-testing-guide.md`
57+
58+
The E2E test automates the same workflow that developers follow manually, ensuring
59+
consistency between automated and manual testing procedures.
60+
61+
## Related Tests
62+
63+
- **Unit Tests**: `infrastructure/tests/test-unit-*.sh` - Component-level validation
64+
- **Syntax Tests**: `make test-syntax` - Fast validation without deployment
65+
- **Prerequisites**: `make test-prereq` - System requirements validation

0 commit comments

Comments
 (0)