Code: Add application-level health checks and idempotency improvements

[monadic]

Priority: MEDIUM

Description

Current health checks only validate pod status, not actual application health. Scripts also lack full idempotency.

Issues Found

1. Health checks don't test application endpoints
2. Scripts fail if resources already exist instead of continuing
3. Missing shellcheck validation
4. YAML files hardcode namespace instead of using templates

Required Actions

1. Add application-level health endpoint testing:

   • Test actual service endpoints (e.g., /health, /api/status)
   • Verify inter-service connectivity
   • Check database connections

2. Improve script idempotency:

   • Check if resources exist before creating
   • Handle partial failures gracefully
   • Add --force flag for overwrites

3. Add CI/CD validations:

   • Run shellcheck on all bash scripts
   • Run yamllint on all YAML files
   • Add pre-commit hooks

Acceptance Criteria

• Health checks test actual endpoints
• All scripts are fully idempotent
• Shellcheck and yamllint pass with no warnings
• Pre-commit hooks configured

References

• Original findings in CODE-REVIEW.md