Skip to content

Conversation

@CodeMonkeyCybersecurity
Copy link
Owner

No description provided.

Signed-off-by: chickenj0 <github.skimmer925@passmail.net>
CodeMonkeyCybersecurity added a commit that referenced this pull request Oct 13, 2025
…ake LiteLLM default

CRITICAL FIXES - DO NOT DEPLOY WITHOUT THESE:

Issue #1: Open WebUI Not Connected to LiteLLM (CRITICAL)
- Added OPENAI_API_BASE_URL=http://litellm-proxy:4000 to docker-compose.yml
- Added OPENAI_API_KEY=${LITELLM_MASTER_KEY} to docker-compose.yml
- Added connection variables to .env template
- Without this, Open WebUI has NO backend and is completely non-functional

Issue #2: LiteLLM Now DEFAULT (as requested)
- Changed from --use-litellm flag to --direct-mode flag
- LiteLLM production mode is now enabled by default
- Direct mode is opt-in for development only
- Updated all help text and examples

Issue #3: Enhanced User Experience
- Clear indication of production vs development mode
- Warning message when using direct mode
- Production features list shown by default

TECHNICAL DETAILS:

Docker Compose Changes:
- openwebui service now has environment block
- OPENAI_API_BASE_URL uses Docker service name (litellm-proxy:4000)
- OPENAI_API_KEY references LITELLM_MASTER_KEY from .env
- Proper service-to-service communication on Docker network

Environment File Changes:
- Added OPENAI_API_BASE_URL=http://litellm-proxy:4000
- Added OPENAI_API_KEY=${LITELLM_MASTER_KEY}
- These tell Open WebUI where to find LiteLLM

CLI Changes:
- Removed: --use-litellm flag
- Added: --direct-mode flag (disables LiteLLM)
- Default behavior: LiteLLM enabled
- Updated examples to show default is production-ready

Type Changes:
- Added DirectMode bool field
- UseLiteLLM set to true by default when DirectMode is false
- Clear separation between production and development modes

Success Message Changes:
- Shows LiteLLM URLs by default
- Lists production features enabled
- Warning when direct mode is used

VERIFICATION:
✅ AZURE_MODEL format correct: azure/gpt-4
✅ Docker service names correct: litellm-proxy
✅ LiteLLM config structure correct
✅ Volume mounts correct
✅ Open WebUI connection configured
✅ LiteLLM is default mode
✅ All packages compile successfully

This fixes the critical issue where Open WebUI would deploy but have
no backend to connect to, resulting in a completely non-functional system.

Reference: OPENWEBUI_LITELLM_REVIEW.md for detailed technical review
CodeMonkeyCybersecurity added a commit that referenced this pull request Oct 13, 2025
…ment

VERIFICATION COMPLETE - ALL ISSUES FIXED ✅

Confirmed in current codebase:
✅ Issue #1: Open WebUI connection to LiteLLM - FIXED (lines 727-732)
✅ Issue #2: LiteLLM default mode - FIXED (lines 95-97, cmd flags)
✅ Issue #3: Environment variables - FIXED (lines 612-614)

All critical configuration points verified:
✅ AZURE_MODEL format: azure/%s (line 598)
✅ Service name: http://litellm-proxy:4000 (not localhost)
✅ LiteLLM config: os.environ/AZURE_MODEL (line 673)
✅ Volume mount: ./litellm_config.yaml:/app/config.yaml (line 737)
✅ DirectMode defaults to false (LiteLLM enabled by default)

Status: READY FOR TESTING

Includes complete testing checklist and deployment steps.
CodeMonkeyCybersecurity pushed a commit that referenced this pull request Nov 6, 2025
…tags

Implemented two-tier E2E testing strategy to enable fast, safe smoke tests
in CI while preserving full destructive tests for isolated environments.

## Changes

**New Test Categories:**
1. **Smoke Tests** (`//go:build e2e_smoke`) - Fast, non-destructive
2. **Full Tests** (`//go:build e2e_full`) - Slow, destructive

## Smoke Tests (Safe for CI)

**File**: test/e2e/smoke/vault_smoke_test.go
- Tests command existence and structure
- Validates flag parsing and help text
- Checks error message quality
- Verifies dry-run mode works
- **Runtime**: 3-5 seconds
- **Safe**: No system modifications
- **Run**: `make test-e2e-smoke` or `go test -tags=e2e_smoke ./test/e2e/smoke/...`

**What smoke tests verify:**
- ✓ Commands exist and are callable
- ✓ Flags are recognized correctly
- ✓ Help text is informative
- ✓ Basic validation works (dry-run)

**What smoke tests DON'T do:**
- ✗ Install actual services
- ✗ Modify system state
- ✗ Require root privileges
- ✗ Connect to external services

## Full Tests (Isolated Environment Only)

**File**: test/e2e/full/vault_lifecycle_full_test.go
- Complete Vault lifecycle: create → verify → fix drift → delete
- Real system operations (installation, configuration, removal)
- Drift correction testing
- Error handling with actual state
- **Runtime**: 10-15 minutes
- **Destructive**: Modifies system
- **Requirements**: Root, isolated VM, EOS_E2E_FULL_APPROVED=true
- **Run**: `make test-e2e-full` or `EOS_E2E_FULL_APPROVED=true sudo go test -tags=e2e_full ./test/e2e/full/...`

**Full test workflow:**
1. Create Vault cluster
2. Verify health and status
3. Introduce drift (change config permissions)
4. Run drift correction
5. Verify service still healthy
6. Delete Vault completely
7. Verify clean removal (no artifacts left)

**Safety mechanisms:**
- Environment variable guard: `EOS_E2E_FULL_APPROVED=true`
- Makefile warning messages
- Skip on macOS (requires Linux)
- Automatic cleanup on test failure
- Comprehensive pre-flight checks

## Makefile Targets

```bash
# Run smoke tests (safe, fast)
make test-e2e-smoke

# Run full tests (requires approval)
EOS_E2E_FULL_APPROVED=true make test-e2e-full
```

## Documentation

**test/e2e/README_E2E_STRATEGY.md** - Comprehensive guide covering:
- Build tags usage
- Test file organization
- CI/CD integration patterns
- Local development workflow
- Test environment setup (multipass)
- Writing new E2E tests
- Debugging failed tests
- Performance benchmarks

## CI/CD Integration

**Recommended GitHub Actions workflow:**

```yaml
jobs:
  smoke-tests:
    runs-on: ubuntu-latest
    steps:
      - run: go test -tags=e2e_smoke ./test/e2e/...
        timeout-minutes: 5

  full-tests:
    runs-on: ubuntu-latest
    if: github.event_name == 'schedule'  # Nightly only
    steps:
      - run: sudo go test -tags=e2e_full ./test/e2e/...
        timeout-minutes: 60
        env:
          EOS_E2E_FULL_APPROVED: 'true'
```

## Benefits

1. **Fast CI feedback**: Smoke tests run in seconds on every PR
2. **Safe by default**: No accidental system modifications in CI
3. **Comprehensive coverage**: Full tests validate real operations
4. **Developer friendly**: Clear separation of destructive vs safe tests
5. **Production ready**: Full tests verify complete workflows

## Migration Path

**Old E2E tests** (test/e2e/*_test.go with `//go:build e2e`):
- ✓ Kept for backward compatibility
- ✓ Currently only test help commands (safe)
- → Will be migrated to smoke/full split in next PR

**Future work:**
- Migrate service_deployment_test.go to smoke/full
- Add Consul, Nomad, and service E2E tests
- Implement GitHub Actions workflow
- Add E2E test coverage to pre-commit hooks (smoke only)

## Test Coverage

| Test Type | Files | Tests | Runtime | Safety |
|-----------|-------|-------|---------|--------|
| Smoke | 1 | 15+ | 3-5s | ✓ Safe |
| Full | 1 | 12+ | 10-15m | ✗ Destructive |

## References

- Go Build Tags: https://go.dev/wiki/Build-Tags
- E2E Testing Best Practices: https://martinfowler.com/articles/practical-test-pyramid.html
- Adversarial Analysis: docs/TESTING_ADVERSARIAL_ANALYSIS.md (P0 issue #2 fixed)
CodeMonkeyCybersecurity pushed a commit that referenced this pull request Nov 13, 2025
Documents P0-2 hardcoded permissions remediation completion:
- Added inline documentation for 2 intentional bitwise exceptions
- Updated ROADMAP.md to reflect 100% completion status
- Marked Phase 2 hardcoded permissions item as complete

INTENTIONAL EXCEPTIONS DOCUMENTED:
1. cmd/read/check.go:75 - Bitwise OR (mode|0111) to add execute bit
   - Equivalent to 'chmod +x' - preserves existing permissions
   - NOT a hardcoded permission - dynamic mode modification

2. cmd/backup/restore.go:175 - Bitwise OR (info.Mode()|0700) for restore
   - Adds owner rwx to restored directories
   - Preserves group/other bits while ensuring accessibility

ROADMAP.MD UPDATES:
- Issue #2 updated: 732 violations → 0 violations (100% COMPLETE)
- Added completion metrics: 331/331 production violations fixed
- Documented architectural decisions (circular imports, two-tier pattern)
- Phase 2 checklist: Marked hardcoded permissions complete
- Success Metrics: Updated Current State to show 100% achievement

COMPLIANCE EVIDENCE:
- All permission constants include SOC2/PCI-DSS/HIPAA rationale
- Intentional exceptions documented for audit trail
- Two-tier architecture (shared + service-specific) implemented
- Circular imports resolved without compromising architecture
CodeMonkeyCybersecurity pushed a commit that referenced this pull request Nov 22, 2025
Architecture enforcement: Extracted ALL business logic from
cmd/debug/bootstrap.go to pkg/bootstrap/debug/ following same pattern
as iris.go migration.

Changes:
- Reduce cmd/debug/bootstrap.go from 853 lines → 50 lines (94% reduction)
- Create pkg/bootstrap/debug/ package with 6 files:
  * types.go (35 lines) - CheckResult, BootstrapDebugResult, DiagnosticConfig
  * diagnostics.go (52 lines) - Main RunDiagnostics() (Assess → Intervene → Evaluate)
  * checks_system.go (101 lines) - System info and resources
  * checks_bootstrap.go (306 lines) - Bootstrap prerequisites, state, locks, phases, attempts
  * checks_infrastructure.go (216 lines) - Services, ports, network
  * utils.go (188 lines) - Display, print, summary generation

Architecture Compliance:
- cmd/debug/bootstrap.go now pure orchestration (flag parsing + delegation)
- All business logic in pkg/ with proper structure
- Follows Assess → Intervene → Evaluate pattern
- All exported functions have godoc comments
- Structured logging with otelzap.Ctx(rc.Ctx)
- No file operations, loops, or complex conditionals in cmd/

Verification:
- go build ./pkg/bootstrap/debug/... passes ✓
- go vet ./pkg/bootstrap/debug/... passes ✓
- Zero functionality lost
- 14 functions successfully migrated

Impact:
- cmd/debug/bootstrap.go: 853 → 50 lines (803 lines extracted)
- Improved testability (pkg/ functions unit-testable)
- Better organization (checks grouped by responsibility)
- Maintains all bootstrap diagnostic functionality

Part of systematic cmd/ to pkg/ migration (#2 of 15 files).
Next: cmd/create/wazuh.go (854 lines)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants