Standardize Testing Framework: Migrate from vitest to bun:test

[wtfsayo]

Standardize Testing Framework: Migrate from vitest to bun:test

Problem Statement

The ElizaOS monorepo currently uses a mixed testing setup with both vitest and bun:test, creating inconsistency and potential maintenance overhead. Since the project already uses bun as the package manager and runtime, standardizing on bun:test would provide better integration and consistency across the codebase.

Current State Analysis

vitest Usage (extensive):

• 21+ vitest.config.ts files across packages
• vitest listed as devDependency in root package.json and multiple package-specific package.json files
• Used in: @elizaos/core, @elizaos/cli, @elizaos/plugin-sql, @elizaos/server, @elizaos/client, and others
• Documentation specifically mentions "Component tests use Vitest"
• Test scripts in packages use vitest run, vitest watch, etc.

bun:test Usage (minimal):

• Only 1 bunfig.toml file found in packages/cli
• Limited configuration: just test preload setup
• No evidence of extensive bun:test usage in test files

Proposed Solution

Migrate the entire monorepo to use bun:test exclusively, removing vitest dependencies and configurations while maintaining all existing test functionality.

Benefits of bun:test

1. Consistency: Single testing framework aligned with bun package manager usage
2. Performance: Native bun testing typically faster than vitest
3. Simplicity: Reduced dependency count and configuration complexity
4. Maintenance: One less external dependency to maintain and update
5. Integration: Better integration with bun's built-in development tools

Implementation Plan

Phase 1: Preparation

• Audit all test files to identify vitest-specific APIs that need migration
• Create bun:test configuration templates for different package types
• Update test utilities in @elizaos/plugin-bootstrap/__tests__/test-utils for bun:test compatibility

Phase 2: Core Packages Migration

• Migrate @elizaos/core package
  • Replace vitest.config.ts with appropriate bunfig.toml
  • Update test imports (describe, it, expect, etc.)
  • Update package.json test scripts
  • Verify all 9 test files work with bun:test
• Migrate @elizaos/plugin-bootstrap (contains test utilities used by other packages)

Phase 3: Application Packages Migration

• Migrate @elizaos/cli package
  • Update existing bunfig.toml configuration
  • Replace vitest scripts with bun:test equivalents
  • Update run-all-tests.sh script
  • Ensure compatibility with BATS tests (keep separate)
• Migrate @elizaos/server package
  • Handle both vitest.config.ts and vitest.config.integration.ts
  • Update integration test setup
• Migrate @elizaos/client package

Phase 4: Plugin Packages Migration

• Migrate @elizaos/plugin-sql (complex test suite)
• Migrate remaining plugin packages
• Update plugin templates to use bun:test

Phase 5: Documentation & Templates

• Update /packages/docs/docs/core/testing.md
• Update plugin and project starter templates
• Update CLI template generation to use bun:test configuration

Phase 6: Cleanup

• Remove vitest dependencies from all package.json files
• Remove all vitest.config.ts files
• Update root-level test scripts and turbo.json configuration
• Update CI/CD workflows if needed

Acceptance Criteria

• All existing tests pass with bun:test
• No vitest dependencies remain in any package.json
• No vitest configuration files remain
• Test coverage functionality maintained (if used)
• CI/CD pipeline works with bun:test
• Documentation updated to reflect bun:test usage
• New project templates use bun:test by default
• All test scripts use bun:test commands
• Performance benchmarks show no regression (preferably improvement)

Migration Considerations

Potential Challenges:

1. API Differences: Some vitest-specific APIs may not have direct bun:test equivalents
2. Coverage Reports: Ensure test coverage functionality is maintained
3. IDE Integration: Verify IDE/editor test runners work with bun:test
4. CI Performance: Monitor CI build times during transition
5. Complex Test Configurations: Some packages have sophisticated vitest configurations that need careful translation

Risk Mitigation:

• Migrate packages incrementally to catch issues early
• Maintain comprehensive test suite throughout migration
• Keep detailed documentation of configuration changes
• Test in CI environment before merging changes

Related Files

High-priority migration targets:

• packages/core/vitest.config.ts (9 test files)
• packages/cli/vitest.config.ts (extensive test suite)
• packages/plugin-sql/vitest.config.ts (complex integration tests)
• packages/server/vitest.config.ts & vitest.config.integration.ts
• packages/docs/docs/core/testing.md (documentation)

Configuration files to update:

• Root package.json (remove vitest devDependency)
• Individual package package.json files
• turbo.json (if test pipeline configuration needs updates)

---

This migration will significantly simplify the testing infrastructure while maintaining all existing functionality and improving consistency across the ElizaOS monorepo.