Closed
Description
Problem
Currently, bun run test
does not work consistently across all packages in the ElizaOS monorepo. This creates several issues:
- Inconsistent Developer Experience: Developers cannot reliably run tests in individual packages
- CI/CD Fragility: The root
bun test
command fails, making it difficult to validate changes - Low Test Coverage: Only 28% of source files have tests (excluding dist files)
- Missing Test Infrastructure: 29% of packages have no test scripts defined
Current State Analysis
Test Coverage by Package:
- ✅ 10/14 packages (71%) have test scripts defined
- ❌ 4/14 packages (29%) have no test infrastructure
- 🔴 Root
bun test
fails due to@elizaos/plugin-bootstrap
mock initialization errors
Packages with Issues:
-
No Tests At All:
@elizaos/app
- Tauri application@elizaos/autodoc
- Documentation generatorcreate-eliza
- Scaffolding tool@elizaos/docs
- Docusaurus site (expected)
-
Failing Tests:
@elizaos/plugin-bootstrap
- Mock initialization errors@elizaos/project-tee-starter
- Environment setup issues
-
Excluded from Root Tests:
@elizaos/plugin-starter
(template)@elizaos/docs
(documentation)@elizaos/plugin-sql
(has tests but excluded)
Proposed Solution
Implement a phased approach to ensure all packages have working tests:
Phase 1: Fix Failing Tests (Priority: High)
- Fix
@elizaos/plugin-bootstrap
mock initialization errors - Fix
@elizaos/project-tee-starter
environment setup issues - Ensure root
bun test
command passes
Phase 2: Add Missing Test Infrastructure (Priority: High)
- Add test setup to
@elizaos/app
(Tauri app testing) - Add test setup to
@elizaos/autodoc
- Add test setup to
create-eliza
- Create minimal test files to validate setup
Phase 3: Standardize Test Configuration (Priority: Medium)
- Create shared test configuration for consistency
- Standardize coverage reporting (exclude dist/, build/, node_modules/)
- Add coverage thresholds per package
- Ensure all packages use Bun test runner consistently
Phase 4: Documentation & CI Updates (Priority: Medium)
- Update contributing guide with testing requirements
- Add pre-commit hooks for test validation
- Update CI workflows to run package-specific tests
- Create testing best practices documentation
Implementation Details
1. Shared Test Configuration
Create a base test configuration that all packages can extend:
// packages/test-config/base.config.ts
export default {
testMatch: ["**/*.test.ts", "**/*.spec.ts"],
coverage: {
exclude: [
"**/dist/**",
"**/build/**",
"**/node_modules/**",
"**/*.d.ts",
"**/coverage/**"
],
threshold: {
statements: 60,
branches: 60,
functions: 60,
lines: 60
}
}
}
2. Package Test Script Standardization
Ensure every package.json has:
{
"scripts": {
"test": "bun test",
"test:coverage": "bun test --coverage"
}
}
3. Fix Root Test Command
Update root package.json to handle package-specific test requirements:
{
"scripts": {
"test": "turbo run test --filter=\!@elizaos/docs --filter=\!@elizaos/plugin-starter"
}
}
Success Criteria
-
bun run test
works in every package directory - Root
bun test
command passes without errors - All packages have at least minimal test coverage
- Test coverage reporting excludes dist/build artifacts
- CI/CD pipeline runs all tests successfully
- Developer documentation updated with testing guidelines
Benefits
- Improved Developer Experience: Consistent testing commands across all packages
- Better Code Quality: Increased test coverage from 28% to target 60%+
- Reliable CI/CD: All PRs validated with comprehensive test suite
- Easier Onboarding: New contributors can confidently run tests
- Reduced Bugs: Catch issues early with standardized testing
Alternatives Considered
- Using Different Test Runners: Considered Jest/Vitest but Bun test is already established
- Monorepo-level Testing Only: Would miss package-specific issues
- Excluding Packages from Testing: Would leave gaps in coverage
Additional Context
- Current test coverage is ~28% (excluding dist files)
- The monorepo uses Turbo for orchestration
- Bun test runner is the standard across the project
- Some packages have E2E tests (client) that need special handling
This improvement will significantly enhance the development workflow and code quality across the ElizaOS project.