Skip to content

fix: ensure bun run test works consistently across all packages #5218

Closed
@wtfsayo

Description

@wtfsayo

Problem

Currently, bun run test does not work consistently across all packages in the ElizaOS monorepo. This creates several issues:

  1. Inconsistent Developer Experience: Developers cannot reliably run tests in individual packages
  2. CI/CD Fragility: The root bun test command fails, making it difficult to validate changes
  3. Low Test Coverage: Only 28% of source files have tests (excluding dist files)
  4. Missing Test Infrastructure: 29% of packages have no test scripts defined

Current State Analysis

Test Coverage by Package:

  • 10/14 packages (71%) have test scripts defined
  • 4/14 packages (29%) have no test infrastructure
  • 🔴 Root bun test fails due to @elizaos/plugin-bootstrap mock initialization errors

Packages with Issues:

  1. No Tests At All:

    • @elizaos/app - Tauri application
    • @elizaos/autodoc - Documentation generator
    • create-eliza - Scaffolding tool
    • @elizaos/docs - Docusaurus site (expected)
  2. Failing Tests:

    • @elizaos/plugin-bootstrap - Mock initialization errors
    • @elizaos/project-tee-starter - Environment setup issues
  3. Excluded from Root Tests:

    • @elizaos/plugin-starter (template)
    • @elizaos/docs (documentation)
    • @elizaos/plugin-sql (has tests but excluded)

Proposed Solution

Implement a phased approach to ensure all packages have working tests:

Phase 1: Fix Failing Tests (Priority: High)

  • Fix @elizaos/plugin-bootstrap mock initialization errors
  • Fix @elizaos/project-tee-starter environment setup issues
  • Ensure root bun test command passes

Phase 2: Add Missing Test Infrastructure (Priority: High)

  • Add test setup to @elizaos/app (Tauri app testing)
  • Add test setup to @elizaos/autodoc
  • Add test setup to create-eliza
  • Create minimal test files to validate setup

Phase 3: Standardize Test Configuration (Priority: Medium)

  • Create shared test configuration for consistency
  • Standardize coverage reporting (exclude dist/, build/, node_modules/)
  • Add coverage thresholds per package
  • Ensure all packages use Bun test runner consistently

Phase 4: Documentation & CI Updates (Priority: Medium)

  • Update contributing guide with testing requirements
  • Add pre-commit hooks for test validation
  • Update CI workflows to run package-specific tests
  • Create testing best practices documentation

Implementation Details

1. Shared Test Configuration

Create a base test configuration that all packages can extend:

// packages/test-config/base.config.ts
export default {
  testMatch: ["**/*.test.ts", "**/*.spec.ts"],
  coverage: {
    exclude: [
      "**/dist/**",
      "**/build/**",
      "**/node_modules/**",
      "**/*.d.ts",
      "**/coverage/**"
    ],
    threshold: {
      statements: 60,
      branches: 60,
      functions: 60,
      lines: 60
    }
  }
}

2. Package Test Script Standardization

Ensure every package.json has:

{
  "scripts": {
    "test": "bun test",
    "test:coverage": "bun test --coverage"
  }
}

3. Fix Root Test Command

Update root package.json to handle package-specific test requirements:

{
  "scripts": {
    "test": "turbo run test --filter=\!@elizaos/docs --filter=\!@elizaos/plugin-starter"
  }
}

Success Criteria

  • bun run test works in every package directory
  • Root bun test command passes without errors
  • All packages have at least minimal test coverage
  • Test coverage reporting excludes dist/build artifacts
  • CI/CD pipeline runs all tests successfully
  • Developer documentation updated with testing guidelines

Benefits

  1. Improved Developer Experience: Consistent testing commands across all packages
  2. Better Code Quality: Increased test coverage from 28% to target 60%+
  3. Reliable CI/CD: All PRs validated with comprehensive test suite
  4. Easier Onboarding: New contributors can confidently run tests
  5. Reduced Bugs: Catch issues early with standardized testing

Alternatives Considered

  1. Using Different Test Runners: Considered Jest/Vitest but Bun test is already established
  2. Monorepo-level Testing Only: Would miss package-specific issues
  3. Excluding Packages from Testing: Would leave gaps in coverage

Additional Context

  • Current test coverage is ~28% (excluding dist files)
  • The monorepo uses Turbo for orchestration
  • Bun test runner is the standard across the project
  • Some packages have E2E tests (client) that need special handling

This improvement will significantly enhance the development workflow and code quality across the ElizaOS project.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions