Add observability and metrics collection

[vlavrynovych]

Description

Add built-in metrics collection and observability features for monitoring migration performance and health in production.

Requirements

Metrics to Collect

• Execution Time - Per migration and total
• Success/Failure Rate - Track reliability
• Database Size Changes - Before/after migration
• Rollback Frequency - How often rollbacks occur
• Migration Queue Depth - How many pending
• Validation Errors - Pre-execution issues

Configuration

config.metrics = {
  enabled: true,
  collectors: [
    new PrometheusCollector({ port: 9090 }),
    new DataDogCollector({ apiKey: 'xxx' }),
    new CloudWatchCollector({ region: 'us-east-1' }),
    new CustomCollector() // User-defined
  ]
};

Metrics Interface

interface IMetricsCollector {
  recordMigrationStart(script: IMigrationScript): void;
  recordMigrationComplete(script: IMigrationScript, duration: number): void;
  recordMigrationError(script: IMigrationScript, error: Error): void;
  recordRollback(strategy: RollbackStrategy, success: boolean): void;
  recordValidationError(errors: ValidationError[]): void;
}

Built-in Collectors

1. ConsoleMetricsCollector - Log metrics to console
2. FileMetricsCollector - Write metrics to JSON file
3. PrometheusCollector - Expose Prometheus endpoint
4. DataDogCollector - Send to DataDog
5. CloudWatchCollector - Send to AWS CloudWatch

Example Prometheus Metrics

# HELP msr_migrations_total Total migrations executed
# TYPE msr_migrations_total counter
msr_migrations_total{status="success"} 150
msr_migrations_total{status="failed"} 2

# HELP msr_migration_duration_seconds Migration execution time
# TYPE msr_migration_duration_seconds histogram
msr_migration_duration_seconds_bucket{le="0.1"} 50
msr_migration_duration_seconds_bucket{le="1.0"} 120

# HELP msr_rollbacks_total Total rollbacks triggered
# TYPE msr_rollbacks_total counter
msr_rollbacks_total{strategy="BACKUP"} 2

Use Cases

• Production Monitoring - Track migration health
• Performance Analysis - Identify slow migrations
• Alerting - Get notified of failures
• Capacity Planning - Database size trends
• SLA Compliance - Migration time objectives

Implementation Notes

• Create IMetricsCollector interface
• Implement built-in collectors
• Add metrics collection hooks throughout execution
• Support multiple collectors simultaneously
• Make it lightweight (low overhead)

Acceptance Criteria

• IMetricsCollector interface defined
• 3+ built-in collectors implemented
• Metrics collected at key lifecycle points
• Integration examples for popular tools
• Performance impact < 5% overhead
• Documentation with dashboard examples

[vlavrynovych]

Implementation Approach: Build on Existing Hooks Infrastructure

After analyzing the current MSR architecture, we can implement observability features on top of the existing hooks system rather than creating parallel infrastructure. This approach is cleaner, more maintainable, and leverages what we already have.

---

Architecture Overview

Current State (What We Already Have)

1. Hooks System (IMigrationHooks)

• Lifecycle callbacks at key execution points
• onStart, onComplete, onBeforeMigrate, onAfterMigrate, onError, etc.
• User-defined custom logic
• Composable via CompositeHooks

2. ExecutionSummaryLogger

• Records execution details to JSON/TEXT files
• Provides audit trail for compliance and debugging
• Great for CI/CD logs

What's Missing:

• Time-series metrics collection (counters, histograms, gauges)
• Integration with APM tools (Prometheus, DataDog, CloudWatch)
• Real-time metrics endpoints
• Performance percentiles (p50, p95, p99)
• Alerting integration

---

Proposed Solution: MetricsCollectorHook Pattern

Core Idea

Instead of creating a separate metrics system, we create a MetricsCollectorHook that wraps IMetricsCollector implementations and translates hook events into metrics calls.

1. IMetricsCollector Interface

// src/interface/IMetricsCollector.ts

/**
 * Interface for collecting metrics during migration execution.
 * Implementations can send metrics to various backends (Prometheus, DataDog, etc.)
 */
export interface IMetricsCollector {
  /**
   * Record the start of entire migration process
   */
  recordMigrationStart?(context: IMigrationContext): void | Promise<void>;
  
  /**
   * Record the completion of entire migration process
   */
  recordMigrationComplete?(result: IMigrationResult, duration: number): void | Promise<void>;
  
  /**
   * Record the start of individual migration script
   */
  recordScriptStart?(script: IMigrationScript): void | Promise<void>;
  
  /**
   * Record successful completion of individual migration script
   */
  recordScriptComplete?(script: IMigrationScript, duration: number): void | Promise<void>;
  
  /**
   * Record migration script error
   */
  recordScriptError?(script: IMigrationScript, error: Error): void | Promise<void>;
  
  /**
   * Record rollback attempt
   */
  recordRollback?(strategy: RollbackStrategy, success: boolean, duration?: number): void | Promise<void>;
  
  /**
   * Record validation errors
   */
  recordValidationErrors?(errors: ValidationError[]): void | Promise<void>;
  
  /**
   * Record backup operation
   */
  recordBackup?(backupPath: string, duration: number): void | Promise<void>;
  
  /**
   * Record general errors
   */
  recordError?(error: Error): void | Promise<void>;
  
  /**
   * Optional cleanup/flush when execution completes
   */
  close?(): void | Promise<void>;
}

2. MetricsCollectorHook Implementation

// src/hooks/MetricsCollectorHook.ts

/**
 * Hook implementation that delegates to multiple IMetricsCollector instances.
 * Translates hook lifecycle events into metrics collection calls.
 * 
 * Supports multiple collectors simultaneously (e.g., Prometheus + DataDog + Console).
 * Collector failures are logged but don't break migrations.
 */
export class MetricsCollectorHook implements IMigrationHooks {
  constructor(
    private collectors: IMetricsCollector[],
    private logger?: ILogger
  ) {}
  
  async onStart(context: IMigrationContext): Promise<void> {
    await this.callCollectors(
      collector => collector.recordMigrationStart?.(context),
      'recordMigrationStart'
    );
  }
  
  async onComplete(result: IMigrationResult): Promise<void> {
    await this.callCollectors(
      collector => collector.recordMigrationComplete?.(result, result.duration),
      'recordMigrationComplete'
    );
    
    // Cleanup all collectors
    await this.callCollectors(
      collector => collector.close?.(),
      'close'
    );
  }
  
  async onBeforeMigrate(script: IMigrationScript): Promise<void> {
    await this.callCollectors(
      collector => collector.recordScriptStart?.(script),
      'recordScriptStart'
    );
  }
  
  async onAfterMigrate(script: IMigrationScript, duration: number): Promise<void> {
    await this.callCollectors(
      collector => collector.recordScriptComplete?.(script, duration),
      'recordScriptComplete'
    );
  }
  
  async onMigrationError(script: IMigrationScript, error: Error): Promise<void> {
    await this.callCollectors(
      collector => collector.recordScriptError?.(script, error),
      'recordScriptError'
    );
  }
  
  async onError(error: Error): Promise<void> {
    await this.callCollectors(
      collector => collector.recordError?.(error),
      'recordError'
    );
  }
  
  async onAfterBackup(backupPath: string, duration: number): Promise<void> {
    await this.callCollectors(
      collector => collector.recordBackup?.(backupPath, duration),
      'recordBackup'
    );
  }
  
  /**
   * Call all collectors in parallel with error handling.
   * Collector failures are logged but don't break migrations.
   */
  private async callCollectors(
    fn: (collector: IMetricsCollector) => void | Promise<void>,
    methodName: string
  ): Promise<void> {
    const results = await Promise.allSettled(
      this.collectors.map(async (collector) => {
        try {
          await fn(collector);
        } catch (error) {
          this.logger?.warn(
            `Metrics collector ${collector.constructor.name}.${methodName}() failed: ${error.message}`
          );
        }
      })
    );
    
    // Don't throw - metrics collection failures shouldn't break migrations
  }
}

3. Dependency Injection in MigrationScriptExecutor

// src/interface/IMigrationExecutorDependencies.ts

export interface IMigrationExecutorDependencies {
  logger?: ILogger;
  backupService?: IBackupService;
  schemaVersionService?: ISchemaVersionService;
  validationService?: IMigrationValidationService;
  rollbackService?: IRollbackService;
  migrationRenderer?: IMigrationRenderer;
  migrationService?: IMigrationService;
  migrationScanner?: IMigrationScanner;
  loaderRegistry?: ILoaderRegistry;
  
  /**
   * Multiple metrics collectors for observability.
   * Examples: PrometheusCollector, DataDogCollector, CloudWatchCollector
   */
  metricsCollectors?: IMetricsCollector[];
  
  /**
   * Custom hooks for user-defined lifecycle logic.
   */
  hooks?: IMigrationHooks;
}

// src/service/MigrationScriptExecutor.ts (constructor)

constructor(
  handler: IDatabaseMigrationHandler,
  config: Config,
  dependencies?: IMigrationExecutorDependencies
) {
  // ... existing initialization ...
  
  // Auto-initialize hooks with metrics collectors
  const hooksList: IMigrationHooks[] = [];
  
  // Add MetricsCollectorHook if collectors provided
  if (dependencies?.metricsCollectors && dependencies.metricsCollectors.length > 0) {
    hooksList.push(
      new MetricsCollectorHook(dependencies.metricsCollectors, this.logger)
    );
  }
  
  // Add user-provided hooks
  if (dependencies?.hooks) {
    hooksList.push(dependencies.hooks);
  }
  
  // Combine all hooks or use empty hooks
  this.hooks = hooksList.length > 0 
    ? new CompositeHooks(hooksList) 
    : new EmptyHooks();
}

---

Built-in Collectors

1. ConsoleMetricsCollector (For Development/Debugging)

// src/metrics/collectors/ConsoleMetricsCollector.ts

export class ConsoleMetricsCollector implements IMetricsCollector {
  recordMigrationStart(context: IMigrationContext): void {
    console.log(`[METRICS] Migration started - ${context.pending.length} pending scripts`);
  }
  
  recordScriptComplete(script: IMigrationScript, duration: number): void {
    console.log(`[METRICS] ${script.name} completed in ${duration}ms`);
  }
  
  recordScriptError(script: IMigrationScript, error: Error): void {
    console.error(`[METRICS] ${script.name} failed: ${error.message}`);
  }
  
  recordMigrationComplete(result: IMigrationResult, duration: number): void {
    console.log(`[METRICS] Migration completed - ${result.executed.length} scripts in ${duration}ms`);
  }
}

2. FileMetricsCollector (JSON/CSV Format)

// src/metrics/collectors/FileMetricsCollector.ts

export class FileMetricsCollector implements IMetricsCollector {
  private metrics: any[] = [];
  
  constructor(private config: { filePath: string; format: 'json' | 'csv' }) {}
  
  recordScriptComplete(script: IMigrationScript, duration: number): void {
    this.metrics.push({
      timestamp: new Date().toISOString(),
      script: script.name,
      duration,
      status: 'success'
    });
  }
  
  async close(): Promise<void> {
    // Write metrics to file
    if (this.config.format === 'json') {
      await fs.writeFile(this.config.filePath, JSON.stringify(this.metrics, null, 2));
    } else {
      // CSV format
      const csv = this.metrics.map(m => 
        `${m.timestamp},${m.script},${m.duration},${m.status}`
      ).join('\n');
      await fs.writeFile(this.config.filePath, csv);
    }
  }
}

3. PrometheusCollector (For Grafana Dashboards)

// src/metrics/collectors/PrometheusCollector.ts
import { register, Counter, Histogram, Gauge } from 'prom-client';

export interface PrometheusCollectorConfig {
  port: number;
  prefix?: string;  // default: 'msr_'
}

export class PrometheusCollector implements IMetricsCollector {
  private migrationsTotal: Counter;
  private migrationDuration: Histogram;
  private pendingMigrations: Gauge;
  private server: any;
  
  constructor(private config: PrometheusCollectorConfig) {
    const prefix = config.prefix || 'msr_';
    
    // Counter: Total migrations executed
    this.migrationsTotal = new Counter({
      name: `${prefix}migrations_total`,
      help: 'Total number of migrations executed',
      labelNames: ['status', 'script_name']
    });
    
    // Histogram: Migration duration
    this.migrationDuration = new Histogram({
      name: `${prefix}migration_duration_seconds`,
      help: 'Migration execution duration in seconds',
      labelNames: ['script_name'],
      buckets: [0.1, 0.5, 1, 2, 5, 10, 30, 60]
    });
    
    // Gauge: Pending migrations
    this.pendingMigrations = new Gauge({
      name: `${prefix}pending_migrations`,
      help: 'Number of pending migrations'
    });
    
    // Start HTTP server for /metrics endpoint
    this.startServer();
  }
  
  recordMigrationStart(context: IMigrationContext): void {
    this.pendingMigrations.set(context.pending.length);
  }
  
  recordScriptComplete(script: IMigrationScript, duration: number): void {
    this.migrationsTotal.inc({ status: 'success', script_name: script.name });
    this.migrationDuration.observe({ script_name: script.name }, duration / 1000);
  }
  
  recordScriptError(script: IMigrationScript, error: Error): void {
    this.migrationsTotal.inc({ status: 'failed', script_name: script.name });
  }
  
  private startServer(): void {
    const express = require('express');
    const app = express();
    
    app.get('/metrics', (req, res) => {
      res.set('Content-Type', register.contentType);
      res.end(register.metrics());
    });
    
    this.server = app.listen(this.config.port, () => {
      console.log(`Prometheus metrics available at http://localhost:${this.config.port}/metrics`);
    });
  }
  
  async close(): Promise<void> {
    if (this.server) {
      this.server.close();
    }
  }
}

4. DataDogCollector (For APM Monitoring)

// src/metrics/collectors/DataDogCollector.ts
import StatsD from 'hot-shots';

export interface DataDogCollectorConfig {
  apiKey: string;
  host?: string;
  prefix?: string;
}

export class DataDogCollector implements IMetricsCollector {
  private client: StatsD;
  
  constructor(config: DataDogCollectorConfig) {
    this.client = new StatsD({
      host: config.host || 'localhost',
      port: 8125,
      prefix: config.prefix || 'msr.',
      globalTags: { env: process.env.NODE_ENV || 'production' }
    });
  }
  
  recordScriptComplete(script: IMigrationScript, duration: number): void {
    this.client.increment('migrations.success', 1, { script: script.name });
    this.client.timing('migrations.duration', duration, { script: script.name });
  }
  
  recordScriptError(script: IMigrationScript, error: Error): void {
    this.client.increment('migrations.failed', 1, { 
      script: script.name,
      error: error.constructor.name 
    });
  }
  
  recordRollback(strategy: RollbackStrategy, success: boolean): void {
    this.client.increment('rollbacks.total', 1, { 
      strategy, 
      success: success.toString() 
    });
  }
  
  async close(): Promise<void> {
    this.client.close();
  }
}

5. CloudWatchCollector (For AWS Monitoring)

// src/metrics/collectors/CloudWatchCollector.ts
import { CloudWatchClient, PutMetricDataCommand } from '@aws-sdk/client-cloudwatch';

export interface CloudWatchCollectorConfig {
  region: string;
  namespace?: string;  // default: 'MSR/Migrations'
}

export class CloudWatchCollector implements IMetricsCollector {
  private client: CloudWatchClient;
  private namespace: string;
  
  constructor(config: CloudWatchCollectorConfig) {
    this.client = new CloudWatchClient({ region: config.region });
    this.namespace = config.namespace || 'MSR/Migrations';
  }
  
  async recordScriptComplete(script: IMigrationScript, duration: number): Promise<void> {
    await this.client.send(new PutMetricDataCommand({
      Namespace: this.namespace,
      MetricData: [
        {
          MetricName: 'MigrationDuration',
          Value: duration,
          Unit: 'Milliseconds',
          Dimensions: [{ Name: 'ScriptName', Value: script.name }]
        },
        {
          MetricName: 'MigrationSuccess',
          Value: 1,
          Unit: 'Count'
        }
      ]
    }));
  }
  
  async recordScriptError(script: IMigrationScript, error: Error): Promise<void> {
    await this.client.send(new PutMetricDataCommand({
      Namespace: this.namespace,
      MetricData: [{
        MetricName: 'MigrationFailures',
        Value: 1,
        Unit: 'Count',
        Dimensions: [{ Name: 'ScriptName', Value: script.name }]
      }]
    }));
  }
}

---

Usage Examples

Example 1: Single Collector (Development)

import { MigrationScriptExecutor, ConsoleMetricsCollector } from '@migration-script-runner/core';

const executor = new MigrationScriptExecutor(handler, config, {
  metricsCollectors: [
    new ConsoleMetricsCollector()
  ]
});

await executor.up();

// Output:
// [METRICS] Migration started - 3 pending scripts
// [METRICS] V1_CreateUsers completed in 234ms
// [METRICS] V2_AddEmail completed in 156ms
// [METRICS] V3_AddIndex completed in 892ms
// [METRICS] Migration completed - 3 scripts in 1282ms

Example 2: Multiple Collectors (Production)

const executor = new MigrationScriptExecutor(handler, config, {
  metricsCollectors: [
    new PrometheusCollector({ port: 9090 }),           // For Grafana dashboards
    new DataDogCollector({ apiKey: process.env.DD_API_KEY }), // For APM
    new CloudWatchCollector({ region: 'us-east-1' })  // For AWS monitoring
  ]
});

await executor.up();

// Metrics sent to all three systems simultaneously

Example 3: Collectors + Custom Hooks

const executor = new MigrationScriptExecutor(handler, config, {
  metricsCollectors: [
    new PrometheusCollector({ port: 9090 }),
    new DataDogCollector({ apiKey: process.env.DD_API_KEY })
  ],
  hooks: {
    onComplete: async (result) => {
      // Custom business logic
      await notifySlack(`Migration completed: ${result.executed.length} scripts`);
      await updateDeploymentTracker(result);
    },
    onError: async (error) => {
      await notifyPagerDuty(error);
    }
  }
});

// Both metrics collectors AND custom hooks execute via CompositeHooks

Example 4: File-Based Metrics for CI/CD

const executor = new MigrationScriptExecutor(handler, config, {
  metricsCollectors: [
    new FileMetricsCollector({ 
      filePath: './migration-metrics.json',
      format: 'json'
    })
  ]
});

await executor.up();

// Writes migration-metrics.json:
// [
//   { "timestamp": "2025-01-15T10:30:00Z", "script": "V1_CreateUsers", "duration": 234, "status": "success" },
//   { "timestamp": "2025-01-15T10:30:01Z", "script": "V2_AddEmail", "duration": 156, "status": "success" }
// ]

---

Future Enhancement: Config-Based Auto-Enable

Phase 2: Automatic Collector Initialization

// msr.config.js
export default {
  folder: './migrations',
  tableName: 'schema_version',
  
  // Metrics configuration
  metrics: {
    // If port specified, auto-create PrometheusCollector
    prometheus: {
      port: 9090,
      prefix: 'myapp_'
    },
    
    // If apiKey specified, auto-create DataDogCollector
    datadog: {
      apiKey: process.env.DD_API_KEY,
      host: 'localhost',
      prefix: 'myapp.'
    },
    
    // If enabled, auto-create ConsoleMetricsCollector
    console: {
      enabled: process.env.NODE_ENV === 'development'
    },
    
    // If region specified, auto-create CloudWatchCollector
    cloudwatch: {
      region: process.env.AWS_REGION,
      namespace: 'MyApp/Migrations'
    },
    
    // File-based metrics
    file: {
      enabled: true,
      path: './logs/migration-metrics.json',
      format: 'json'
    }
  }
};

// Implementation in MigrationScriptExecutor or ConfigLoader

private autoEnableMetricsCollectors(config: Config): IMetricsCollector[] {
  const collectors: IMetricsCollector[] = [];
  
  if (config.metrics?.prometheus?.port) {
    collectors.push(new PrometheusCollector(config.metrics.prometheus));
  }
  
  if (config.metrics?.datadog?.apiKey) {
    collectors.push(new DataDogCollector(config.metrics.datadog));
  }
  
  if (config.metrics?.console?.enabled) {
    collectors.push(new ConsoleMetricsCollector());
  }
  
  if (config.metrics?.cloudwatch?.region) {
    collectors.push(new CloudWatchCollector(config.metrics.cloudwatch));
  }
  
  if (config.metrics?.file?.enabled) {
    collectors.push(new FileMetricsCollector(config.metrics.file));
  }
  
  return collectors;
}

// In constructor:
const autoCollectors = this.autoEnableMetricsCollectors(config);
const manualCollectors = dependencies?.metricsCollectors || [];
const allCollectors = [...autoCollectors, ...manualCollectors];

if (allCollectors.length > 0) {
  hooksList.push(new MetricsCollectorHook(allCollectors, this.logger));
}

---

Benefits of This Approach

1. Reuses Existing Infrastructure ✅

• No duplicate event system
• Hooks already capture all lifecycle points
• Single responsibility: hooks = events, collectors = metrics

2. Multiple Collectors Support ✅

• Send metrics to multiple destinations simultaneously
• Example: Prometheus + DataDog + Console + CloudWatch
• Redundancy: if one collector fails, others continue

3. Clean Separation of Concerns ✅

• IMetricsCollector: What metrics to collect
• MetricsCollectorHook: When to collect (hook events)
• MigrationScriptExecutor: Orchestration

4. Composable with User Hooks ✅

• Metrics + custom business logic via CompositeHooks
• No conflicts between metrics and user hooks
• User hooks can still access all events

5. Error Isolation ✅

• Collector failures logged but don't break migrations
• Each collector isolated (one failure doesn't affect others)
• Metrics collection never blocks migration execution

6. Testable ✅

• Easy to mock IMetricsCollector in tests
• Test collectors independently
• Test hook integration separately

7. Extensible ✅

• Users can implement custom collectors
• Follow same interface pattern
• Register alongside built-in collectors

8. Zero Overhead if Unused ✅

• No metrics code runs if no collectors provided
• No performance penalty for non-metrics users

9. Backward Compatible ✅

• Existing code works without changes
• Opt-in feature (no breaking changes)
• Existing hooks continue to work

---

Error Handling Strategy

Critical Design Decision: Metrics collection failures must never break migrations.

private async callCollectors(
  fn: (collector: IMetricsCollector) => void | Promise<void>,
  methodName: string
): Promise<void> {
  await Promise.allSettled(
    this.collectors.map(async (collector) => {
      try {
        await fn(collector);
      } catch (error) {
        // Log warning but don't throw
        this.logger?.warn(
          `Metrics collector ${collector.constructor.name}.${methodName}() failed: ${error.message}`
        );
      }
    })
  );
  
  // Don't throw - continue with migration
}

Rationale:

• Metrics are observability, not business logic
• Network failures to DataDog/CloudWatch shouldn't break migrations
• Log warnings for debugging but continue execution
• Each collector isolated (one failure doesn't affect others)

---

Implementation Phases

Phase 1: Core Infrastructure (v0.7.0 or v0.8.0)

• Define IMetricsCollector interface
• Implement MetricsCollectorHook
• Update IMigrationExecutorDependencies with metricsCollectors?
• Auto-initialize in MigrationScriptExecutor constructor
• Add comprehensive tests
• Update documentation

Phase 2: Built-in Collectors (v0.7.0 or v0.8.0)

• Implement ConsoleMetricsCollector
• Implement FileMetricsCollector (JSON/CSV)
• Implement PrometheusCollector
• Add tests for each collector
• Add usage examples to docs

Phase 3: APM Integrations (v0.8.0 or v0.9.0)

• Implement DataDogCollector
• Implement CloudWatchCollector
• Optional: NewRelicCollector
• Optional: GrafanaCloudCollector
• Add integration guides

Phase 4: Config-Based Auto-Enable (v0.9.0 or v1.0.0)

• Extend Config with metrics? property
• Implement autoEnableMetricsCollectors()
• Environment variable support
• Migration guide from manual to config-based
• Update all documentation

---

Open Questions

1. Collector Dependencies: Should collectors be peer dependencies or optional dependencies?

   • Prometheus client (prom-client)
   • DataDog client (hot-shots)
   • AWS SDK (@aws-sdk/client-cloudwatch)

2. Error Handling: Should we provide a global error handler for all collectors?

3. Performance: Should we batch metrics or send immediately?

4. Configuration: Should metrics config go in main Config or separate MetricsConfig?

5. Naming: Is metricsCollectors clear enough or should we use observability?

---

Testing Strategy

Unit Tests

• Test each collector independently
• Mock external services (Prometheus, DataDog APIs)
• Verify metrics sent with correct values
• Test error handling (network failures)

Integration Tests

• Test MetricsCollectorHook with real hooks
• Test multiple collectors together
• Test CompositeHooks with metrics + custom hooks
• Test auto-initialization in MigrationScriptExecutor

Performance Tests

• Measure overhead of metrics collection
• Ensure < 5% performance impact
• Test with 100+ migration scripts

---

Comparison with Alternatives

Alternative 1: Separate Metrics System

❌ Duplicate event handling
❌ More code to maintain
❌ Potential conflicts with hooks

Alternative 2: Extend Hooks Interface

❌ Forces all hook implementations to handle metrics
❌ Breaks Single Responsibility Principle
❌ Harder to make optional

✅ Chosen Approach: MetricsCollectorHook

✅ Reuses existing infrastructure
✅ Clean separation of concerns
✅ Composable with user hooks
✅ Optional and non-invasive
✅ Easy to extend

---

Documentation Needs

• API Reference: IMetricsCollector, MetricsCollectorHook
• User Guide: How to enable metrics collection
• Integration Guides:
  • Prometheus + Grafana dashboards
  • DataDog APM integration
  • AWS CloudWatch monitoring
  • Custom collector implementation
• Examples: Complete working examples for each collector
• Migration Guide: From manual hooks to metrics collectors

---

Success Criteria

• IMetricsCollector interface fully defined
• MetricsCollectorHook implemented with error handling
• Auto-initialization in MigrationScriptExecutor
• At least 3 built-in collectors (Console, File, Prometheus)
• 100% test coverage maintained
• No breaking changes to existing API
• Performance overhead < 5%
• Documentation complete with examples
• Integration guide for popular APM tools

---

This approach gives us enterprise-grade observability while maintaining MSR's clean architecture and not duplicating the existing hooks infrastructure.

[vlavrynovych]

Phase 3 & 4 Implementation Tracked

Phase 1 & 2 are now complete! ✅

Created separate issues for Phase 3 (APM integrations) and Phase 4 (config-based auto-enable):

Phase 3: APM Integrations

• Implement DatadogCollector as built-in metrics collector #118 - Implement DatadogCollector as built-in metrics collector
• Implement CloudWatchCollector as built-in metrics collector #119 - Implement CloudWatchCollector as built-in metrics collector
• Implement PrometheusCollector as built-in metrics collector #120 - Implement PrometheusCollector as built-in metrics collector

Phase 4: Configuration

• Phase 4: Config-based auto-enable for metrics collectors #121 - Config-based auto-enable for metrics collectors

All new issues are in the Backlog milestone and linked to this issue for tracking.

Note: The collectors are currently documented as custom collector examples with full implementation code, so users can implement them themselves while we work on making them built-in.