Phase 4: Config-based auto-enable for metrics collectors

[vlavrynovych]

Overview

Implement config-based automatic initialization of metrics collectors, allowing users to enable metrics through configuration files instead of manual instantiation.

Current State

Metrics collectors must be manually instantiated and passed to MigrationScriptExecutor:

const executor = new MigrationScriptExecutor({
  handler,
  metricsCollectors: [
    new PrometheusCollector({ port: 9090 }),
    new DataDogCollector({ apiKey: process.env.DD_API_KEY })
  ]
}, config);

Proposed Enhancement

Enable collectors through configuration:

// msr.config.js
export default {
  folder: './migrations',
  tableName: 'schema_version',
  
  metrics: {
    // Auto-create PrometheusCollector
    prometheus: {
      enabled: true,
      port: 9090,
      prefix: 'myapp_'
    },
    
    // Auto-create DataDogCollector
    datadog: {
      enabled: true,
      apiKey: process.env.DD_API_KEY,
      host: 'localhost',
      prefix: 'myapp.'
    },
    
    // Auto-create ConsoleMetricsCollector
    console: {
      enabled: process.env.NODE_ENV === 'development'
    },
    
    // Auto-create CloudWatchCollector
    cloudwatch: {
      enabled: true,
      region: process.env.AWS_REGION,
      namespace: 'MyApp/Migrations'
    },
    
    // Auto-create JsonMetricsCollector
    json: {
      enabled: true,
      filePath: './metrics/migration.json',
      pretty: true
    },
    
    // Auto-create CsvMetricsCollector
    csv: {
      enabled: true,
      filePath: './metrics/history.csv'
    }
  }
};

Implementation Plan

1. Extend Config Interface

// src/config/Config.ts
export interface MetricsConfig {
  prometheus?: {
    enabled?: boolean;
    port: number;
    prefix?: string;
  };
  datadog?: {
    enabled?: boolean;
    apiKey: string;
    host?: string;
    prefix?: string;
  };
  console?: {
    enabled?: boolean;
  };
  cloudwatch?: {
    enabled?: boolean;
    region: string;
    namespace?: string;
    dimensions?: Array<{ Name: string; Value: string }>;
  };
  json?: {
    enabled?: boolean;
    filePath: string;
    pretty?: boolean;
  };
  csv?: {
    enabled?: boolean;
    filePath: string;
    includeHeader?: boolean;
    delimiter?: string;
  };
}

export class Config {
  // ... existing properties
  metrics?: MetricsConfig;
}

2. Auto-Initialization Logic

// src/service/MigrationScriptExecutor.ts

private autoEnableMetricsCollectors(config: Config): IMetricsCollector[] {
  const collectors: IMetricsCollector[] = [];
  
  if (config.metrics?.prometheus?.enabled) {
    collectors.push(new PrometheusCollector(config.metrics.prometheus));
  }
  
  if (config.metrics?.datadog?.enabled) {
    collectors.push(new DataDogCollector(config.metrics.datadog));
  }
  
  if (config.metrics?.console?.enabled) {
    collectors.push(new ConsoleMetricsCollector());
  }
  
  if (config.metrics?.cloudwatch?.enabled) {
    collectors.push(new CloudWatchCollector(config.metrics.cloudwatch));
  }
  
  if (config.metrics?.json?.enabled) {
    collectors.push(new JsonMetricsCollector(config.metrics.json));
  }
  
  if (config.metrics?.csv?.enabled) {
    collectors.push(new CsvMetricsCollector(config.metrics.csv));
  }
  
  return collectors;
}

constructor(handler, config, dependencies?) {
  // Auto-enable collectors from config
  const autoCollectors = this.autoEnableMetricsCollectors(config);
  
  // Merge with manual collectors
  const manualCollectors = dependencies?.metricsCollectors || [];
  const allCollectors = [...autoCollectors, ...manualCollectors];
  
  if (allCollectors.length > 0) {
    const hooksList: IMigrationHooks[] = [];
    hooksList.push(new MetricsCollectorHook(allCollectors, this.logger));
    
    if (dependencies?.hooks) {
      hooksList.push(dependencies.hooks);
    }
    
    this.hooks = new CompositeHooks(hooksList);
  }
}

3. Environment Variable Support

Support MSR_METRICS_* environment variables:

MSR_METRICS_PROMETHEUS_ENABLED=true
MSR_METRICS_PROMETHEUS_PORT=9090
MSR_METRICS_DATADOG_ENABLED=true
MSR_METRICS_DATADOG_API_KEY=${DD_API_KEY}
MSR_METRICS_CONSOLE_ENABLED=true

Requirements

• Extend Config with metrics?: MetricsConfig
• Implement autoEnableMetricsCollectors() method
• Support all built-in collectors in config
• Environment variable support (MSR_METRICS_*)
• Merge auto-enabled and manual collectors
• Update config loader to parse metrics section
• Write comprehensive tests
• Update documentation with config examples
• Migration guide from manual to config-based

Dependencies

Requires completion of:

• Implement DatadogCollector as built-in metrics collector #118 - DatadogCollector
• Implement CloudWatchCollector as built-in metrics collector #119 - CloudWatchCollector
• Implement PrometheusCollector as built-in metrics collector #120 - PrometheusCollector

Related

• Add observability and metrics collection #80 - Parent issue: Add observability and metrics collection
• Implement DatadogCollector as built-in metrics collector #118 - DatadogCollector implementation
• Implement CloudWatchCollector as built-in metrics collector #119 - CloudWatchCollector implementation
• Implement PrometheusCollector as built-in metrics collector #120 - PrometheusCollector implementation

Acceptance Criteria

• Metrics collectors auto-enabled from config
• Environment variable support
• Manual collectors still work (not breaking change)
• Config and manual collectors merged correctly
• 100% test coverage maintained
• Documentation with examples
• Migration guide provided

Benefits

• Zero code changes - Configure via msr.config.js
• 12-factor app - Environment variable support
• Easier adoption - No code needed for basic setup
• Backward compatible - Manual collectors still work