Add AI-powered regex pattern detection for automatic schema enhancement

[Copilot]

This PR implements an optional AI-powered pattern detection feature that automatically identifies regex patterns in data fields to enhance schema generation with appropriate validation matchers.

Overview

The data stream comparator now supports intelligent pattern detection that can automatically generate regex patterns for common data types like emails, phone numbers, URLs, and more. This feature works in two modes:

• Offline Mode: Uses built-in pattern recognition algorithms for common data patterns
• Online Mode: Integrates with Claude/Anthropic APIs for sophisticated AI-powered pattern analysis

Key Features

Automatic Pattern Detection

# Before: Basic schema without matchers
email:
  type: string
  
# After: Enhanced schema with AI-detected patterns  
email:
  type: string
  matchers:
    - regex: ^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$

Flexible Configuration

pattern_detection:
  enabled: true
  mode: offline  # or "online" for AI APIs
  
  # Online mode configuration
  online_api:
    provider: claude
    api_key: "your-api-key"
    model: "claude-3-haiku-20240307"

Built-in Pattern Recognition

The offline mode includes recognition for:

• Email addresses
• Phone numbers (with smart validation to avoid false positives)
• URLs and web addresses
• IP addresses
• UUIDs
• Numeric and datetime values

Implementation Details

• Modular Architecture: New patterndetection package with pluggable detector interface
• Backward Compatible: Existing functionality unchanged, pattern detection is completely optional
• Error Resilient: Graceful fallback when pattern detection fails
• Well Tested: Comprehensive test coverage including unit tests and integration tests

Usage Example

// Enable pattern detection in schema generation
schema, err := schema.GenerateWithPatternDetection(reader, samplerConfig, &config.PatternDetection{
    Enabled: true,
    Mode:    "offline",
})

The feature significantly enhances the data validation capabilities of the schema generation process while maintaining full backward compatibility with existing configurations.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.