ChaosToTheRescue

Safe-by-default LLM-powered method generation and Rails error rescue suggestions

ChaosToTheRescue is a Ruby gem that uses Large Language Models (LLMs) to:

1. Generate missing methods on-the-fly via method_missing
2. Suggest fixes for Rails exceptions in development

⚠️ Safety First

This gem is DISABLED BY DEFAULT and includes multiple safety features:

• ✅ Disabled by default - Must be explicitly enabled
• ✅ No auto-execution - Generated code returned as strings only (unless explicitly enabled)
• ✅ Allowlist-only - Methods must match patterns or be explicitly allowed
• ✅ Secret redaction - ENV vars, API keys, tokens filtered before LLM prompts
• ✅ No file writes - Suggestions only logged, no automatic code edits
• ✅ Development-only - Intended for development/test environments only

DO NOT USE IN PRODUCTION - This gem is designed for development and debugging workflows only.

Installation

Add to your Gemfile:

gem "chaos_to_the_rescue"

For LLM features, also add:

gem "ruby_llm", "~> 0.1"  # Required for LLM integration

Then run:

bundle install

For Rails applications, generate the initializer:

rails generate chaos_to_the_rescue:install

Configure API Keys

ChaosToTheRescue uses RubyLLM under the hood, which supports multiple LLM providers. You need to configure at least one API key:

Option 1: Environment Variables (Recommended)

export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."

Option 2: Configuration Block

ChaosToTheRescue.configure do |config|
  config.openai_api_key = "sk-..."
  # or
  config.anthropic_api_key = "sk-ant-..."
end

Supported Models:

• OpenAI: gpt-5.2, gpt-5.2-pro, gpt-5, gpt-5-mini, gpt-5-nano, gpt-4.1
• Anthropic: claude-sonnet-4-5, claude-opus-4-5, claude-sonnet-4, claude-opus-4
• Google: gemini-2.0-flash-exp, gemini-1.5-pro
• And many more via RubyLLM (xAI, DeepSeek, local models, etc.)

Features

1. Method Generation with ChaosRescue

Generate missing methods on-the-fly using LLMs. When a method is called that doesn't exist, ChaosToTheRescue can generate and define it automatically.

Supports both instance methods and class methods:

require "chaos_to_the_rescue"

ChaosToTheRescue.configure do |config|
  config.enabled = true
  config.auto_define_methods = true
  config.allowed_method_name_patterns = [/^calc_/]
end

class Calculator
  include ChaosToTheRescue::ChaosRescue
  chaos_rescue_enabled!
end

# Instance method generation
calc = Calculator.new
result = calc.calc_fibonacci(10)  # Method generated via LLM

# Class method generation
result = Calculator.calc_sum(1, 2, 3)  # Class method generated via LLM

2. Method Guidance and Verification

Provide guidance to the LLM for better method generation and verify method outputs:

Adding Guidance for Method Generation

class Calculator
  include ChaosToTheRescue::ChaosRescue
  chaos_rescue_enabled!

  # Provide guidance BEFORE the method is generated
  chaos_guidance :pow, <<~GUIDANCE
    This method should correctly handle negative bases:
    - Negative base with even exponent returns positive: (-5)^2 = 25
    - Negative base with odd exponent returns negative: (-5)^3 = -125
    Common mistake: Using base.abs which always returns positive.
  GUIDANCE
end

# Global guidance (applies to all classes)
ChaosToTheRescue.configure do |config|
  config.method_guidance[:divide] = "Handle division by zero gracefully"
end

Adding Guidance to Existing Methods

class Calculator
  def pow(base, exponent)
    base ** exponent  # Method already defined
  end

  # Add guidance AFTER definition for verification purposes
  additional_chaos_guidance :pow, "Should handle negative bases correctly"
end

Verifying Method Outputs

Verify with class method:

result = Calculator.verify_chaos(:pow, -5, 2)

if result.failed?
  puts "Issues found:"
  puts result.diagnosis
  puts result.suggestion
end

Auto-fix incorrect implementations:

# Automatically applies fix if verification fails
Calculator.verify_chaos(:pow, -5, 2, auto_apply: true)

Verify using block syntax:

# Simple verification
result = ChaosToTheRescue.verify do
  Calculator.new.pow(-5, 2)
end

# With auto-fix
result = ChaosToTheRescue.verify(auto_apply: true) do
  Calculator.new.pow(-5, 2)
end

if result.failed?
  puts "Method was incorrect:"
  puts result.diagnosis
  puts "Actual result: #{result.actual_result}"
end

See examples/guidance_and_verification.rb for comprehensive examples.

3. Rails Exception Rescue Suggestions

Get LLM-powered suggestions for fixing Rails exceptions:

class ApplicationController < ActionController::Base
  include ChaosToTheRescue::RescueFrom

  rescue_from StandardError do |exception|
    suggestion = chaos_suggest_fix(
      exception,
      guidance: "This is a payment processing error"
    )

    Rails.logger.info("Fix suggestion: #{suggestion[:title]}")
    Rails.logger.info("Diagnosis: #{suggestion[:diagnosis]}")
    Rails.logger.info("Proposed changes: #{suggestion[:proposed_changes]}")
    Rails.logger.info("Files to edit: #{suggestion[:files_to_edit].join(', ')}")

    render "errors/500", status: :internal_server_error
  end
end

Configuration

All configuration is done via the ChaosToTheRescue.configure block:

ChaosToTheRescue.configure do |config|
  # === CORE SETTINGS ===

  # Enable/disable globally (default: false)
  config.enabled = Rails.env.development? || Rails.env.test?

  # Auto-define generated methods (default: false)
  config.auto_define_methods = false

  # === METHOD GENERATION ALLOWLIST ===

  # Regex patterns for allowed method names
  config.allowed_method_name_patterns = [/^calc_/, /^compute_/]

  # Explicit allowlist of method names
  config.allowlist = [:my_dynamic_method, "another_method"]

  # Or allow all method names (use with caution!)
  config.allow_everything!

  # === LLM SETTINGS ===

  # LLM model to use (default: "gpt-5-mini")
  config.model = "gpt-5-mini"

  # Maximum characters in prompts (default: 8000)
  config.max_prompt_chars = 8000

  # === LOGGING SETTINGS ===

  # Log level (default: :info)
  config.log_level = :info

  # Log suggestions to file (default: true)
  config.log_suggestions = true

  # Suggestion log path (default: "tmp/chaos_to_the_rescue_suggestions.log")
  config.suggestions_log_path = "tmp/chaos_to_the_rescue_suggestions.log"

  # === SECURITY SETTINGS ===

  # Add custom redaction patterns (already includes common secrets)
  config.redact_patterns << /CUSTOM_SECRET[=:]\s*['"]?([^'"\s]+)['"]?/i
end

Default Redaction Patterns

The gem automatically redacts:

• Environment variable references (ENV["KEY"], ENV.fetch)
• AWS credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AKIA...)
• OpenAI tokens (sk-..., OPENAI_API_KEY)
• GitHub tokens (ghp_..., gho_..., ghu_..., ghs_..., ghr_..., GITHUB_TOKEN)
• Rails credentials (Rails.application.credentials.*)
• Generic patterns (API_KEY, SECRET, TOKEN, PASSWORD)
• Private keys (-----BEGIN PRIVATE KEY-----)

Usage Examples

Example 1: Calculator with Dynamic Methods

ChaosToTheRescue.configure do |config|
  config.enabled = true
  config.auto_define_methods = true
  config.allowed_method_name_patterns = [/^calc_/]
end

class Calculator
  include ChaosToTheRescue::ChaosRescue
  chaos_rescue_enabled!
end

# Instance methods
calc = Calculator.new
puts calc.calc_sum(1, 2, 3, 4, 5)        # LLM generates sum method
puts calc.calc_average(10, 20, 30)        # LLM generates average method
puts calc.calc_fibonacci(10)              # LLM generates fibonacci method

# Class methods
puts Calculator.calc_factorial(5)         # LLM generates class method
puts Calculator.calc_gcd(48, 18)          # LLM generates class method

Example 2: Per-Class Enable/Disable

class SafeClass
  include ChaosToTheRescue::ChaosRescue
  # Not enabled - will not generate methods
end

class ExperimentalClass
  include ChaosToTheRescue::ChaosRescue
  chaos_rescue_enabled!  # Explicitly enabled for this class
end

Example 3: Rails Controller Error Handling

class OrdersController < ApplicationController
  include ChaosToTheRescue::RescueFrom

  rescue_from StandardError do |exception|
    if ChaosToTheRescue.configuration.enabled
      suggestion = chaos_suggest_fix(
        exception,
        guidance: "User was trying to complete checkout"
      )

      # Log the suggestion
      Rails.logger.error("Exception: #{exception.class.name}: #{exception.message}")
      Rails.logger.info("Suggestion: #{suggestion[:title]}")
      Rails.logger.info("Diagnosis: #{suggestion[:diagnosis]}")

      # Check suggestion log file at tmp/chaos_to_the_rescue_suggestions.log
    end

    render "errors/500", status: :internal_server_error
  end
end

Example 4: Allowlist Specific Methods

ChaosToTheRescue.configure do |config|
  config.enabled = true
  config.allowlist = [:format_currency, :parse_date, :generate_slug]
end

class Helper
  include ChaosToTheRescue::ChaosRescue
  chaos_rescue_enabled!
end

helper = Helper.new
helper.format_currency(1234.56)  # Allowed - will generate
helper.dangerous_method          # Not allowed - raises NoMethodError

Example 5: Allow All Methods (Quick Testing)

# WARNING: Use with caution - allows any method to be generated
ChaosToTheRescue.configure do |config|
  config.enabled = true
  config.allow_everything!  # Convenience method to allow all method names
end

class Calculator
  include ChaosToTheRescue::ChaosRescue
  chaos_rescue_enabled!
end

calc = Calculator.new
calc.divide(10, 2)           # Will work - all methods allowed
calc.anything_you_want(1, 2) # Will work - all methods allowed

How It Works

Method Generation Flow

1. You call a missing method (instance or class method) on a class that includes ChaosRescue
2. ChaosToTheRescue checks if the gem is enabled (globally and per-class)
3. Checks if the method name matches allowlist or patterns
4. Builds context (class name, method name, arguments, method type)
5. Redacts any sensitive information from the context
6. Sends prompt to LLM to generate method code
7. Optionally defines the method on the object or class (if auto_define_methods enabled)
8. Returns the result

Exception Suggestion Flow

1. An exception occurs in a Rails controller
2. You call chaos_suggest_fix in your rescue_from block
3. ChaosToTheRescue builds exception context (class, message, backtrace)
4. Sanitizes request data and redacts secrets
5. Sends prompt to LLM to analyze the error
6. Returns structured suggestion with diagnosis and proposed changes
7. Logs suggestion to file if enabled

Security Considerations

What Gets Redacted

Before sending any data to an LLM, ChaosToTheRescue redacts:

• All environment variable references
• API keys, secrets, tokens, passwords
• AWS, OpenAI, GitHub credentials
• Rails credentials
• Private keys

What Doesn't Get Sent

• File contents (only file paths from backtraces)
• User session data (unless in exception context)
• Database credentials

Recommendations

1. Never enable in production - Only use in development/test
2. Review prompts - Set log_level: :debug to see what's sent to LLM
3. Use allowlists - Don't allow all methods, use specific patterns
4. Don't auto-define - Keep auto_define_methods: false and review generated code
5. Custom redaction - Add your own patterns for domain-specific secrets

Development

After checking out the repo:

bin/setup              # Install dependencies
bundle exec rspec      # Run tests
bin/console            # Interactive prompt
bundle exec standardrb # Lint code

Testing

Run the full test suite:

bundle exec rspec

Run specific test files:

bundle exec rspec spec/configuration_spec.rb
bundle exec rspec spec/chaos_rescue_spec.rb

Troubleshooting

"RubyLLM is not available"

You need to add ruby_llm to your Gemfile:

gem "ruby_llm", "~> 0.1"

Then run bundle install.

Methods Not Being Generated

Check that:

1. config.enabled = true is set
2. Class has chaos_rescue_enabled! called
3. Method name matches patterns or is in allowlist
4. RubyLLM is installed and configured

No Suggestions Logged

Check that:

1. config.log_suggestions = true is set
2. The tmp/ directory exists and is writable
3. ChaosToTheRescue is enabled

Roadmap

Potential future features:

• Multiple LLM provider support (Anthropic, Cohere, local models)
• Caching of generated methods
• Web UI for reviewing suggestions
• Type signature validation
• Integration with CI/CD for automated error analysis

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/codenamev/chaos_to_the_rescue.

This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the ChaosToTheRescue project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.

Credits

Created by Valentino Stoll

Disclaimer

This gem is experimental and should only be used in development environments. The authors are not responsible for any issues arising from using generated code in production. Always review generated code before using it in any capacity.