fix(walker): handle large directories

[tusharmath]

Summary

This refactoring centralizes filesystem traversal configuration by moving the Walker implementation from forge_services to forge_app, streamlines workflow configuration handling with improved merging capabilities, and integrates the Walker pattern throughout the codebase for more consistent file discovery operations.

Key Changes

• Centralized Walker Configuration: Moved Walker and WalkedFile from forge_services::infra to forge_app::walker for better organization and reusability
• Streamlined Workflow Handling: Replaced direct workflow reading with merged configuration approach using read_merged() method
• Improved File Discovery: Updated FileDiscoveryService to use Walker configuration instead of simple depth parameters
• Configuration Consolidation: Moved max_walker_depth from agent-specific to global configuration in forge.default.yaml

Technical Details

Walker Refactoring

• Created new forge_app::walker module with Walker struct providing conservative and unlimited presets
• Updated all services to use the centralized Walker configuration
• Maintained backward compatibility while improving configurability

Workflow Integration

• Replaced WorkflowService::read_workflow() calls with read_merged() for better configuration handling
• Integrated workflow-based walker depth configuration into the new Walker pattern
• Simplified configuration passing between services

API Updates

• Updated FileDiscoveryService::collect() to collect_files() with Walker parameter
• Modified discovery service implementations to use unified Walker configuration
• Updated API endpoints to use new configuration approach

Files Modified

• crates/forge_app/src/walker.rs - New centralized Walker implementation
• crates/forge_app/src/app.rs - Integrated Walker with workflow configuration
• crates/forge_services/src/infra.rs - Removed duplicate Walker types
• crates/forge_services/src/discovery.rs - Updated to use centralized Walker
• forge.default.yaml - Moved max_walker_depth to global configuration

This refactoring improves code organization, reduces duplication, and provides a more flexible foundation for filesystem traversal operations across the application.