-
Notifications
You must be signed in to change notification settings - Fork 3k
improvement(kb): add fallbacks for kb configs #1199
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Summary
This PR introduces configurable environment variables for Knowledge Base (KB) document processing configuration, replacing hardcoded values throughout the codebase with environment variable lookups that include sensible fallback defaults. The changes affect three key files:
Environment Configuration (apps/sim/lib/env.ts): Adds 9 new KB_CONFIG_* environment variables with default values using Zod schema validation. These include processing duration (300s), retry attempts (3), backoff factor (2), timeout ranges (1-10 seconds), concurrency limits (20), batch sizes (20), and processing delays (50-100ms).
Background Processing (apps/sim/background/knowledge-processing.ts): Updates the Trigger.dev task configuration to use the new environment variables instead of hardcoded values for maxDuration, retry policies, and queue concurrency settings.
Document Service (apps/sim/lib/knowledge/documents/service.ts): Replaces hardcoded configuration values with environment variable lookups across multiple processing configurations including timeouts, concurrency limits, batch sizes, and Redis processing parameters.
This change centralizes KB processing configuration management, making the system more operationally flexible while maintaining backward compatibility. The fallback mechanism ensures the application continues to function properly even when environment variables are not explicitly set, which is particularly important for the multi-tier processing system that includes Trigger.dev background jobs, Redis queue processing, and in-memory processing fallbacks.
Confidence score: 4/5
- This PR is safe to merge with low risk of breaking existing functionality
- Score reflects well-structured environment configuration changes with proper fallbacks, though the complexity of multiple processing modes requires attention
- Pay close attention to
apps/sim/lib/knowledge/documents/service.tsfor the complex fallback logic and mathematical operations
3 files reviewed, 2 comments
4063eac to
1381489
Compare
Summary
add fallbacks for kb configs
Type of Change
Testing
Tested manually.
Checklist