You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added
Maintenance queue for non-fetch jobs. New third queue (source_monitor_maintenance) separates non-time-sensitive jobs from the fetch pipeline. Health checks, cleanup, favicon fetching, image downloading, and OPML import jobs now run on the maintenance queue. Configure via config.maintenance_queue_name and config.maintenance_queue_concurrency.
Configurable scheduler batch size.config.fetching.scheduler_batch_size (default 25, was hardcoded at 100) controls how many sources are picked up per scheduler run.
Configurable stale fetch timeout.config.fetching.stale_timeout_minutes (default 5, was hardcoded at 10) controls how long a source can remain in "fetching" status before reset.
Stagger fetch times rake task.source_monitor:maintenance:stagger_fetch_times distributes overdue sources across a configurable time window.
Fixed-interval sources now get scheduling jitter. ±10% jitter on next_fetch_at, preventing thundering herd effects.
ScheduleFetchesJob uses configured batch size instead of legacy constant.
Changed
Default scheduler batch size reduced from 100 to 25.
Default stale fetch timeout reduced from 10 to 5 minutes.
7 jobs moved from fetch queue to maintenance queue: SourceHealthCheckJob, ImportSessionHealthCheckJob, ImportOpmlJob, LogCleanupJob, ItemCleanupJob, FaviconFetchJob, DownloadContentImagesJob.
Upgrade Notes
Action required: Update your solid_queue.yml to include the new maintenance queue.
Run bin/rails source_monitor:maintenance:stagger_fetch_times after upgrading if many sources are overdue.