Skip to content

Scheduled connector sync (cron) #178

@RobertLD

Description

@RobertLD

Problem

Connector syncs must be triggered manually via CLI or API. There is no built-in way to schedule periodic syncs, forcing users to set up external cron jobs or scheduled tasks.

Implementation Plan

  1. Add schedule config to connector configuration:
    {
      "connectors": {
        "notion": {
          "schedule": "0 */6 * * *"
        }
      }
    }
  2. Install node-cron (or similar lightweight cron library) as a dependency
  3. Background scheduler: Create src/core/scheduler.ts:
    • startScheduler(config) — reads connector configs, registers cron jobs
    • Each job triggers the corresponding connector sync
    • Logs next-run time for each scheduled connector
    • Graceful shutdown: stop all jobs on SIGTERM/SIGINT
  4. Integrate with libscope serve: Start the scheduler when the API server starts
  5. Health check enhancement: Include next-run times in GET /api/v1/health response
  6. CLI visibility: Show schedule info in libscope connectors status

Files to Modify

  • package.json — add node-cron dependency
  • src/core/scheduler.ts — new file with scheduler logic
  • src/api/server.ts — start scheduler on serve
  • src/api/routes.ts — add schedule info to health endpoint
  • src/config.ts — add schedule field to connector config type

Testing Strategy

  • Unit test: cron expression parsing and validation
  • Unit test: scheduler registers and fires jobs (mock clock)
  • Unit test: graceful shutdown stops all jobs
  • Integration test: configure a short interval, verify sync triggers
  • Test invalid cron expressions → clear error at startup

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions