Skip to content
forked from zlzforever/Swarm

Swarm is a distributed scheduled job framework, based on Quartz.

License

Notifications You must be signed in to change notification settings

jangocheng/Swarm

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SWARM

Swarm is a distributed scheduled job framework, based on Quartz.

DESGIN

  +------------------+      +------------+        +------------------+      +------------+ 
  |  Swarm Server 1  +------>            |        |  Swarm Server 3  +------>            |
  +------------------+      |            |        +------------------+      |            | 
                            |Quartz DB 1 |                                  |Quartz DB 2 |
  +------------------+      |            |        +------------------+      |            |
  |  Swarm Server 2  +------>            |        |  Swarm Server 4  +------>            |
  +------------------+      +------------+        +------------------+      +------------+

                            +----------------------------------+
                            |             Swarm DB             |
                            +----------------------------------+

--------------------------------------SHARDING MODULE---------------------------------------
                                             |
                                 HTTP, WebSocket, Kafka, etc
                                             |
               +-------------------+---------+---------+-------------------+
               |                   |                   |                   | 
       +-------v--------+  +-------v--------+  +-------v--------+  +-------v--------+
       | Swarm Client 1 |  | Swarm Client 2 |  | Swarm Client 3 |  | Swarm Client 4 |
       +----------------+  +----------------+  +----------------+  +----------------+

SWARM SHARDING NODE
  • 2 or more Swarm Server as a quartz cluster node to make sure triggers will be performed. SSN use a independent DB.
SHARDING MODULE
  • Sharding request server load, database load, job count from Swarm servers every 5 second
  • Sharding choose a Swarm server to create/update/delete job
EXECUTE JOB TYPE
  • Process: Client receive message from Swarm server, then start a process to execute the job
  • Reflection: Swarm client load all DLLs from job directory, create a job instance and invoke the Task Handler(JobContext context) method
TRIGGER TYPE
  • Cron,
  • Simple,
  • DailyTimeInterval,
  • CalendarInterval
PERFOM TYPE
  • WebSocket
  • HTTP
  • Kafka
  • Other

WHY SWARM?

Quartz is a great scheduler framework, but if we have millions or more jobs or trigger very frequent, one database can't load this scenario. So we want to implement a distributed scheduler system can scheudler millions jobs, millions clients

CONSIDERATION

  • Client can't connect to Swarm server, but processes of jobs are still running, when the server restart, server should know those information and update job's state to database?
  • Client down, all processes it opened still alive? may be we should store process<-> job info to help rescue client.

CONTRIBUTION

  1. Fork the project
  2. Create Feat_xxx branch
  3. Commit your code
  4. Create Pull Request

About

Swarm is a distributed scheduled job framework, based on Quartz.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C# 81.2%
  • CSS 11.5%
  • JavaScript 7.3%