Skip to content

Elastic-Job is a distributed scheduled job framework, based on Quartz and Zookeeper.

License

Notifications You must be signed in to change notification settings

QuVideo/elastic-job

 
 

Repository files navigation

Elastic-Job - distributed scheduled job solution

Build Status Maven Status Coverage Status GitHub release Hex.pm

Overview

Elastic-Job is a distributed scheduled job solution. Elastic-Job is composited from 2 independent sub projects: Elastic-Job-Lite and Elastic-Job-Cloud.

Elastic-Job-Lite is a centre-less solution, use lightweight jar to coordinate distributed jobs. Elastic-Job-Cloud is a Mesos framework which use Mesos + Docker(todo) to manage and isolate resources and processes.

Elastic-Job-Lite and Elastic-Job-Cloud provide unified API. Developers only need code one time, then decide to deploy Lite or Cloud as you want.

Features

1. Elastic-Job-Lite

  • Distributed schedule job coordinate
  • Elastic scale in and scale out supported
  • Failover
  • Misfired jobs refire
  • Sharding consistently, same sharding item for a job only one running instance
  • Self diagnose and recover when distribute environment unstable
  • Parallel scheduling supported
  • Job lifecycle operation
  • Lavish job types
  • Spring integrated and namespace supported
  • Web console

2. Elastic-Job-Cloud

  • All Elastic-Job-Lite features included
  • Application distributed automatically
  • Fenzo based resources allocated elastically
  • Docker based processes isolation support (TBD)

Architecture

Elastic-ob-Lite

Elastic-Job-Lite Architecture


Elastic-Job-Cloud

Elastic-Job-Cloud Architecture

Quick Start

Elastic-Job-Lite

Add maven dependency

<!-- import elastic-job lite core -->
<dependency>
    <groupId>com.dangdang</groupId>
    <artifactId>elastic-job-lite-core</artifactId>
    <version>${lasted.release.version}</version>
</dependency>

<!-- import other module if need -->
<dependency>
    <groupId>com.dangdang</groupId>
    <artifactId>elastic-job-lite-spring</artifactId>
    <version>${lasted.release.version}</version>
</dependency>

Job development

public class MyElasticJob implements SimpleJob {
    
    @Override
    public void execute(ShardingContext context) {
        switch (context.getShardingItem()) {
            case 0: 
                // do something by sharding item 0
                break;
            case 1: 
                // do something by sharding item 1
                break;
            case 2: 
                // do something by sharding item 2
                break;
            // case n: ...
        }
    }
}

Job configuration

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:reg="http://www.dangdang.com/schema/ddframe/reg"
    xmlns:job="http://www.dangdang.com/schema/ddframe/job"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
                        http://www.springframework.org/schema/beans/spring-beans.xsd
                        http://www.dangdang.com/schema/ddframe/reg
                        http://www.dangdang.com/schema/ddframe/reg/reg.xsd
                        http://www.dangdang.com/schema/ddframe/job
                        http://www.dangdang.com/schema/ddframe/job/job.xsd
                        ">
    <!--configure registry center -->
    <reg:zookeeper id="regCenter" server-lists="yourhost:2181" namespace="dd-job" base-sleep-time-milliseconds="1000" max-sleep-time-milliseconds="3000" max-retries="3" />

    <!--configure job -->
    <job:simple id="myElasticJob" class="xxx.MyElasticJob" registry-center-ref="regCenter" cron="0/10 * * * * ?"   sharding-total-count="3" sharding-item-parameters="0=A,1=B,2=C" />
</beans>

Elastic-Job-Cloud

Add maven dependency

<!-- import elastic-job cloud executor -->
<dependency>
    <groupId>com.dangdang</groupId>
    <artifactId>elastic-job-cloud-executor</artifactId>
    <version>${lasted.release.version}</version>
</dependency>

Job development

Same with Elastic-Job-Lite

Job App configuration

curl -l -H "Content-type: application/json" -X POST -d 
'{"appName":"yourAppName","appURL":"http://app_host:8080/foo-job.tar.gz","cpuCount":0.1,"memoryMB":64.0,"bootstrapScript":"bin/start.sh","appCacheEnable":true}' 
http://elastic_job_cloud_host:8899/app

Job configuration

curl -l -H "Content-type: application/json" -X POST -d 
'{"jobName":"foo_job","jobClass":"yourJobClass","jobType":"SIMPLE","jobExecutionType":"TRANSIENT","cron":"0/5 * * * * ?","shardingTotalCount":5,"cpuCount":0.1,"memoryMB":64.0,"appURL":"http://app_host:8080/foo-job.tar.gz","failover":true,"misfire":true,"bootstrapScript":"bin/start.sh"}' 
http://elastic_job_cloud_host:8899/job/register

About

Elastic-Job is a distributed scheduled job framework, based on Quartz and Zookeeper.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Java 63.1%
  • JavaScript 30.7%
  • HTML 5.1%
  • Other 1.1%