It is an easy to use distributed task scheduling platform. It is derived from Taobao lottery scheduling platform and has been used in the Taobao content crawling platform and The Tiejia Used Equipment Of Construction Machinery task scheduling platform. It is currently refactored and open source. It has the following characteristics:
- Decentralization.
- Support applying to hundreds of application server for different services. Tasks are executed independently on each application, taking full advantage of the application's own resources.
- Failover. If the application has multiple execution machines, one machine hangs and the tasks on it will switch to other machines.
- The management console unifies modification, start and stop the task, and so on. Hanging out of the console does not affect the execution of the task.
- You can view the task log online to get a real-time view of task execution.
- The general task scheduling system, the code logic is integrated into the scheduling system, it is necessary to move the business system scheduling code and dependent code into the scheduling system, which is very troublesome. However, Easy Task is only responsible for scheduling, and the complex code logic is still developed and executed in the respective business systems, ensuring the efficiency of development and avoiding the influence of business systems.
- You can view the execution logs of all tasks online by page, and task execution is no longer a black box.
If you have several business systems (as worker) that want to use the task scheduling platform, you can download and launch a console directly. The console is also a worker, and can also execute tasks. By default, a demoPlugin task for the demo is started (the actual scene console is only managed, not task execution).
- Download and start
Download from the release or download the executable jar file directly https://github.com/cehome-com/resource/raw/master/easy-task/2.0.3/task-console.jar
Then execute the command to start: java -jar task-console.jar - Visit http://localhost:8080 and you can see a demo task
- Task Management.
Click the "Log" button, you can see the task execution log (if not, you can stop for 10 seconds and then refresh)
Click "Modify" to view or modify the task configuration. The system is based on spring, and the bean name "demoPlugin" is a built-in spring bean.
Click "Stop" to stop the task.
- For the service system applying, see below how to apply to the “spring boot” and “sring mvc” applications.
If you only want to quickly deploy a simple and usable scheduling system, you don't need to consider the existing business system, you can directly use the task-console source code development.
- Check out the task-console spring boot code module, import it into the IDE, and execute com.cehome.task.console.TaskConsoleApplication for start.
- Visit http://localhost:8080
- Click the "Log" button to view the log of the built-in demoPlugin task.
- Modify the demoPlugin corresponding class com.cehome.task.console.DemoPlugin to redeploy and view the execution results.
In actual use, the console is just an administrative task, no tasks are executed, and tasks are executed in the business system. The following simulates scheduling platform is applied to two service systems named "app1" and "app2". For convenience, still use the task-console.jar for simulation. Before proceeding, first make sure that the above console is still running.
- Launch another command line window and execute commands as follows for the start of app1 (port 8091)
java -jar task-console.jar --task.factory.appName=app1 --server.port=8091 - Launch another command line window and execute commands as follows for the start of app1 (port 8092)
java -jar task-console.jar --task.factory.appName=app2 --server.port=8092 - Switch the task to app2 to execute. Go to http://localhost:8080, click the "Modify" button, pop up the modification page, click the "Application" drop-down box, you should be able to see APP1, APP2 is also inside, select APP2, then save and close.
- Observe the command line output window of app2 and find that the demo has been transferred to app2 for execution.
Use the task-spring-boot-client-demo module to explain how to apply the scheduling platform to the spring boot application.
- Increase dependence
<dependency>
<groupId>com.cehome</groupId>
<artifactId>task</artifactId>
<version>2.0.3</version>
</dependency>
- Join the @EnableTimeTaskClient annotation
package com.cehome.task.client.demo;
import com.cehome.task.annotation.EnableTimeTaskClient;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
@EnableTimeTaskClient
public class BootApplication {
public static void main(String[] args) {
SpringApplication.run(BootApplication.class,args);
}
}
- Modify the configuration file named "application.properties"
Description of the requirements configuration:
task.factory.appName - The application name, which should be different for different applications.
task.factory.name - The name of the cluster, which is also the name of the database table. Do not arbitrarily change it after you have made it.
task.datasource.* Configure database information to support H2 and mysql databases. It is recommended that the production environment use the mysql database.
task.log.path - logback task log output path.
Task.log.packages - The package name of the task execution class, used for logging. Multiple package names are separated by a semicolon. If it is not sure, use the ROOT root log.
spring.application.name=boot-client-demo
server.port=8081
#------ main options --------
#应用的名称
task.factory.appName=boot-client-demo
#集群名称(同时也是数据库表名)
task.factory.name=easy_task
#h2数据库配置
task.datasource.driverClassName=org.h2.Driver
task.datasource.url=jdbc:h2:tcp://localhost:9092/~/easy_task_db;MODE=MYSQL
task.datasource.username=sa
task.datasource.password=
#------ client options --------
task.log.packages=ROOT
task.log.path=/logs/easy_task/boot_demo
- Development task plugin
The task plugin can extend com.cehome.task.client.TimeTaskPlugin. Since the execution method named "run()" is fixed, you can not specify the method name when configuring the task information in the console. The stop() method fires when the stop task is clicked, and the code application stops the task execution and releases the necessary resources.
package com.cehome.task.client.demo;
import com.alibaba.fastjson.JSONObject;
import com.cehome.task.client.TimeTaskContext;
import com.cehome.task.client.TimeTaskPlugin;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import java.net.Inet4Address;
@Component
public class BootDemoPlugin extends TimeTaskPlugin {
private static final Logger logger = LoggerFactory.getLogger(BootDemoPlugin.class);
@Override
public void run(TimeTaskContext context, JSONObject args) throws Exception {
logger.info("plugin class name="+this);
logger.info("task id="+context.getId()+",name="+context.getName());
logger.info("task run on ip="+ Inet4Address.getLocalHost().getHostAddress());
logger.info("task run count="+context.getRunTimes());
}
@Override
public void stop(TimeTaskContext context) throws Exception {
logger.info("task "+context.getName()+" is stopped ");
}
}
The task plugin can also be a normal spring bean, but when configuring the task information in the console, you need to specify the method to be executed.
- Start the application task-spring-boot-client-demo
- Visit http://localhost:8080 (make sure the console is the state of starting), click on the "Add" task, "Application" select "boot-client-demo"; plan time to fill in 5s (5 seconds); Bean name and above the developed plugin is consistent, fill in the "bootDemoPlugin"; other required fields are fill in at random
- Save and close, click "Start", then click "Log" button after more than 10 seconds, if you see the log of "task run...", everything is fine.
Use the task-spring-mvc-client-demo module to explain .
- Increase dependence
<dependency>
<groupId>com.cehome</groupId>
<artifactId>task</artifactId>
<version>2.0.3</version>
</dependency>
- Import the bean in the spring.xml file:
<import resource="classpath*:task-client-spring-config.xml"></import>
- Introducing spring/config.properties configuration information in spring.xml
<context:annotation-config/>
<context:property-placeholder location="classpath*:spring/config.properties"/>
- Add configuration information in the corresponding spring/config.properties:
task.factory.appName - The application name, which should be different for different applications.
task.factory.name - The name of the cluster, which is also the name of the database table. Do not arbitrarily change it after you have made it.
task.datasource.* Configure database information to support H2 and mysql databases. It is recommended that the production environment use the mysql database.
task.log.path - logback task log output path.
task.log.packages - is the package’s name of the task execution class, used for logging. Multiple package names are separated by a semicolon. If it is not sure, use the ROOT root log.
task.factory.appName=mvc-client-demo
task.factory.name=easy_task
#h2
task.datasource.driverClassName=org.h2.Driver
task.datasource.url=jdbc:h2:tcp://localhost:9092/~/easy_task_db;MODE=MYSQL
task.datasource.username=sa
task.datasource.password=
#------ client options --------
task.log.packages=com.cehome.task.client.demo
task.log.path=/logs/easy_task/mvc_demo
- Configure the controller of remote online log view in file named "springmvc.xml". Without this controller, the console cannot connect to the application to view the log online.
<context:component-scan
base-package="com.cehome.task.client.controller"/>
<mvc:annotation-driven />
- Development task plugin
The task plugin can extend com.cehome.task.client.TimeTaskPlugin. Since the execution method named "run()" is fixed, you can not specify the method name when configuring the task information in the console. The stop() method fires when the stop task is clicked, and the code application stops the task execution and releases the necessary resources.
package com.cehome.task.client.demo;
import com.alibaba.fastjson.JSONObject;
import com.cehome.task.client.TimeTaskContext;
import com.cehome.task.client.TimeTaskPlugin;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import java.net.Inet4Address;
@Component
public class MvcDemoPlugin extends TimeTaskPlugin {
private static final Logger logger = LoggerFactory.getLogger(MvcDemoPlugin.class);
@Override
public void run(TimeTaskContext context, JSONObject args) throws Exception {
logger.info("plugin class name="+this);
logger.info("task id="+context.getId()+"task name="+context.getName());
logger.info("task run on ip="+ Inet4Address.getLocalHost().getHostAddress());
logger.info("task run count="+context.getRunTimes());
}
@Override
public void stop(TimeTaskContext context) throws Exception {
logger.info("task "+context.getName()+" is stopped ");
}
}
The task plugin can also be a normal spring bean, but when configuring the task information in the console, you need to specify the method to be executed.
- Add the scan path of the plugin in spring.xml
<context:component-scan base-package="com.cehome.task.client.demo"/>
- Launch the application named "task-spring-mvc-client-demo"
- Visit console (not mvc-demo) http://localhost:8080 (make sure the console is the state of starting), click on the "Add" task, select "mvc-client-demo" for the application; fill in 5s (5 seconds) of the scheduled time; The bean name is the same as the above development plugin, fill in the "mvcDemoPlugin"; other required fields are fill in at random.
- Save and close, click "Start", then click "Log" button after more than 10 seconds, if you see the log of "task run..." , everything is fine.
By default, the console starts an internal database, and the production environment recommends using an external database. Still use the H2 database to illustrate:
- Start the H2 database
Go to http://www.h2database.com/html/download.html and download the h2 database and extract it. Go to the “bin” directory and execute the command to start the database (9092 is the database access port).
java -cp h2*.jar org.h2.tools.Server -tcpPort 9092 -tcpAllowOthers -webPort 8082 -webAllowOthers - Modify the console or client database configuration information
task.datasource.driverClassName=org.h2.Driver
task.datasource.url=jdbc:h2:tcp://192.168.0.10:9092/~/easy_task_db;MODE=MYSQL
task.datasource.username=sa
task.datasource.password=
If it is a console, you can modify “task.h2.start=false” to disable the internal database.
task.h2.start=false
task.datasource.driverClassName=com.mysql.jdbc.Driver
task.datasource.url=jdbc:mysql://192.168.0.13:3306/scheduler?useUnicode=true&characterEncoding=utf8&allowMultiQueries=true
task.datasource.username=root
task.datasource.password=123456
- Independent mysql database
- Console has at least two nodes
- Several applications, each application maintains at least two nodes
English(en) and chinese(cn) is supported. Default is english, Add "language=cn" to application.properties to enable chinese.