Spring Batch

Spring Batch + Boot

• Batch 수행에 대해 Boot에서 간단하게 처리해 줌. (@EnableBatchProcessing (from Spring Batch))
• 모든 Job을 실행시키는게 디폴트이지만 특정 JOB을 지정할 수도 있음. (프로퍼티 파일을 통해 지정)

Spring Batch auto configuration is enabled by adding @EnableBatchProcessing (from Spring Batch) somewhere in your context. By default it executes all Jobs in the application context on startup (see JobLauncherCommandLineRunner for details). You can narrow down to a specific job or jobs by specifying spring.batch.job.names (comma separated job name patterns). If the application context includes a JobRegistry then the jobs in spring.batch.job.names are looked up in the registry instead of being autowired from the context. This is a common pattern with more complex systems where multiple jobs are defined in child contexts and registered centrally. See BatchAutoConfiguration and @EnableBatchProcessing for more details.

Spring Batch 3.0.0 (new feature)

• JSR-352 Support (http://docs.spring.io/spring-batch/trunk/reference/html/jsr-352.html )

<?xml version="1.0" encoding="UTF-8"?>
<job id="myJob3" xmlns="http://xmlns.jcp.org/xml/ns/javaee" version="1.0">
    <step id="step1" >
        <batchlet ref="testBatchlet" />
    </step>
</job>

• Upgrade to Support Spring 4 and Java 8
• Promote Spring Batch Integration to Spring Batch (Launching jobs via messages, Async ItemProcessor, feedback message, remote partitioning&remote chunking)
• JobScope Support
• SQLite Support

Domain

• Job : 전체 배치 프로세스 자체.
• Step : Job 내부에 세부적으로 나눠진 단계들. 모든 Job은 1개 이상의 step을 가짐.
• Chunk : 한번에 실행된 item들의 묶음.

Scaling & Parallel

Reference Doc

• Multi-threaded Step
• Parallel Steps
• Remote Chunking of Step
• Partitioning a Step

Multi Threaded step

<step id="loading"> 
  <tasklet task-executor="taskExecutor" throttle-limit="20">...</tasklet>
</step>

• Step executes by reading, processing and writing each chunk of items (each commit interval) in a separate thread of execution. commit interval 단위로 item을 쪼개서 여러 스레드에서 step이 수행됨.
• there is no fixed order for the items to be processed, and a chunk might contain items that are non-consecutive compared to the single-threaded case. 순서 보장되지 않음. 실행되는 chunk 내의 item 역시 sequencial하지 않을 수 있음.
• default throttle limit : 4

Partitioning

<step id="step1.master">
    <partition step="step1" handler="handler"/></step>

<bean class="org.spr...TaskExecutorPartitionHandler">
    <property name="taskExecutor" ref="taskExecutor"/>
    <property name="step" ref="step1" />
    <property name="gridSize" value="10" /></bean>

• The Job is executing on the left hand side as a sequence of Steps, and one of the Steps is labelled as a Master. The Slaves in this picture are all identical instances of a Step, which could in fact take the place of the Master resulting in the same outcome for the Job. 특정한 Step에 대해 M/S 구조로 구동하게 함. Master가 된 step은 slave들에 대해 구동 시그널을 보내고 응답만 받음.
• The Slaves are typically going to be remote services, but could also be local threads of execution. Slave의 경우 remote/local 모두 가능함.
• PartitionHandler : simple RMI remoting, EJB remoting, custom web service, JMS, Java Spaces, shared memory grids (like Terracotta or Coherence), grid execution fabrics (like GridGain) <-- Remote에 대해 지원되는 방식
• gridSize : determines the number of separate step executions to create. default value is 10. 생성되는 slave의 max count.
• spring batch ensure that one execution of slave step per one job. 한 JOB의 step에 대해 slave가 한번만 실행되는 걸 프레임웍 레벨에서 보장함.
• Many articles recommand this solution for solving issues of batch scaling.

Remote partitioning

• Spring Integration을 통한 remote management.
• Using MessageChannelPartitionHandler
• remote worker (remote partitionor)를 구성하고 step에 대한 응답을 받음.
• JMS, AMQP 로 abstraction 제공함. (따로 미들웨어 설치 없이 설정만으로 가능...)