Spring Batch

Spring Batch + Boot

• Batch 수행에 대해 Boot에서 간단하게 처리해 줌. (@EnableBatchProcessing (from Spring Batch))
• 모든 Job을 실행시키는게 디폴트이지만 특정 JOB을 지정할 수도 있음. (프로퍼티 파일을 통해 지정)

Spring Batch auto configuration is enabled by adding @EnableBatchProcessing (from Spring Batch) somewhere in your context. By default it executes all Jobs in the application context on startup (see JobLauncherCommandLineRunner for details). You can narrow down to a specific job or jobs by specifying spring.batch.job.names (comma separated job name patterns). If the application context includes a JobRegistry then the jobs in spring.batch.job.names are looked up in the registry instead of being autowired from the context. This is a common pattern with more complex systems where multiple jobs are defined in child contexts and registered centrally. See BatchAutoConfiguration and @EnableBatchProcessing for more details.

Spring Batch 3.0.0 (new feature)

• JSR-352 Support (http://docs.spring.io/spring-batch/trunk/reference/html/jsr-352.html )
• Upgrade to Support Spring 4 and Java 8
• Promote Spring Batch Integration to Spring Batch
• JobScope Support
• SQLite Support

Domain

• Job : 전체 배치 프로세스 자체.
• Step : Job 내부에 세부적으로 나눠진 단계들. 모든 Job은 1개 이상의 step을 가짐.
• Chunk : 한번에 실행된 item들의 묶음.

Scaling & Parallel

• Multi-threaded Step
• Parallel Steps
• Remote Chunking of Step
• Partitioning a Step

Multi Threaded step

<step id="loading"> 
  <tasklet task-executor="taskExecutor" throttle-limit="20">...</tasklet>
</step>

• Step executes by reading, processing and writing each chunk of items (each commit interval) in a separate thread of execution.
• there is no fixed order for the items to be processed, and a chunk might contain items that are non-consecutive compared to the single-threaded case.
• default throttle limit : 4

Partitioning

<step id="step1.master">
    <partition step="step1" handler="handler"/></step>

<bean class="org.spr...TaskExecutorPartitionHandler">
    <property name="taskExecutor" ref="taskExecutor"/>
    <property name="step" ref="step1" />
    <property name="gridSize" value="10" /></bean>

• The Job is executing on the left hand side as a sequence of Steps, and one of the Steps is labelled as a Master. The Slaves in this picture are all identical instances of a Step, which could in fact take the place of the Master resulting in the same outcome for the Job.
• The Slaves are typically going to be remote services, but could also be local threads of execution
• PartitionHandler : simple RMI remoting, EJB remoting, custom web service, JMS, Java Spaces, shared memory grids (like Terracotta or Coherence), grid execution fabrics (like GridGain)
• gridSize : determines the number of separate step executions to create. default value is 10.
• spring batch ensure that one execution of slave step per one job.
• Many articles recommand this solution for solving issues of batch scaling.

Remote partitioning

• Spring Integration을 통한 remote management.
• Using MessageChannelPartitionHandler
• remote worker (remote partitionor)를 구성하고 step에 대한 응답을 받음.
• JMS, AMQP 로 abstraction 제공함. (따로 미들웨어 설치 없이 설정만으로 가능...)