Retry not working with FaultTolerantStepBuilder [BATCH-2562]

[spring-projects-issues]

Christoph Kamil Wozniak opened BATCH-2562 and commented

When using the FaultTolerantStepBuilder to define a retry policy, no reties are triggered on Exceptions in ItemReader.

POC:

@Configuration
@EnableBatchProcessing
public static class TestConfiguration {

    // other beans

    @Bean
    @Qualifier("importFullJob")
    public Job importFullJob(ItemReader itemReader) {
        TaskletStep mockStep = stepBuilderFactory.get("mockStep")
                .chunk(1)
                .faultTolerant()
                .retry(RestClientException.class)
                .retryLimit(10)
                .reader(itemReader)
                .processor(item -> "processed")
                .writer(items -> {
                })
                .build();

        return jobBuilderFactory.get("importFullJob").start(mockStep).build();
    }

    @Bean
    @JobScope
    public ItemReader itemReader() {
        return new ItemReader() {

            @Override
            public Object read() throws Exception {
                System.out.println("try read");

                throw new RestClientException("booom");
            }
        };
    }
}

Instead of 10 retries Exception is thrown only once:

org.springframework.batch.core.step.skip.NonSkippableReadException: Non-skippable exception during read
    at org.springframework.batch.core.step.item.FaultTolerantChunkProvider.read(FaultTolerantChunkProvider.java:105) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:116) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374) ~[spring-batch-infrastructure-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144) ~[spring-batch-infrastructure-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.step.item.SimpleChunkProvider.provide(SimpleChunkProvider.java:110) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:69) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133) ~[spring-tx-4.2.7.RELEASE.jar:4.2.7.RELEASE]
    at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:271) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374) ~[spring-batch-infrastructure-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144) ~[spring-batch-infrastructure-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:257) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:200) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:148) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.job.AbstractJob.handleStep(AbstractJob.java:392) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.job.SimpleJob.doExecute(SimpleJob.java:135) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:306) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) [spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_72]
Caused by: org.springframework.web.client.RestClientException: buuuum
    at org.woezelmann.batch.starter.FullImportStarterIT$TestConfiguration$1.read(FullImportStarterIT.java:256) ~[test-classes/:?]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_72]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_72]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_72]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_72]
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) ~[spring-aop-4.2.7.RELEASE.jar:4.2.7.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.2.7.RELEASE.jar:4.2.7.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.2.7.RELEASE.jar:4.2.7.RELEASE]
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133) ~[spring-aop-4.2.7.RELEASE.jar:4.2.7.RELEASE]
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121) ~[spring-aop-4.2.7.RELEASE.jar:4.2.7.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.7.RELEASE.jar:4.2.7.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) ~[spring-aop-4.2.7.RELEASE.jar:4.2.7.RELEASE]
    at com.sun.proxy.$Proxy68.read(Unknown Source) ~[?:?]
    at org.springframework.batch.core.step.item.SimpleChunkProvider.doRead(SimpleChunkProvider.java:91) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    at org.springframework.batch.core.step.item.FaultTolerantChunkProvider.read(FaultTolerantChunkProvider.java:87) ~[spring-batch-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
    ... 22 more

---

Affects: 3.0.7

Issue Links:

• BATCH-2643 retry can't use in reader?
  ("is duplicated by")

[spring-projects-issues]

Surya Prakash Raja commented

Reading is not retryable i.e., if an exception occurs while reading, the skip policy is immediately called because it's not possible to "retry" a reading. It is a read only contract. Refer to the below link.
http://forum.spring.io/forum/spring-projects/batch/75256-retry-in-case-of-a-failed-reading

[spring-projects-issues]

Mahmoud Ben Hassine commented

The retry policy in a chunk oriented step is not applied to the reader. It is applied to the chunkProcessor (processor and writer). The reason for that is because the contract of an item reader is "forward only", there is no way (in the current interface) to go back to a previous position in case of a retry.

At a high level, a chunk oriented step calls two components: a ChunkProvider (provides a chunk by calling the item reader) and a ChunkProcessor (processes a chunk by calling the processor + writer). When you specify a retry policy in a fault tolerant step, a FaultTolerantChunkProcessor is configured with the retry policy (which is not the case for the chunk provider). Hence, the retry policy is applied to the processor and writer but not for the reader.

Here are two SO threads that might help:

• https://stackoverflow.com/questions/56170179/retry-not-working-with-spring-batch-with-java-config
• https://stackoverflow.com/questions/56045376/spring-batch-using-skippolicy-and-retrypolicy-with-springboot