Repetitive execution of FaultTolerantChunkProcessor's method process

[alappmeng]

Bug description
When FaultTolerantChunkProcessor's method process throw a exeception(not the retriable exception), the method process will execute once more.

Environment
The version of the spring-batch-core that I use is 5.0.3.

Steps to reproduce
Steps to reproduce the issue.

Expected behavior
When method process throws a exception that dont need to retry, do not execute process.

[fmbenhassine]

How is the fault-tolerant step configured? Have you added that type of exception to the non-retrybale ones with FaultTolerantStepBuilder#noRetry? It could be that chunk scanning is triggered while it shouldn't.

Please elaborate with a code example to help us reproduce your issue and help you efficiently.

[chengwei710]

How is the fault-tolerant step configured? Have you added that type of exception to the non-retrybale ones with FaultTolerantStepBuilder#noRetry? It could be that chunk scanning is triggered while it shouldn't.

Please elaborate with a code example to help us reproduce your issue and help you efficiently.

Dear fmbenhassine,

I meet same question with spring batch 4.3.2. The step configuration like bellow:

@Bean
    public Step expiryPolicyStep() {
        return stepBuilderFactory.get("expiryPolicyStep")
                .<PolicyContent, PolicyContent>chunk(1)
                .reader(expiryPolicyReader(null, null,null))
                .writer(expiryPolicyWriter(null, null))
                .faultTolerant()
                .skip(Exception.class)
                .skipLimit(4000)
                .noRetry(Exception.class)
                .taskExecutor(threadPoolTaskExecutor)
                .build();
    }

When an exception was throwed in writer, the FaultTolerantChunkProcessor would recall the writer again by scan method RecoveryCallback interface. I config the step with readerIsTransactionalQueue to make buffering flag to false. It can avoid repetitive execution. The code like bellow:

return stepBuilderFactory.get("expiryPolicyStep")
                .<PolicyContent, PolicyContent>chunk(1)
                .reader(expiryPolicyReader(null, null,null))
                .writer(expiryPolicyWriter(null, null))
                .readerIsTransactionalQueue()
                .faultTolerant()
                .skip(Exception.class)
                .skipLimit(4000)
                .noRetry(Exception.class)
                .taskExecutor(threadPoolTaskExecutor)
                .build();

Looking forward to fixing this issue as soon as possible

[chengwei710]

@fmbenhassine

[fmbenhassine]

Chunk scanning is activated when a skippable exception is thrown from the chunk processor. In that case, the chunk is resized to 1 and items are reprocessed (process + write) one by one, each one in its own transaction. So you might think items are being retried but it is not the case, this is part of the scanning process. @alappmeng I think this is your case (so it is normal to see the process method being re-executed). I suggest you to check the chunk scanning samples and try out various retry and skip policies.

@chengwei710 A skip is a recovery option for an exhausted retry. So setting .skip(Exception.class) and .noRetry(Exception.class) is actually incompatible. I believe this is similar to https://stackoverflow.com/questions/74482257 and https://stackoverflow.com/questions/61756484, which you might find useful.

[chengwei710]

Dear @fmbenhassine,

I don't quite understand this point. When chunk size of the job is 1, and got an exact exception in chunk, is it necessary to rescan items to retry. I just want my job to skip the exception without retried. If I config the job without noRetry, it would retry when an exception raised. So how can I config the job?