onSkipInProcess is not called if the exception is marked as no-rollback [BATCH-1383]

[spring-projects-issues]

Marcin Kuthan opened BATCH-1383 and commented

The method onSkipInProcess in the SkipListener is not called. My application logs the step execution after each step:

After: execution(execute) with: StepExecution: id=264, name=stage, status=COMPLETED, exitStatus=COMPLETED, readCount=28, filterCount=15, writeCount=13 readSkipCount=0, writeSkipCount=0, processSkipCount=15, commitCount=1, rollbackCount=0, exitDescription=

I expected that the onSkipInProcess method should be called 15 times, but it is not. The listener class is defined as follows:

public class CommandStepListener extends SkipListenerSupport<Command, Command> {
@Override
public void onSkipInProcess(Command item, Throwable t) {
// some skip logic
}
@BeforeStep
public void initialize(StepExecution stepExecution) {
// some initialization setup
}
}

The step definitions:

<batch:step id="abstractStep" abstract="true">
<batch:tasklet job-repository="jobRepository" transaction-manager="transactionManager" allow-start-if-complete="false">
</batch:tasklet>
</batch:step>

<batch:job id="myJob" restartable="true">
<batch:step id="stage" next="operation" parent="abstractStep">
batch:tasklet
<batch:chunk reader="commandFileReader" processor="commandProcessor" writer="commandWriter" commit-interval="100" skip-limit="1000000" retry-limit="10">
batch:skippable-exception-classesorg.springframework.batch.item.validator.ValidationException</batch:skippable-exception-classes>
batch:retryable-exception-classesjava.io.IOException</batch:retryable-exception-classes>
</batch:chunk>
batch:no-rollback-exception-classesorg.springframework.batch.item.validator.ValidationException</batch:no-rollback-exception-classes>
batch:listeners
<batch:listener ref="commandStepListener" />
</batch:listeners>
</batch:tasklet>
</batch:step>

and processor definition is:

<bean id="commandProcessor" scope="step" class="org.springframework.batch.item.validator.ValidatingItemProcessor">
<constructor-arg ref="commandValidator" />
</bean>

The commandValidator just throws the ValidationException if the processed command is invalid.

The listener bean is registered, the initialize method is called. When I added onSkipInRead method to the listener and simulate error during command reading, the method was called. It seems that my setup is fine, but onSkipInProcess is not called :-(

I also tried switch to pure annotation configuration with @OnSkipInProcess. The method is not called, also.

---

Affects: 2.0.2

[spring-projects-issues]

Dave Syer commented

If you remove the @BeforeStep is it called?

[spring-projects-issues]

Marcin Kuthan commented

When I remove @BeforeStep the listener is not initialized anymore (default constructor is not called). Interesting ...

[spring-projects-issues]

Dave Syer commented

There is no default contructor in the snippet you posted, but if you create a bean definition with that class then Spring will initialize it when it is created. If it is step scoped (I guess this one is, but you didn't say) then that only happens when one of its methods is called. I guess that would explain the lack of initialization (no method was called including onSkipInProcess(). I'm not sure why that is though. Your processor definitely throws an exception?

[spring-projects-issues]

Marcin Kuthan commented

The listener is defined within step scope.

My processor throws an exception, the framework recognized it (processSkipCount=15):
After: execution(execute) with: StepExecution: id=264, name=stage, status=COMPLETED, exitStatus=COMPLETED, readCount=28, filterCount=15, writeCount=13 readSkipCount=0, writeSkipCount=0, processSkipCount=15, commitCount=1, rollbackCount=0, exitDescription=

For me it looks that:

1. When the exception is thrown during processing.
2. The SkipListener is not called.
3. Because of step scope the listener is not initialized even.

I tried to find out the callers of onSkipInProcess. The only place is FaultTolerantChunkProcessor.callSkipListeners() method. The parameter inputs.getSkip() is empty collection (why?) so the skipInProcess is not called.

As a workaround I use ItemProcessListener and implement the skip logic in the onProcessError method.

[spring-projects-issues]

Dave Syer commented

I tidied up the skip sample and added a listener, and you can see from the logs that it is called on a skip in processing, but only if the exception is not marked as no-rollback. That was an easy fix in FaultTolerantChunkProcessor.