Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conditional (Java configured) steps cannot be scoped [BATCH-2747] #857

Open
spring-issuemaster opened this issue Aug 26, 2018 · 15 comments
Open
Assignees
Milestone

Comments

@spring-issuemaster
Copy link
Collaborator

@spring-issuemaster spring-issuemaster commented Aug 26, 2018

Mattias Jiderhamn opened BATCH-2747 and commented

Configuring conditional steps using Java config (XML untested) fails if any of the steps is scoped using Job scope or Step scope.

To reproduce, use the example at https://docs.spring.io/spring-batch/4.0.x/reference/html/step.html#controllingStepFlow, i.e.

@Bean
public Job job() {
        return this.jobBuilderFactory.get("job")
                                .start(stepA())
                                .on("*").to(stepB())
                                .from(stepA()).on("FAILED").to(stepC())
                                .end()
                                .build();
}

and then add @JobScope or @StepScope on stepA(), stepB() or stepC().

The underlying reason is that to configure a flow step, the framework wants to get the steps name using getName(), at which point the bean is resolved and the scope inferred.

Stack trace (4.0.1)

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'myJob' defined in se.jiderhamn.ScopeTestConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.batch.core.Job]: Factory method 'parseCallLogJob' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.stepA': Scope 'job' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for job scope
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:583)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1249)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1098)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:502)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:312)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:310)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:756)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:868)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
	at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:128)
	at org.springframework.test.context.support.AbstractGenericContextLoader.loadContext(AbstractGenericContextLoader.java:60)
	at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.delegateLoading(AbstractDelegatingSmartContextLoader.java:109)
	at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.loadContext(AbstractDelegatingSmartContextLoader.java:246)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99)
	at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:117)
	... 24 more
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.batch.core.Job]: Factory method 'parseCallLogJob' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.stepA': Scope 'job' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for job scope
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:186)
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:575)
	... 41 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.stepA': Scope 'job' is not active for the current thread; consider defining a scoped proxy for this bean if you intend to refer to it from a singleton; nested exception is java.lang.IllegalStateException: No context holder available for job scope
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:357)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
	at org.springframework.aop.target.SimpleBeanTargetSource.getTarget(SimpleBeanTargetSource.java:35)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:193)
	at com.sun.proxy.$Proxy32.getName(Unknown Source)
	at org.springframework.batch.core.job.builder.FlowBuilder.createState(FlowBuilder.java:282)
	at org.springframework.batch.core.job.builder.FlowBuilder.doStart(FlowBuilder.java:265)
	at org.springframework.batch.core.job.builder.FlowBuilder.start(FlowBuilder.java:122)
	at org.springframework.batch.core.job.builder.JobFlowBuilder.<init>(JobFlowBuilder.java:39)
	at org.springframework.batch.core.job.builder.SimpleJobBuilder.on(SimpleJobBuilder.java:91)
	at se.jiderhamn.ScopeTestConfiguration.parseCallLogJob(ScopeTestConfiguration.java:36)
	at se.jiderhamn.ScopeTestConfiguration$$EnhancerBySpringCGLIB$$240ba886.CGLIB$parseCallLogJob$3(<generated>)
	at se.jiderhamn.ScopeTestConfiguration$$EnhancerBySpringCGLIB$$240ba886$$FastClassBySpringCGLIB$$faf3ccf4.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:361)
	at se.jiderhamn.ScopeTestConfiguration$$EnhancerBySpringCGLIB$$240ba886.parseCallLogJob(<generated>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:155)
	... 42 more
Caused by: java.lang.IllegalStateException: No context holder available for job scope
	at org.springframework.batch.core.scope.JobScope.getContext(JobScope.java:159)
	at org.springframework.batch.core.scope.JobScope.get(JobScope.java:92)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:345)
	... 62 more

Affects: 4.0.1, 4.1.0.M2

Reference URL: https://stackoverflow.com/questions/55571210/

2 votes, 4 watchers

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Aug 30, 2018

Michael Minella commented

What is the use case where the beans must be scoped?

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Sep 1, 2018

Mattias Jiderhamn commented

Michael Minella, the most obvious use case would be when JobParameters are used in the step definition

@Bean
@JobScope
Step readDataFromFile(@Value("#{jobParameters[filePath]}") String filePath) {
...
}

 

There does however seem to be at least one workaround, by separately defining the ItemReader/Processor/Writer as @JobScope:d beans like so

@Bean
@JobScope
ItemReader reader(@Value("#{jobParameters[filePath]}") String filePath) {
...
}

@Bean(name = "stepA")
Step readDataFromFile(ItemReader reader) {
...
}

@Bean
Job myJob(@Qualifier("stepA") stepA) {
        return this.jobBuilderFactory.get("job")
                                .start(stepA)
                                .on("*").to(stepB())
                                .from(stepA).on("FAILED").to(stepC())
                                .end()
                                .build();
}
@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 28, 2018

ArtyomGabeev commented

Hi,

I think issue is related to FlowBuilder - it's impossible to use scoped steps/flows inside any other flow.

Here is a small example:

@Configuration
public class JobWithFlowConfig {

    @Bean
    public Job job(JobBuilderFactory jobBuilderFactory, Flow flow) {
        return jobBuilderFactory.get("job")
                .start(flow)
                .end()
                .build();
    }

    @Bean
    public Flow flow(Step step) {
        return new FlowBuilder<Flow>("flow")
                .from(step)
                .end();
    }

    @Bean
    @JobScope
    public Step step(StepBuilderFactory stepBuilderFactory, @Value("#{jobParameters['test']}") String test) {
        return stepBuilderFactory.get("step")
                .tasklet(((contribution, chunkContext) -> {
                    System.out.println(test);
                    return RepeatStatus.FINISHED;
                }))
                .allowStartIfComplete(true)
                .build();
    }

}

 

Issue happens in FlowBuilder.createState - calling getName on a step/flow causes lazy proxy initialization, but there is no Job/Step scope at this time.

In order to fix it, I suggest reuse logic of conters naming for step/flow:

...
private int stepCounter = 0;

private int flowCounter = 0;
...

private State createState(Object input) {
...
states.put(input, new StepState(prefix + "step" + (stepCounter++), step)); //remove getName call
...
states.put(input, new FlowState((Flow) input, prefix + "flow" + (flowCounter++))); //same for the flow
}

Thanks,
Artyom

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 28, 2018

ArtyomGabeev commented

Regarding original issue: calling on over jobBuilder, simply converts previous steps into FlowBuilder, that explains the behaviour.
It can be observed though the stack trace:

at com.sun.proxy.$Proxy32.getName(Unknown Source)
	at org.springframework.batch.core.job.builder.FlowBuilder.createState(FlowBuilder.java:282)
	at org.springframework.batch.core.job.builder.FlowBuilder.doStart(FlowBuilder.java:265)
	at org.springframework.batch.core.job.builder.FlowBuilder.start(FlowBuilder.java:122)
	at org.springframework.batch.core.job.builder.JobFlowBuilder.<init>(JobFlowBuilder.java:39)
	at org.springframework.batch.core.job.builder.SimpleJobBuilder.on(SimpleJobBuilder.java:91)
@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 28, 2018

Michael Minella commented

This still doesn't make sense to me.  You need to be in the context of a step in order to to use step scope.  What you are trying to do is causing premature initialization.  I don't see this as a bug but works as designed.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 28, 2018

ArtyomGabeev commented

What about JobScope?

In general step can be annotated with @JobScope in order to have access to jobParameters.

 

Michael Minella, you right, it was my mistake when I mentioned StepScope in this issue, because it doesn't make sense.

But this fix is still reasonable if you want to create a JobScoped step, and pass some parameters (like chunk size for example)

 

BTW: original author has an issue with JobScoped step, according to stack trace.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 28, 2018

Michael Minella commented

This really also works as designed.  There must be a running job in order for the proxy to be resolved.  The steps are created before the job is executed so the idea that you can use job scope on a step still doesn't apply.  I'd argue that we may need to be more specific around our documentation, but you do not have a running job when a step is configured so there is no way for job scope to work on the configuration of a step as you are attempting to do.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 28, 2018

ArtyomGabeev commented

Thanks, now I see your point. My initial understanding was that JobScope can be applied on anything except Job.

So there is no way to bind jobParameters to step configuration, so let's say I can not adjust step chunk size based on jobParameters? 

Another use case is to configure gridSize based on jobParameters.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 29, 2018

ArtyomGabeev commented

Also, it's possible to configure JobScope'd step, if there is no flow:

@Configuration
public class JobConfig {

    @Bean
    public Job job(JobBuilderFactory jobBuilderFactory, Step step) {
        return jobBuilderFactory.get("job")
                .start(step)
                .build();
    }
    
    @Bean
    @JobScope
    public Step step(StepBuilderFactory stepBuilderFactory, @Value("#{jobParameters['test']}") String test) {
        return stepBuilderFactory.get("step")
                .tasklet(((contribution, chunkContext) -> {
                    System.out.println(test);
                    return RepeatStatus.FINISHED;
                }))
                .allowStartIfComplete(true)
                .build();
    }

} 

This configuration works fine. Also I want to try same two configurations in XML to confirm behaviour.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Nov 30, 2018

ArtyomGabeev commented

Michael Minella

Finally, I agree with that step can not be scoped, when I get to the partitioned steps (there is no job scope).

Probably documentation should be more specific in terms what can be proxied.

Thank you for clarification.

 

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Dec 15, 2018

Mattias Jiderhamn commented

Sorry for being late back to the party.

Even though it may "work as designed", at least personally I think that design is very far from intuitive and definitely could use clarifying in the documentation. I spent quite some time figuring out a working configuration using trial and error.

 

The steps are created before the job is executed so the idea that you can use job scope on a step still doesn't apply.

+Despite having read the documentation+ this really doesn't make sense to me. If you create a job scoped or step scoped bean - such as a reader, as in my example from September 1st - you may use that bean when configuring your step. But you may not make the step itself job scoped nor step scoped - am I right? This seems like saying steps may be implicitly job scoped or step scoped (due to the dependency on the scoped bean), but not explicitly. That is - with the current documentation - counter-intuitive. I mean, what are the job and step scopes for, if not for being used in the configuration???

 

Here is a variation of my September 1st example. The fact that it also works for me only adds to the confusion.

@Bean
@JobScope
ItemReader reader(@Value("#{jobParameters[filePath]}") String filePath) {
...
}

@Bean
Step readDataFromFile(ItemReader reader) {
  return steps.get("readDataFromFileStep")
        .<X, Y>chunk(100)
        .reader(reader("Overridden by expression"))
        ...
}

(Sorry for spamming you with updates fixing Jira copy/paste issues)

@wabrit
Copy link

@wabrit wabrit commented Apr 3, 2020

My apologies for asking what may be a dumb question.

I hit this issue today by using a combination of Java-configured conditional flow with some steps that have @StepScope (none have @JobScope).

I'm not quite understanding the current conclusion of the above thread - is that something that "should work", or is it "by design"?

If the latter, is there a recommended workaround if I need my @StepScope's but also want conditional flows?

@mminella
Copy link
Member

@mminella mminella commented Apr 6, 2020

@wabrit Please ask these kind of questions via StackOverflow and tag them with the spring-batch tag. We monitor that tag and can answer questions there. Thanks!

@wabrit
Copy link

@wabrit wabrit commented Apr 6, 2020

@wabrit Please ask these kind of questions via StackOverflow and tag them with the spring-batch tag. We monitor that tag and can answer questions there. Thanks!

Thanks @mminella - regardless of the status/scope of this particular issue (which I found unclear from reading the trail above) if it's of any use to you I can confirm that:

  • Annotating StepScope on a Step bean causes the error if you use conditional flows.
  • Pushing StepScope "down" into a non-Step bean (e.g. an ItemReader, ItemWriter, ItemProcessor etc.) makes the problem go away.
@mminella
Copy link
Member

@mminella mminella commented Apr 6, 2020

@wabrit Those are as expected. We will be updating the documentation to make that more clear. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.