Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Job Parameters are appended to same job when using incrementer [BATCH-2825] #789

Closed
spring-issuemaster opened this issue Jun 11, 2019 · 1 comment

Comments

@spring-issuemaster
Copy link
Collaborator

@spring-issuemaster spring-issuemaster commented Jun 11, 2019

Glenn Renfro opened BATCH-2825 and commented

When using an incrementer on a batch job that will be executed multiple times and for each execution the job keeps the job parameters from the previous examples.

For example:

Run1 (with job param foo=bar) will have a job param of foo=bar
Run2 (with job param foo=baz) will have a job param of foo=baz
Run 3 (with job param baz=goo) will have a job param of foo=baz, baz=goo.  Notice that foo=baz is still present even thought it was not passed in as a job param.

And we are jut using the RunIdIncrementer as shown below:

@Bean
public Job ingestJob() {
 return jobBuilderFactory.get("ingestJobMe")
    .incrementer(new RunIdIncrementer())
    .flow(step1())
    .end()
    .build();
}

The batch execution stops this additive behavior if you remove the incrementer.

 


Affects: 4.1.2

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Jun 17, 2019

Mahmoud Ben Hassine commented

Run1 (with job param foo=bar) will have a job param of foo=bar
Run2 (with job param foo=baz) will have a job param of foo=baz
Run 3 (with job param baz=goo) will have a job param of foo=baz, baz=goo. Notice that foo=baz is still present even thought it was not passed in as a job param.

This is actually not correct. Since you provided a RunIdIncrecementer, job parameters will be as follows (notice the additional run.id parameter):

Run1 (with job param foo=bar) will have a job param of run.id=1,foo=bar
Run2 (with job param foo=baz) will have a job param of run.id=2,foo=baz
Run 3 (with job param baz=goo) will have a job param of run.id=3, foo=baz, baz=goo. Notice that foo=baz is still present even though it was not passed in as a job param.

This works as designed. When you specify an incrementer, here is what happens:

  1. next job parameters are calculated from the previous ones using the incrementer (nextParameters = incrementer.getNext(previousExecution.getJobParameters());)
  2. existing parameters will be overridden with any new parameters having the same key
  3. new parameters will be appended to the previous ones

This is explained in the Javadoc of CommandLineJobRunner: "If the -next option is used the parameters on the command line (if any) are appended to those retrieved from the incrementer, overriding any with the same key.". Boot has been made consistent with this behaviour in BATCH-2711.

So in your example:

Run1:

  1. previous parameters: null
  2. new parameters: foo=bar
  3. next (merged) parameters: run.id=1,foo=bar

Run2:

  1. previous parameters: run.id=1,foo=bar
  2. new parameters: foo=baz
  3. next (merged) parameters: run.id=2,foo=baz

Run3:

  1. previous parameters: run.id=2,foo=baz
  2. new parameters: baz=goo
  3. next (merged) parameters: run.id=3,foo=baz,baz=goo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.