Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JobParameters deserialization fails [BATCH-2680] #927

Closed
spring-issuemaster opened this issue Feb 7, 2018 · 5 comments
Closed

JobParameters deserialization fails [BATCH-2680] #927

spring-issuemaster opened this issue Feb 7, 2018 · 5 comments
Milestone

Comments

@spring-issuemaster
Copy link
Collaborator

@spring-issuemaster spring-issuemaster commented Feb 7, 2018

Madis Liias opened BATCH-2680 and commented

org.springframework.batch.core.step.job.JobStep#doExecute adds JobParameters into executionContext. This will be serialized into BATCH_STEP_EXECUTION_CONTEXT, but deserialization will fail if using the default org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.

Example unit test if using empty JobParameters:

  @Test
  public void testDeserializingJob() throws Exception {
    Jackson2ExecutionContextStringSerializer serializer = new Jackson2ExecutionContextStringSerializer();

    Map<String, Object> m = new HashMap<>();
    m.put("params", new JobParameters());

    ByteArrayOutputStream out = new ByteArrayOutputStream();
    serializer.serialize(m, out);
    String serialized = new String(out.toByteArray(), "ISO-8859-1");

    InputStream is = new ByteArrayInputStream(serialized.getBytes(StandardCharsets.UTF_8));
    Map<String, Object> deserialized = serializer.deserialize(is);
  }
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "empty" (class org.springframework.batch.core.JobParameters), not marked as ignorable (one known property: "parameters"])
 at [Source: (ByteArrayInputStream); line: 1, column: 116] (through reference chain: java.util.HashMap["params"]->org.springframework.batch.core.JobParameters["empty"])

	at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:60)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:822)
	at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1152)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1567)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1545)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:374)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:116)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromAny(AsArrayTypeDeserializer.java:71)
	at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserializeWithType(UntypedObjectDeserializer.java:712)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:529)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3065)
	at org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.deserialize(Jackson2ExecutionContextStringSerializer.java:59)

Example unit test if using non-empty JobParameters:

  @Test
  public void testDes() throws Exception {
    Jackson2ExecutionContextStringSerializer serializer = new Jackson2ExecutionContextStringSerializer();

    Map<String, JobParameter> jobParametersMap = new HashMap<>();
    jobParametersMap.put("paramName", new JobParameter("paramValue"));

    Map<String, Object> m = new HashMap<>();
    m.put("params", new JobParameters(jobParametersMap));

    ByteArrayOutputStream out = new ByteArrayOutputStream();
    serializer.serialize(m, out);
    String serialized = new String(out.toByteArray(), "ISO-8859-1");

    InputStream is = new ByteArrayInputStream(serialized.getBytes(StandardCharsets.UTF_8));
    Map<String, Object> deserialized = serializer.deserialize(is);
  }
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `org.springframework.batch.core.JobParameter` (although at least one Creator exists): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (ByteArrayInputStream); line: 1, column: 114] (through reference chain: java.util.HashMap["params"]->org.springframework.batch.core.JobParameters["parameters"]->java.util.LinkedHashMap["paramName"])

	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1342)
	at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1031)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1275)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:325)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:116)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromObject(AsArrayTypeDeserializer.java:61)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserializeWithType(MapDeserializer.java:400)
	at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:138)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:368)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:116)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromAny(AsArrayTypeDeserializer.java:71)
	at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserializeWithType(UntypedObjectDeserializer.java:712)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:529)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3065)
	at org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.deserialize(Jackson2ExecutionContextStringSerializer.java:59)

Affects: 4.0.0

Referenced from: pull request #580, and commits b0ffe55

1 votes, 4 watchers

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Feb 7, 2018

Michael Minella commented

Thanks for pointing this out. Just to confirm the behavior, based on the exception this is only happening when you have no job parameters, correct?

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Feb 7, 2018

Madis Liias commented

With non-empty job parameters it will fail with a different error. Added example unit test to issue description.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Feb 7, 2018

Michael Minella commented

Ok. Again, thanks for pointing this out and for the sample test cases. We'll take a look.

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Feb 9, 2018

Anthony Foulfoin commented

Hi,

I'm facing a similar issue with a job that can be runs one time, but fails to be launched twice. With the following config and spring batch 4.0.0 :

@Configuration
public class BatchError {

    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Bean
    public Job myJob(Step myStep) {
        return jobBuilderFactory.get("myJob")
                                .incrementer(new RunIdIncrementer())
                                .start(myStep)
                                .build();
    }

    @Bean
    public Step myStep(Job nestedJob) {
        return this.stepBuilderFactory  .get("myStep")
                                        .job(nestedJob)
                                        .build();
    }

    @Bean
    public Job nestedJob(Step emptyStep) {
        return jobBuilderFactory.get("nestedJob")
                                .incrementer(new RunIdIncrementer())
                                .start(emptyStep)
                                .build();
    }

    @Bean
    public Step emptyStep(StepBuilderFactory stepBuilderFactory) {
        return stepBuilderFactory.get("emptyStep").tasklet(new EmptyTasklet()).build();
    }

    public class EmptyTasklet implements Tasklet {
        @Override
        public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
            return RepeatStatus.FINISHED;
        }
    }
}

The current date is used as job parameter:

public static void main(String[] args) {
   System.exit(SpringApplication.exit(SpringApplication.run(Main.class, new String[]{"date=" + new Date()})));
}

The batch runs well the first time, then fails silently (do nothing, and no error) the second time it is launched.
After some digging I found that an exception in thrown in the JdbcExecutionContextDao$ExecutionContextRowMapper class at the line 322 (map = serializer.deserialize(in)).

Here is the exception that is catched by the try-catch block:

com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `org.springframework.batch.core.JobParameter` (although at least one Creator exists): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (ByteArrayInputStream); line: 1, column: 165] (through reference chain: java.util.HashMap["org.springframework.batch.core.step.job.JobStep.JOB_PARAMETERS"]->org.springframework.batch.core.JobParameters["parameters"]->java.util.LinkedHashMap["date"])
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1342)
	at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1031)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1275)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:325)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:116)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromObject(AsArrayTypeDeserializer.java:61)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserializeWithType(MapDeserializer.java:400)
	at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:138)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:368)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:116)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromAny(AsArrayTypeDeserializer.java:71)
	at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserializeWithType(UntypedObjectDeserializer.java:712)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:529)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3065)
	at org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.deserialize(Jackson2ExecutionContextStringSerializer.java:59)
	at org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.deserialize(Jackson2ExecutionContextStringSerializer.java:40)
	at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao$ExecutionContextRowMapper.mapRow(JdbcExecutionContextDao.java:322)
	at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao$ExecutionContextRowMapper.mapRow(JdbcExecutionContextDao.java:309)
	at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:93)
	at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:60)
	at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:667)
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:605)
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:657)
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:688)
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:700)
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:756)
	at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.getExecutionContext(JdbcExecutionContextDao.java:127)
	at org.springframework.batch.core.explore.support.SimpleJobExplorer.getStepExecutionDependencies(SimpleJobExplorer.java:208)
	at org.springframework.batch.core.explore.support.SimpleJobExplorer.getJobExecutions(SimpleJobExplorer.java:85)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
	at com.sun.proxy.$Proxy66.getJobExecutions(Unknown Source)
	at org.springframework.batch.core.JobParametersBuilder.getNextJobParameters(JobParametersBuilder.java:264)
	at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:162)
	at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179)
	at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134)
	at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:790)
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:774)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:335)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1246)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1234)
	at fr.cbm.altares.Main.main(Main.java:14)

And the context that failed to be deserialized:

{"org.springframework.batch.core.step.job.JobStep.JOB_PARAMETERS":["org.springframework.batch.core.JobParameters",{"parameters":["java.util.LinkedHashMap",{"date":{"identifying":true,"value":"Fri Feb 09 17:43:55 CET 2018","type":"STRING"},"run.id":{"identifying":true,"value":["java.lang.Long",1],"type":"LONG"}}],"empty":false}],"batch.stepType":"org.springframework.batch.core.step.job.JobStep"}

I hope these details will help !

@spring-issuemaster
Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Feb 14, 2018

Mahmoud Ben Hassine commented

Thank you for pointing this out. Indeed, there are a couple of issues when serializing/deserializing job parameters:

1. First issue

Serializing an empty JobParameters instance results in:

{
  "params": [
    "org.springframework.batch.core.JobParameters",
    {
      "parameters": [
        "java.util.LinkedHashMap",
        {}
      ],
      "empty": true
    }
  ]
}

The "empty":true comes from the "isEmpty" getter. When Jackson tries to deserialize this String back to a JobParameters instance, it does not find a field named empty. Hence the issue.

2. Second issue

The problem seems to be related to a non empty JobParameters instance but actually can be narrowed to a single JobParameter. Serializing a JobParameter instance, for example new JobParameter("foo") results in:

{"parameter": "foo", "identifying": true, "value": "foo", "type": "STRING"}

When deserializing this String into a JobParameter instance, Jackson is not able to decide which constructor to use (Note there is no default constructor for a good reason). Hence the issue.

We are working on a fix which will be part of the 4.0.1 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.