JobParameters deserialization fails [BATCH-2680]

[spring-projects-issues]

Madis Liias opened BATCH-2680 and commented

org.springframework.batch.core.step.job.JobStep#doExecute adds JobParameters into executionContext. This will be serialized into BATCH_STEP_EXECUTION_CONTEXT, but deserialization will fail if using the default org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.

Example unit test if using empty JobParameters:

  @Test
  public void testDeserializingJob() throws Exception {
    Jackson2ExecutionContextStringSerializer serializer = new Jackson2ExecutionContextStringSerializer();

    Map<String, Object> m = new HashMap<>();
    m.put("params", new JobParameters());

    ByteArrayOutputStream out = new ByteArrayOutputStream();
    serializer.serialize(m, out);
    String serialized = new String(out.toByteArray(), "ISO-8859-1");

    InputStream is = new ByteArrayInputStream(serialized.getBytes(StandardCharsets.UTF_8));
    Map<String, Object> deserialized = serializer.deserialize(is);
  }

com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "empty" (class org.springframework.batch.core.JobParameters), not marked as ignorable (one known property: "parameters"])
 at [Source: (ByteArrayInputStream); line: 1, column: 116] (through reference chain: java.util.HashMap["params"]->org.springframework.batch.core.JobParameters["empty"])

	at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:60)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:822)
	at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1152)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1567)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1545)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:374)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:116)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromAny(AsArrayTypeDeserializer.java:71)
	at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserializeWithType(UntypedObjectDeserializer.java:712)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:529)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3065)
	at org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.deserialize(Jackson2ExecutionContextStringSerializer.java:59)

Example unit test if using non-empty JobParameters:

  @Test
  public void testDes() throws Exception {
    Jackson2ExecutionContextStringSerializer serializer = new Jackson2ExecutionContextStringSerializer();

    Map<String, JobParameter> jobParametersMap = new HashMap<>();
    jobParametersMap.put("paramName", new JobParameter("paramValue"));

    Map<String, Object> m = new HashMap<>();
    m.put("params", new JobParameters(jobParametersMap));

    ByteArrayOutputStream out = new ByteArrayOutputStream();
    serializer.serialize(m, out);
    String serialized = new String(out.toByteArray(), "ISO-8859-1");

    InputStream is = new ByteArrayInputStream(serialized.getBytes(StandardCharsets.UTF_8));
    Map<String, Object> deserialized = serializer.deserialize(is);
  }

com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `org.springframework.batch.core.JobParameter` (although at least one Creator exists): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (ByteArrayInputStream); line: 1, column: 114] (through reference chain: java.util.HashMap["params"]->org.springframework.batch.core.JobParameters["parameters"]->java.util.LinkedHashMap["paramName"])

	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1342)
	at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1031)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1275)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:325)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:116)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromObject(AsArrayTypeDeserializer.java:61)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserializeWithType(MapDeserializer.java:400)
	at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:138)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:368)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:116)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromAny(AsArrayTypeDeserializer.java:71)
	at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserializeWithType(UntypedObjectDeserializer.java:712)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:529)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3065)
	at org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.deserialize(Jackson2ExecutionContextStringSerializer.java:59)

---

Affects: 4.0.0

Referenced from: pull request #580, and commits b0ffe55

1 votes, 4 watchers

[spring-projects-issues]

Michael Minella commented

Thanks for pointing this out. Just to confirm the behavior, based on the exception this is only happening when you have no job parameters, correct?

[spring-projects-issues]

Madis Liias commented

With non-empty job parameters it will fail with a different error. Added example unit test to issue description.

[spring-projects-issues]

Michael Minella commented

Ok. Again, thanks for pointing this out and for the sample test cases. We'll take a look.

[spring-projects-issues]

Anthony Foulfoin commented

Hi,

I'm facing a similar issue with a job that can be runs one time, but fails to be launched twice. With the following config and spring batch 4.0.0 :

@Configuration
public class BatchError {

    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Bean
    public Job myJob(Step myStep) {
        return jobBuilderFactory.get("myJob")
                                .incrementer(new RunIdIncrementer())
                                .start(myStep)
                                .build();
    }

    @Bean
    public Step myStep(Job nestedJob) {
        return this.stepBuilderFactory  .get("myStep")
                                        .job(nestedJob)
                                        .build();
    }

    @Bean
    public Job nestedJob(Step emptyStep) {
        return jobBuilderFactory.get("nestedJob")
                                .incrementer(new RunIdIncrementer())
                                .start(emptyStep)
                                .build();
    }

    @Bean
    public Step emptyStep(StepBuilderFactory stepBuilderFactory) {
        return stepBuilderFactory.get("emptyStep").tasklet(new EmptyTasklet()).build();
    }

    public class EmptyTasklet implements Tasklet {
        @Override
        public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
            return RepeatStatus.FINISHED;
        }
    }
}

The current date is used as job parameter:

public static void main(String[] args) {
   System.exit(SpringApplication.exit(SpringApplication.run(Main.class, new String[]{"date=" + new Date()})));
}

The batch runs well the first time, then fails silently (do nothing, and no error) the second time it is launched.
After some digging I found that an exception in thrown in the JdbcExecutionContextDao$ExecutionContextRowMapper class at the line 322 (map = serializer.deserialize(in)).

Here is the exception that is catched by the try-catch block:

com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `org.springframework.batch.core.JobParameter` (although at least one Creator exists): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (ByteArrayInputStream); line: 1, column: 165] (through reference chain: java.util.HashMap["org.springframework.batch.core.step.job.JobStep.JOB_PARAMETERS"]->org.springframework.batch.core.JobParameters["parameters"]->java.util.LinkedHashMap["date"])
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1342)
	at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1031)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1275)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:325)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:116)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromObject(AsArrayTypeDeserializer.java:61)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserializeWithType(MapDeserializer.java:400)
	at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:138)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:368)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:116)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromAny(AsArrayTypeDeserializer.java:71)
	at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserializeWithType(UntypedObjectDeserializer.java:712)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:529)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
	at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3065)
	at org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.deserialize(Jackson2ExecutionContextStringSerializer.java:59)
	at org.springframework.batch.core.repository.dao.Jackson2ExecutionContextStringSerializer.deserialize(Jackson2ExecutionContextStringSerializer.java:40)
	at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao$ExecutionContextRowMapper.mapRow(JdbcExecutionContextDao.java:322)
	at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao$ExecutionContextRowMapper.mapRow(JdbcExecutionContextDao.java:309)
	at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:93)
	at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:60)
	at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:667)
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:605)
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:657)
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:688)
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:700)
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:756)
	at org.springframework.batch.core.repository.dao.JdbcExecutionContextDao.getExecutionContext(JdbcExecutionContextDao.java:127)
	at org.springframework.batch.core.explore.support.SimpleJobExplorer.getStepExecutionDependencies(SimpleJobExplorer.java:208)
	at org.springframework.batch.core.explore.support.SimpleJobExplorer.getJobExecutions(SimpleJobExplorer.java:85)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
	at com.sun.proxy.$Proxy66.getJobExecutions(Unknown Source)
	at org.springframework.batch.core.JobParametersBuilder.getNextJobParameters(JobParametersBuilder.java:264)
	at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:162)
	at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179)
	at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134)
	at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:790)
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:774)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:335)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1246)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1234)
	at fr.cbm.altares.Main.main(Main.java:14)

And the context that failed to be deserialized:

{"org.springframework.batch.core.step.job.JobStep.JOB_PARAMETERS":["org.springframework.batch.core.JobParameters",{"parameters":["java.util.LinkedHashMap",{"date":{"identifying":true,"value":"Fri Feb 09 17:43:55 CET 2018","type":"STRING"},"run.id":{"identifying":true,"value":["java.lang.Long",1],"type":"LONG"}}],"empty":false}],"batch.stepType":"org.springframework.batch.core.step.job.JobStep"}

I hope these details will help !

[spring-projects-issues]

Mahmoud Ben Hassine commented

Thank you for pointing this out. Indeed, there are a couple of issues when serializing/deserializing job parameters:

1. First issue

Serializing an empty JobParameters instance results in:

{
  "params": [
    "org.springframework.batch.core.JobParameters",
    {
      "parameters": [
        "java.util.LinkedHashMap",
        {}
      ],
      "empty": true
    }
  ]
}

The "empty":true comes from the "isEmpty" getter. When Jackson tries to deserialize this String back to a JobParameters instance, it does not find a field named empty. Hence the issue.

2. Second issue

The problem seems to be related to a non empty JobParameters instance but actually can be narrowed to a single JobParameter. Serializing a JobParameter instance, for example new JobParameter("foo") results in:

{"parameter": "foo", "identifying": true, "value": "foo", "type": "STRING"}

When deserializing this String into a JobParameter instance, Jackson is not able to decide which constructor to use (Note there is no default constructor for a good reason). Hence the issue.

We are working on a fix which will be part of the 4.0.1 release.