AWS: AWSTypesMessageConverter fails to deserialize JSON array as body for functions that take reactive inputs of POJOs

[ArnauAregall]

Describe the bug

Using:

• Spring Boot 3.1.1
• Spring Cloud 2022.0.3 (SCF 4.0.3)

Related to org.springframework.cloud:spring-cloud-function-adapter-aws:

When running a Spring Cloud Function application on an AWS Lambda, AWSTypesMessageConverter seems to not be capable to correctly convert a JSON array body when the functions consume a Flux<T>, where T is a regular application domain class/POJO.

The domain class:

public record GeoLocation(Float latitude, Float longitude) {}

The function definition:

@Configuration
public class WeatherFunctions {

    @Bean
    public Function<Flux<GeoLocation>, Flux<Forecast>> forecast(WeatherService weatherService) {
        return weatherService::getForecast;
    }

}

When invoking the AWS Lambda function with a body such as

[
  {
    "latitude": 42.99,
    "longitude": 2.83
  }
]

It fails with the following Jackson conversion error:

2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 2023-06-25T14:07:34.526Z  WARN 9 --- [pool-5-thread-1] s.c.f.c.c.SmartCompositeMessageConverter : Failure during type conversion by org.springframework.cloud.function.adapter.aws.AWSTypesMessageConverter@7e33992. Will try the next converter.
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 java.lang.IllegalStateException: Failed to convert. Possible bug as the conversion probably shouldn't have been attempted here
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.springframework.cloud.function.json.JacksonMapper.doFromJson(JacksonMapper.java:65) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.springframework.cloud.function.json.JsonMapper.fromJson(JsonMapper.java:66) ~[aws-lambda-spring-cloud-function-reactive-java:4.0.3]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.springframework.cloud.function.adapter.aws.AWSTypesMessageConverter.convertFromInternal(AWSTypesMessageConverter.java:90) ~[aws-lambda-spring-cloud-function-reactive-java:4.0.3]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.springframework.messaging.converter.AbstractMessageConverter.fromMessage(AbstractMessageConverter.java:185) ~[aws-lambda-spring-cloud-function-reactive-java:6.0.10]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.springframework.messaging.converter.AbstractMessageConverter.fromMessage(AbstractMessageConverter.java:176) ~[aws-lambda-spring-cloud-function-reactive-java:6.0.10]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.springframework.cloud.function.context.config.SmartCompositeMessageConverter.fromMessage(SmartCompositeMessageConverter.java:63) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.convertInputMessageIfNecessary(SimpleFunctionRegistry.java:1353) ~[aws-lambda-spring-cloud-function-reactive-java:4.0.3]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.convertInputIfNecessary(SimpleFunctionRegistry.java:1106) ~[aws-lambda-spring-cloud-function-reactive-java:4.0.3]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.springframework.cloud.function.context.catalog.SimpleFunctionRegistry$FunctionInvocationWrapper.lambda$convertInputPublisherIfNecessary$24(SimpleFunctionRegistry.java:1465) ~[aws-lambda-spring-cloud-function-reactive-java:4.0.3]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2545) ~[aws-lambda-spring-cloud-function-reactive-java:3.5.7]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:171) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at reactor.core.publisher.FluxFlatMap$FlatMapMain.onSubscribe(FluxFlatMap.java:371) ~[aws-lambda-spring-cloud-function-reactive-java:3.5.7]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:96) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at reactor.core.publisher.FluxJust.subscribe(FluxJust.java:68) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:62) ~[aws-lambda-spring-cloud-function-reactive-java:3.5.7]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at reactor.core.publisher.BlockingIterable.iterator(BlockingIterable.java:86) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.springframework.cloud.function.adapter.aws.AWSLambdaUtils.generateOutputFromObject(AWSLambdaUtils.java:153) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.springframework.cloud.function.adapter.aws.CustomRuntimeEventLoop.eventLoop(CustomRuntimeEventLoop.java:166) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.springframework.cloud.function.adapter.aws.CustomRuntimeEventLoop.lambda$run$0(CustomRuntimeEventLoop.java:90) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at java.base@17.0.5/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[aws-lambda-spring-cloud-function-reactive-java:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at java.base@17.0.5/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at java.base@17.0.5/java.lang.Thread.run(Thread.java:833) ~[aws-lambda-spring-cloud-function-reactive-java:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:775) ~[aws-lambda-spring-cloud-function-reactive-java:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:203) ~[na:na]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `tech.aaregall.lab.function.weather.domain.GeoLocation` from Array value (token `JsonToken.START_ARRAY`)
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at [Source: (String)"[
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 {
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 "latitude": 42.99,
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 "longitude": 2.83
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 }
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 ]"; line: 1, column: 1]
2023/06/25/[$LATEST]2b2c53a6597a485481146cabafd1247e 2023-06-25T14:07:34.527000 at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1752) ~[aws-lambda-spring-cloud-function-reactive-java:2.15.2]

Nevertheless it works perfectly when the application is not running on AWS (locally, or on a Docker Image) being able to take more "geolocations" in the payload, which is the intention.

[
  {
    "latitude": 42.99,
    "longitude": 2.83
  },
  {
    "latitude": 43.01,
    "longitude": 2.21
  }
]

Sample

I built a sample application to demonstrate the issue: https://github.com/ArnauAregall/aws-lambda-spring-cloud-function-reactive-java

If you clone and run the application locally:

$ ./gradlew nativeRun

$ curl -X POST -H "Content-Type: application/json" -d '[{"latitude": 41.34, "longitude": 2.78}]' http://localhost:8080/question | jq > response.json

The response.json file will be a JSON array containing a single object representing the weather forecast for those coordinates.

But when running in AWS:

$ ./aws-image/build-aws-image.sh

$ sam build --use-container --build-image tech.aaregall.lab/amazonlinux-graalvm:latest

$ export AWS_REGION=...

$ sam deploy --region $AWS_REGION

The same command against the lambda URL will result in "Internal Server Error" with the mentioned stacktrace above.

[ArnauAregall]

Adding a bit more info if it helps:

The error is also somehow reproducible when serializing the return type POJOs.

Managed to reproduce it by invoking the lambda using aws lambda invoke cli command.

aws lambda invoke   \
  --function-name aws-lambda-spring-cloud-f-SpringCloudFunctionLambd-.... \
      --cli-binary-format raw-in-base64-out  \
          --payload '[{"latitude": 42.99, "longitude": 2.83 }, {"latitude": 43.01, "longitude": 2.21 } ]' output.txt

Response:

{
    "StatusCode": 200,
    "FunctionError": "Unhandled",
    "ExecutedVersion": "$LATEST"
}

output.txt:

{
    "errorType": "IllegalArgumentException",
    "errorMessage": "Payload must not be null",
    "stackTrace": "java.lang.IllegalArgumentException: Payload must not be null\n\tat org.springframework.util.Assert.notNull(Assert.java:204)\n\tat org.springframework.messaging.support.MessageBuilder.<init>(MessageBuilder.java:58)\n\tat org.springframework.messaging.support.MessageBuilder.withPayload(MessageBuilder.java:190)\n\tat org.springframework.cloud.function.adapter.aws.AWSLambdaUtils.generateOutputFromObject(AWSLambdaUtils.java:176)\n\tat org.springframework.cloud.function.adapter.aws.CustomRuntimeEventLoop.eventLoop(CustomRuntimeEventLoop.java:166)\n\tat org.springframework.cloud.function.adapter.aws.CustomRuntimeEventLoop.lambda$run$0(CustomRuntimeEventLoop.java:90)\n\tat java.base@17.0.5/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\tat java.base@17.0.5/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\tat java.base@17.0.5/java.lang.Thread.run(Thread.java:833)\n\tat org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:775)\n\tat org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:203)\n"
}

Logs:

2023-06-25T19:00:36.111Z DEBUG 9 --- [pool-5-thread-1] c.f.c.c.BeanFactoryAwareFunctionRegistry : Invoking function forecast<reactor.core.publisher.Flux<tech.aaregall.lab.function.weather.domain.GeoLocation>, reactor.core.publisher.Flux<tech.aaregall.lab.function.weather.domain.Forecast>>
2023-06-25T19:00:36.111000 2023-06-25T19:00:36.111Z DEBUG 9 --- [pool-5-thread-1] c.f.c.c.BeanFactoryAwareFunctionRegistry : Converting JSON string representing collection to a list of Messages. Function 'forecast<reactor.core.publisher.Flux<tech.aaregall.lab.function.weather.domain.GeoLocation>, reactor.core.publisher.Flux<tech.aaregall.lab.function.weather.domain.Forecast>>' will be invoked iteratively
2023-06-25T19:00:36.112000 2023-06-25T19:00:36.112Z DEBUG 9 --- [pool-5-thread-1] c.f.c.c.BeanFactoryAwareFunctionRegistry : Invoking function: forecast<reactor.core.publisher.Flux<tech.aaregall.lab.function.weather.domain.GeoLocation>, reactor.core.publisher.Flux<tech.aaregall.lab.function.weather.domain.Forecast>>with input type: reactor.core.publisher.Flux<tech.aaregall.lab.function.weather.domain.GeoLocation>
2023-06-25T19:00:36.112000 2023-06-25T19:00:36.112Z DEBUG 9 --- [pool-5-thread-1] o.s.c.f.a.aws.CustomRuntimeEventLoop     : Reply from function: FluxMap
2023-06-25T19:00:36.112000 2023-06-25T19:00:36.112Z DEBUG 9 --- [pool-5-thread-1] c.f.c.c.BeanFactoryAwareFunctionRegistry : Converted Message: GenericMessage [payload={latitude=42.99, longitude=2.83}, headers={id=540951aa-c67b-22e2-56a7-6ed76137007a, timestamp=1687719636111}] to: class tech.aaregall.lab.function.weather.domain.GeoLocation
2023-06-25T19:00:36.112000 2023-06-25T19:00:36.112Z DEBUG 9 --- [pool-5-thread-1] c.f.c.c.BeanFactoryAwareFunctionRegistry : Converted Message: GenericMessage [payload={latitude=43.01, longitude=2.21}, headers={id=ab6b39c9-f56f-5d19-2512-22fbca5f7820, timestamp=1687719636111}] to: class tech.aaregall.lab.function.weather.domain.GeoLocation
2023-06-25T19:00:36.145000 2023-06-25T19:00:36.145Z DEBUG 9 --- [pool-5-thread-1] o.s.c.f.adapter.aws.AWSLambdaUtils       : Response value: GenericMessage [payload=byte[11831], headers={contentType=application/json, id=7025cb06-5e20-7770-6884-90ca27b037ae, timestamp=1687719636145}]
2023-06-25T19:00:36.148000 2023-06-25T19:00:36.148Z DEBUG 9 --- [pool-5-thread-1] o.s.c.f.adapter.aws.AWSLambdaUtils       : Response value: GenericMessage [payload=byte[11835], headers={contentType=application/json, id=0c1ca3d6-8c58-438e-d5a6-c0b8cdc8ddda, timestamp=1687719636148}]
2023-06-25T19:00:36.148000 2023-06-25T19:00:36.148Z DEBUG 9 --- [pool-5-thread-1] o.s.c.f.adapter.aws.AWSLambdaUtils       : OUTPUT: [GenericMessage [payload=byte[11831], headers={contentType=application/json, id=7025cb06-5e20-7770-6884-90ca27b037ae, timestamp=1687719636145}], GenericMessage [payload=byte[11835], headers={contentType=application/json, id=0c1ca3d6-8c58-438e-d5a6-c0b8cdc8ddda, timestamp=1687719636148}]] - java.util.ArrayList
2023-06-25T19:00:36.151000 2023-06-25T19:00:36.150Z  INFO 9 --- [pool-5-thread-1] o.s.c.f.a.aws.CustomRuntimeEventLoop     : Result ERROR status: 202 ACCEPTED

Reading the logs it can be appreciated that the two JSON objects in the array of the payload were successfully converted to GeoLocation POJOs.

But apparently it also failed on when converting the return type: org.springframework.cloud.function.adapter.aws.AWSLambdaUtils.generateOutputFromObject(AWSLambdaUtils.java:176)

OUTPUT ... log produced by AWSLambdaUtils also indicates the return class is an java.uti.ArrayList whereas the function is defined with a return type of Flux<Forecast>.

[olegz]

Was able to reproduce with your sample, so thank you. Will let you know once I figure out the issue

[olegz]

I thought it may have something to do with native hints, but just reproduced it locally, by sending a JSON representing an actual API Gateway request, so will have a test case for this.

[ArnauAregall]

That's actually really good news, thanks @olegz!

[olegz]

I believe I got it. I am assuming this is what you are expecting..?

curl -X POST -H "Content-Type: application/json" -d '[{"latitude": 41.34, "longitude": 2.78}]' https://emcdxu5ijj.execute-api.us-east-2.amazonaws.com/question
..
{"geoLocation":{"latitude":41.34,"longitude":2.7799997},"hourlyForecasts":[{"time":"2023-07-12T00:00:00","temperature":24.7,"precipitation":0.0},{"time":"2023-07-12T01:00:00","temperature":24.4,"precipitation":0.0},{"time":"2023-07-12T02:00:00","temperature":24.5,"precipitation":0.0},{"time":"2023-07-12T03:00:00","temperature":24.5,"precipitation":0.0},{"time":"2023-07-12T04:00:00","temperature":24.2,"precipitation":0.0},{"time":"2023-07-12T05:00:00","temperature":24.5,"precipitation":0.0},{"time":"2023-07-12T06:00:00","temperature":24.4,"precipitation":0.0},{"time":"2023-07-12T07:00:00","temperature":24.4,"precipitation":0.0},{"time":"2023-07-12T08:00:00","temperature":24.6,"precipitation":0.0},{"time":"2023-07-12T09:00:00","temperature":24.7,"precipitation":0.0},{"time":"2023-07-12T10:00:00","temperature":24.7,"precipitation":0.0},{"time":"2023-07-12T11:00:00","temperature":24.7,"precipitation":0.0},{"time":"2023-07-12T12:00:00","temperature":25.1,"precipitation":0.0},{"time":"2023-07-12T13:00:00","temperature":25.3,"precipitation":0.0},{"time":"2023-07-12T14:00:00","temperature":25.4,"precipitation":0.0},

[ArnauAregall]

Yes, that's it!

Although would be great to double check with multiple objects in the array, which was the original intended use case.

[
  {
    "latitude": 41.3874,
    "longitude": 2.1686
  },
  {
    "latitude": 47.9990,
    "longitude": 7.8421
  }
]

[olegz]

It's fixed, so give it a shot, I'll kick off a snapshot build so 4.0.5-SNAPSHOT will be available soon