Unable to send null payload with KafkaTemplate#send(Message)

[pswrdf]

In what version(s) of Spring for Apache Kafka are you seeing this issue?
3.0.11

Describe the bug

Unable to send null message through KafkaTemplate#send(Message).
Unable to create GenericMessage with null payload.

To Reproduce

spring:
  kafka:
    producer:
      value-serializer: org.apache.kafka.common.serialization.ByteArraySerializer

    @Bean
    public RecordMessageConverter jsonMessageConverter(ObjectMapper objectMapper) {
        return new ByteArrayJsonMessageConverter(objectMapper);
    }

       final var Object myValue = null;
       final var message = MessageBuilder.withPayload(myValue).setHeader(TOPIC, topic).build();
        kafkaTemplate.send(message);

Expected behavior

Kafka sends null message.
But we get: IllegalArgumentException: Payload must not be null

We could set myValue=KafkaNull.INSTANCE;
But JsonMessageConverter inherited classes cannot proccess KafkaNull.

[artembilan]

I confirm this as a bug.
All those MessagingMessageConverter extensions must support KafkaNull and just short-circuit same way as their super class does.
I would say that JsonMessageConverter.convertPayload() must call super and throw that UnsupportedOperationException only if result not null.
The rest of impls should also call super and perform JSON conversion only of not null.

As a workaround I suggestion to extend that ByteArrayJsonMessageConverter and override its convertPayload() the way we just discussed:

	protected Object convertPayload(Message<?> message) {
		Object payload = message.getPayload();
		if (payload instanceof KafkaNull) {
			return null;
		}
		else {
			try {
				return getObjectMapper().writeValueAsBytes(message.getPayload());
			}
			catch (JsonProcessingException e) {
				throw new ConversionException("Failed to convert to JSON", message, e);
			}
		}
	}

Would be great if you can contribute the fix.

Thank you!

[jakabgyonci97]

Hello, I have a somewhat similar issue in version 3.1.1.
I'm writing a kafka message using KafkaTemplate#send(Message). In my case there are different headers that I attach to the message, some value value null. When the send operation is performed I get the following error:

java.lang.NullPointerException: Cannot invoke "Object.getClass()" because "rawValue" is null

[artembilan]

In my case there are different headers that I attach to the message, some value value null

So, don't add those null headers.
In fact use a MessageBuilder API to create that Message instead.