Modify Message After Calling serverStream.Send()

[lihao-figma]

Hello gRPC-Go maintainers,

Our application requires a substantial number of byte slices to compose outgoing messages, and we are exploring ways to reduce these allocations. We would appreciate any guidance you could provide on this matter.

Currently, we have attempted using a bytes.Buffer to assemble the message content and then reclaim it after calling serverStream.Send(). However, we noticed the following comment in the source code:

	// It is not safe to modify the message after calling SendMsg. Tracing
	// libraries and stats handlers may use the message lazily.

https://github.com/grpc/grpc-go/blob/master/stream.go#L1554-L1555

Given this, if we disable tracing and stats features, can we safely reclaim the bytes.Buffer? Furthermore, would this approach remain safe in future versions?

After reviewing the latest implementation, it appears that the gRPC server stream allocates a bytes.Buffer to store the newly compressed message and sends it out, followed by a defer free():
(https://github.com/grpc/grpc-go/blob/master/stream.go#L1678-L1682)

This suggests that, from a data integrity standpoint, modifying the data after calling Send() might be acceptable. We would greatly appreciate any insights or recommendations regarding this approach.

Thank you for your time and assistance.

[purnesh42H]

@lihao-figma prepareMsg returns the encoded copy of the message for sending it over the wire and that copy buffer get's deallocated once SendMsg function returns. No where the original message is being modified. If you see, same message is being passed to stats handler https://github.com/grpc/grpc-go/blob/master/stream.go#L1724. Yeah I wouldn't recommend modifying the original message after sending it.

[lihao-figma]

@purnesh42H thanks for the response.

Yes we see that original message won't get changed within the grpc stack in latest version.

We are wondering if we disabled stats handler, will it be ok to modify the message? We see that it's safe in latest version, as you pointed out, since prepareMsg won't touch anything in the message. We are wondering if we disabled stats handler and start to modify the message after send(), will it still be safe in future versions.

Or can we have a feature that all message processing are synchronous and message won't get referenced again after Send()?

Thanks again.

[purnesh42H]

@lihao-figma could you try using grpc.PreparedMsg? See example

grpc-go/experimental/shared_buffer_pool_test.go

Line 74 in a51009d

preparedMsg := &grpc.PreparedMsg{}

You can encode the message before and send the encoded message to stream.SendMsg. This will allow you to modify the message outside of stream.Send call without changing the message passed to stream.SendMsg. The encoding part within stream.SendMsg will be skipped since we are passing the encoded message already.

[lihao-figma]

Thanks for the pointer. This works well.