Recursive data structure unmarshalling gives error "cannot parse invalid wire-format data"

[PasinduTennage]

What version of protobuf and what language are you using?
Version: go1.18.1 linux/amd64, github.com/golang/protobuf v1.5.2

What did you do?
I am trying to use recursive proto definitions.

.proto file

message AsyncConsensus {
  int32 sender = 1;
  int32 receiver = 2;
  string unique_id = 3; // to specify the fall back block id to which the vote asyn is for
  int32 type = 4; // 1-propose, 2-vote, 3-timeout, 4-propose-async, 5-vote-async, 6-timeout-internal, 7-consensus-external-request, 8-consensus-external-response, 9-fallback-complete
  string note = 5;
  int32 v = 6 ; // view number
  int32 r = 7;// round number
  message Block {
    string id = 1;
    int32 v = 2 ; // view number
    int32 r = 3;// round number
    Block parent = 4;
    repeated int32 commands = 5;
    int32 level = 6; // for the fallback mode
  }
  Block blockHigh = 8;
  Block blockNew = 9;
  Block blockCommit = 10;
}

The following is how I Marshal and Un-Marshal

func (t *AsyncConsensus) Marshal(wire io.Writer) error {
	data, err := proto.Marshal(t)
	if err != nil {
		return err
	}
	lengthWritten := len(data)
	var b [8]byte
	bs := b[:8]
	binary.LittleEndian.PutUint64(bs, uint64(lengthWritten))
	_, err = wire.Write(bs)
	if err != nil {
		return err
	}
	_, err = wire.Write(data)
	if err != nil {
		return err
	}
	return nil
}

func (t *AsyncConsensus) Unmarshal(wire io.Reader) error {

	var b [8]byte
	bs := b[:8]
	_, err := io.ReadFull(wire, bs)
	if err != nil {
		return err
	}
	numBytes := binary.LittleEndian.Uint64(bs)
	data := make([]byte, numBytes)
	length, err := io.ReadFull(wire, data)
	if err != nil {
		return err
	}
	err = proto.Unmarshal(data[:length], t)
	if err != nil {
		return err
	}
	return nil
}

func (t *AsyncConsensus) New() Serializable {
	return new(AsyncConsensus)
}

What did you expect to see?

When marshaled and sent to the same process via TCP, it should correctly unmarshal and produce correct data structures.

What did you see instead?

error "cannot parse invalid wire-format data"

Anything else we should know about your project / environment?

I tried with non-recursive .proto definitions, and never had this issue

[PasinduTennage]

This is a transient error; happens ones every 1000 messages or so. To reproduce, it requires to send a lot (I reproduced this with 2000 back to back messages) of messages

[dsnet]

A transient error like this suggests that there is possibly a race condition in your program. Have you tried building with -race and running the program?

[puellanivis]

Have you tried locking the Marshal and Unmarshal in a mutex? While the individual writes are guaranteed to be atomically written to the TCP socket, there’s no guarantee that two writes will happen sequentially without further synchronization.

Since this only happens intermittently, this is a strong possibility. If two lengths were written one after the other, and I’m very sure that you’re not writing 64 pebibytes of data, the first byte of each length should be 0x00, which contains an invalid field number: 0.

[PasinduTennage]

@puellanivis

I use locking for each invocation of Marshal and Unmarshal

To check my hypothesis, I check what happens for non-recursive .proto structs, it I can verify that this transient error never occurs.

[dsnet]

It isn't enough that every call of Unmarshal is locked. It is still possible that the inputs to those methods are being concurrently mutated.

[PasinduTennage]

@dsnet

Thank you.

[mohd-arif]

i am facing same issue. and i am serializing in python code and unmarshalling in golang code so obvoiusly there is no race condition in my case. can anybody tell what possible issue in my case ?
i gave description in of my code details in this link.
https://stackoverflow.com/questions/73631871/cross-language-deserialization-using-protobuf-pythongolang

[puellanivis]

@mohd-arif since your problem does not appear to be related to synchronization of marshal/unmarshal, could you please start your own issue, so we can address your issues directly?

When you do so, could you please post a hexdump of the protobuf you’re trying to unmarshal, or at least the actual protobuf definition?