proto.Merge != proto.Marshal, proto.Unmarshal #1163
Description
This isn't a blocking issue for me but results from my developing familiarity with the golang protobuf SDK and a solution I'm toying with. In this solution, the clients succeeds by marshaling the dynamicpb.Message onto the wire and shipping this to the server; I don't need to be able to convert the dynamicpb.Message into ComplexRequest.
I am curious as to why this doesn't work.
If this appears curious to you too, I'm motivated to help pursue it.
If not, please feel free to just close it.
What version of protobuf and what language are you using?
go version
go version go1.14 linux/amd64
protoc --version
libprotoc 3.12.3go.mod:
module complex
go 1.14
require (
github.com/golang/protobuf v1.4.2
google.golang.org/grpc v1.30.0
google.golang.org/protobuf v1.25.0
)
What did you do?
syntax = "proto3";
package examples;
option go_package = "complex/protos";
service ComplexService {
rpc add(ComplexRequest) returns (ComplexResponse);
rpc sub(ComplexRequest) returns (ComplexResponse);
rpc mul(ComplexRequest) returns (ComplexResponse);
rpc div(ComplexRequest) returns (ComplexResponse);
}
message ComplexRequest {
Complex a = 1;
Complex b = 2;
}
message ComplexResponse {
Complex result = 1;
string error = 2;
}
message Complex {
float real = 1;
float imag = 2;
}and:
protoc \
--proto_path=./protos \
--descriptor_set_out=./protos/descriptor.pb \
--go_out=plugins=grpc,module=[[PROJECT]]:. \
protos/complex.protoand (with apologies for the big slab of code):
package main
import (
"context"
"flag"
"io/ioutil"
"log"
pb "complex/protos"
"google.golang.org/protobuf/proto"
"google.golang.org/protobuf/reflect/protodesc"
"google.golang.org/protobuf/reflect/protoreflect"
"google.golang.org/protobuf/types/descriptorpb"
"google.golang.org/protobuf/types/dynamicpb"
)
var (
descriptorFile = flag.String("descriptor_file", "", "Descriptor filename")
)
func main() {
flag.Parse()
// Open the descriptor
log.Println(*descriptorFile)
fds := &descriptorpb.FileDescriptorSet{}
{
b, _ := ioutil.ReadFile(*descriptorFile)
proto.Unmarshal(b, fds)
}
files, _ := protodesc.NewFiles(fds)
d, _ := files.FindDescriptorByName(protoreflect.FullName("examples.ComplexService"))
s, ok := d.(protoreflect.ServiceDescriptor)
if !ok {
log.Fatal("Unable to assert into ServiceDescriptor")
}
md := s.Methods().ByName(protoreflect.Name("add"))
// Create dynamic message from service's input method
i := md.Input()
src := dynamicpb.NewMessage(i) // == examples.ComplexRequest
{
fd := i.Fields().ByName(protoreflect.Name("a"))
if fd.Kind() != protoreflect.MessageKind {
log.Fatal("Expect `a` to be a Message of type Complex")
}
a := fd.Message()
m := dynamicpb.NewMessage(a)
{
fd := a.Fields().ByName(protoreflect.Name("real"))
m.Set(fd, protoreflect.ValueOfFloat32(39))
}
{
fd := a.Fields().ByName(protoreflect.Name("imag"))
m.Set(fd, protoreflect.ValueOfFloat32(3))
}
// m := pb.Complex{Real: 39, Imag: 3}
x := m.ProtoReflect()
v := protoreflect.ValueOfMessage(x)
src.Set(fd, v)
}
{
fd := i.Fields().ByName(protoreflect.Name("b"))
m := pb.Complex{Real: 39, Imag: 3}
x := m.ProtoReflect()
v := protoreflect.ValueOfMessage(x)
src.Set(fd, v)
}
dst := &pb.ComplexRequest{}This:
proto.Merge(dst, src)panics:
panic: descriptor mismatch
goroutine 1 [running]:
google.golang.org/protobuf/proto.Merge(0xc16140, 0xc000035e80, 0xc16c60, 0xc000035e00)
/home/dazwilkin/go/pkg/mod/google.golang.org/protobuf@v1.25.0/proto/merge.go:34 +0x470
main.main()The Fullname() values match but elsewhere the Descriptor() values disagree.
But, the described equivalent (albeit without proto.UnmarshalOptions{Merge:true}):
https://github.com/protocolbuffers/protobuf-go/blob/v1.25.0/proto/merge.go#L25
{
b, err := proto.Marshal(src)
if err != nil {
log.Fatal(err)
}
if err := proto.Unmarshal(b, dst); err != nil {
log.Fatal(err)
}
}Succeeds:
2020/06/30 12:26:53 result:{real:1512 imag:234}What did you expect to see?
Success to be 'equivalent' Marshal (of dynamicpb) followed by Unmarshal (into examples.ComplexRequest).
What did you see instead?
Merge panics.
Anything else we should know about your project / environment?
The src and dst structs are rather deep and so it's not immediately obvious where the discrepancy arises (perhaps a good next step for me?)
If I output the objects, they appear (!?) to match:
log.Printf("[dst] %+v", dst.ProtoReflect().Descriptor())
log.Printf("[src] %+v", src.ProtoReflect().Descriptor())yields:
2020/06/30 12:34:11 [dst] MessageDescriptor{
Syntax: proto3
FullName: examples.ComplexRequest
Fields: [{
Name: a
Number: 1
Cardinality: optional
Kind: message
HasJSONName: true
JSONName: "a"
HasPresence: true
Message: examples.Complex
}, {
Name: b
Number: 2
Cardinality: optional
Kind: message
HasJSONName: true
JSONName: "b"
HasPresence: true
Message: examples.Complex
}]
}
2020/06/30 12:34:11 [src] MessageDescriptor{
Syntax: proto3
FullName: examples.ComplexRequest
Fields: [{
Name: a
Number: 1
Cardinality: optional
Kind: message
HasJSONName: true
JSONName: "a"
HasPresence: true
Message: examples.Complex
}, {
Name: b
Number: 2
Cardinality: optional
Kind: message
HasJSONName: true
JSONName: "b"
HasPresence: true
Message: examples.Complex
}]
}I wonder whether part of my issue is that ComplexRequest contains a nested Message[Descriptor|Type] of Complex?