identical() crashes when dealing with recursive fields in proto

[jarodmeng]

Although it doesn't occur very often in the wild, it's possible to create protos that have recursive fields (see example proto below). When identical() is called to compare messages whose descriptor has recursive fields, R crashes due to stack overflow.

identical() calls C++ function identical_messages_() which iterates all fields in the messages' descriptor for a comprehensive comparison. In the case of recursive fields, this iteration never ends and eventually crashes.

One possible fix is for identical_messages_() to skip a field if it's not populated in both messages. This will naturally terminate the iteration at some point, because although a field's definition can be recursive, its actual data cannot be never-ending. I will send a PR for consideration.

library(RProtoBuf)

# Create an example proto file with recursive field
proto.file <- tempfile(fileext = ".proto")
writeLines(
  "
  syntax = 'proto2';

  package example;

  message Recursion {
      optional int32 id = 1;
      optional Recursion value = 2;
  }
  ", proto.file
)
readProtoFiles(proto.file)

# Create two identical proto messages with recursive field unset
p <- new(example.Recursion, id = 1)
p2 <- new(example.Recursion, id = 1)

# R crashes when identical() is called
identical(p, p2)
all.equal(p, p2)

^{Created on 2018-11-30 by the reprex package (v0.2.1)}

[eddelbuettel]

Thanks for motivating a PR with an example. That is very good (yet rare ...) practice.