draft a protobuf definition of result document

[williballenthin]

so that capa can be more easily integrated into projects that already use protobufs

not sure if/where we should add code that interacts with protobufs. maybe could have it as an optional dependency and output format. or a standalone translator that goes from json -> protobuf.

[williballenthin]

originally i had proposed a translator that can go directly from json to protobuf. this would require that the protobuf format be exactly compatible with the json format. im not sure yet if this is easy to do, or if proto style is slightly different from json style.

if its not easy to to do (i.e. we have to change more than a few lines of the existing json format or do lots of gymnastics in the proto format) then we should keep the python-native format the primary format, always translating to/from this. that is, in order to go json -> proto, then we do json -> python then python -> proto.

at least, that's what im thinking now. keep discussing here.

[williballenthin]

im also thining about how to avoid having the same structure definitions in different formats, i.e., pydantic, protobuf, etc. I wonder how hard it would be to generate protobuf definitions from the pydantic models. then we could use the pydantic models as the source of truth and re-generate the protos as needed.

we'd have to be careful about backwards compatibility. maybe we'd need to embed the major version number in the structure somehow.

[williballenthin]

question: do we need to round-trip the protobuf format back into python? or just emit it from python to protobuf? we'll need code that translates from the protobuf generated instances to the capa instances. might be another use for code generation/interpretation of pydantic models.

edit: i think the primary use case is to get capa results into other external systems. so export is the primary direction. but im not sure how we'd do testing, etc. if we can't bring the data back into python/capa. so as long as its not a lot of work, lets try to support importing, too.

[mr-tz]

im also thining about how to avoid having the same structure definitions in different formats, i.e., pydantic, protobuf, etc. I wonder how hard it would be to generate protobuf definitions from the pydantic models. then we could use the pydantic models as the source of truth and re-generate the protos as needed.

This would be neat, but I don't expect the format to change much/often so may not be worth it.

question: do we need to round-trip the protobuf format back into python? or just emit it from python to protobuf?

My current understanding: with the protobuf definition we can read and convert proto/json, see https://googleapis.dev/python/protobuf/latest/google/protobuf/json_format.html.

Also: "Proto3 supports a canonical encoding in JSON, making it easier to share data between systems. The encoding is described on a type-by-type basis in the table below." - https://protobuf.dev/programming-guides/proto3/#json

[mr-tz]

closed via #1316