Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protobuf is really missing inheritance #5645

Closed
zatziky opened this issue Jan 28, 2019 · 6 comments
Closed

Protobuf is really missing inheritance #5645

zatziky opened this issue Jan 28, 2019 · 6 comments

Comments

@zatziky
Copy link

zatziky commented Jan 28, 2019

What language does this apply to?
proto3 OR proto-future

Describe the problem you are trying to solve.
Lot's of DRY code in statically typed languages.

Describe the solution you'd like
I'd like you to add inheritance to current proto3 or at least the future protobuf versions. An example is:

message Parent { }

message ChildA extends Parent { }
message ChildB extends Parent { }
message ChildC extends Parent { }

Alternatives you've considered
I know about composition and it's really not a cure.

Additional context
I know that protobuf favors composition over inheritance but it's really not a cure to remove inheritance completely. Statically typed languages like Java or C# suffer from composition only approach. These languages cannot use polymorphism because all the messages have the same parent GeneratedMessageV3. If inheritance was available, we could use polymorphism. Instead of having to write:

void method(ChildA arg)
void method(ChildB arg)
void method(ChildC arg)

we could simply write void method(Parent arg). Consequently, the missing inheritance generates a lot of DRY code is hardly or not at all DRYable. It leads to wider test suites that take more time to execute. I could go on but I hope I explained myself. (If not, just ask and I'll clarify it.)

@perezd
Copy link
Contributor

perezd commented Jan 29, 2019

FWIW

To quote the Protocol Buffers tutorial:
"Don't go looking for facilities similar to class inheritance, though – protocol buffers don't do that."

Instead of inheritance, Protocol Buffers tends to favor composition as a pattern.

@zatziky
Copy link
Author

zatziky commented Jan 30, 2019

@perezd

GRPC is not doing "favor composition over inheritance" pattern, it's applying "remove inheritance" pattern. :D

There are cases when inheritance is more suitable. Just because it's a good practice to "favor somehing", it doesn't mean it should be removed completely (or not implemented at all).

Real life example

Favor public transport over driving a car when going to work

Yes, most of the time we should use public transport because it's better for Earth. On the other hand, you have cases when driving a car is a better option, e.g. if there is no public transport in your surroundings.

@ObsidianMinor
Copy link
Contributor

ObsidianMinor commented Jan 31, 2019

How are you supposed to implement inheritance in a language that doesn't support inheritance like Rust or Go which only support struct-like data types?

@TeBoring
Copy link
Contributor

For proto3, use Any.
For proto2, user extension.

@weitzhandler
Copy link

I want to have a base type for all server messages, request and response, then I want to have different types of requests/responses.
Anyone has experimented with this idea and found any adequate way to implement that?

@vphantom
Copy link

I want to have a base type for all server messages, request and response, then I want to have different types of requests/responses. Anyone has experimented with this idea and found any adequate way to implement that?

For one project I did something like this to minimize redundancy:

message Request {
  uint64 magic = 1;  // Optional unique identifier for this request
  oneof _Request_oneof {
    ProfileRequest     profile_request     = 3;
    // ...other kinds of requests
  }
}

message Response {
  uint64 magic = 1;  // Your optional unique identifier echoed back so you may match our responses with your requests
  oneof _Response_oneof {
    string              error                = 2;  // Defined instead of your expected response below, on failure
    ProfileResponse     profile_response     = 3;
    // ...other kinds of responses
  }
}

message Message {
  // ... auth stuff
  repeated Request  requests       = 4;
  repeated Response responses      = 5;
  string            status         = 6;  // Protocol-level status or error message
}

So a Russian doll, basically: a Message is what's always on the wire, each can contain any number of requests and responses (they're multiplexed by magic ID to minimize the number of distinct messages being exchanged), then in the Request and Response I point to the specific one it represents. For example a ProductRequest might request the details of a product by ID and the ProductResponse would contain a Product, etc.

chaokunyang added a commit to apache/incubator-fury that referenced this issue Mar 30, 2024
…tion (#1413)

## What does this PR do?

This PR standardizes fury cross-language serialization specification. It
comes with following changes:
- Remove type tag from the protocol since it introduce space and
performance overhead to the implementation. The `type tag` version can
be seen in
https://github.com/apache/incubator-fury/blob/6ea2e0b83d5449d63ca62296ff0dfd67b96c5bc5/docs/protocols/xlang_object_graph_spec.md
.
- Fury preserves `0~63` for internal types, but let users register type
by id from `0`(added by 64 automatically) to setup type mapping between
languages.
- Streamline the type systems, only
`bool/byte/i16/i32/i64/half-float/float/double/string/enum/list/set/map/Duration/Timestamp/decimal/binary/array/tensor/sparse/tensor/arrow/record/batch/arrow/table`
are allowed.
- Formulized the binary format for above types.
- Add type disambiguation: the deserialization are determined by data
type in serialized binary and target type jointly.
- Introduce meta string encoding algorithm for field name to reduce
space cost by 3/8.
- Introduce schema consist mode format for struct.
- Introduce schema envolution mode for struct: 
- this mode can embeed meta in the data or share across multiple
messages,
- it can avoid the cost of type tag comparison in frameworks like
protobuf

This protocol also supports object inheriance for xlang serializaiton.
This is a feature request that users has been discussed for a long time
in protobuf/flatbuffer:
- google/flatbuffers#4006
- protocolbuffers/protobuf#5645

Although there are some languages such as `rust/golang` doesn't support
inheriance, there are many cases only langauges like
`java/c#/python/javascript` are involved, and the support for inheriance
is not complexed in the protocol level, so we added the inheriance
support in the protocol. And in languages such as `rust/golang`, we can
use some annotation to mark composition field as parent class for
serialization layout, or we can disable inheriance foor such languages
at the protocol level.
 
The protocol support polymorphic natively by type id, so I don't include
types such as `OneOf/Union`. With this protocol, you can even serialize
multiple rust `dyn trait` object which implement same trait., and get
exactly the same objects when deserialization.

## Related issue
This PR Closes #1418

---------

Co-authored-by: Twice <twice@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants