.NET object does not support null for optional fields

[dmani]

We are using the GTFS Realtime bindings package for .NET to convert vehicle and trip update data in our database to the GTFS-RT ProtoBuf format. We do not have values for some of the optional elements such as vehicle bearing, occupancy, speed and vehicle stop status. But the C# object uses non-nullable properties for these and hence defaults values such as 0 for bearing, EMPTY for occupancy, 0 for speed, IN_TRANSIT_TO for vehicle stop status etc. The serialized ProtoBuf output also has these default values (instead of null). Would it be possible to update the code so that nulls are allowed for optional fields?

[barbeau]

@dmani Which version of the .NET bindings are you using?

It sounds like this is the inverse problem of #32, in which default values aren't serialized to the protobuf message.

I believe this should be controllable via the RuntimeTypeModel.UseImplicitZeroDefaults setting.

Could you try setting this to true in code and see if this fixes the problem?

For an example see https://stackoverflow.com/a/12296458/937715.

[dmani]

Thank you for the quick response. We are using v0.0.4 of GtfsRealtimeBindings and v2.4.0 of protobuf-net for serialization.

[dmani]

The correction you have suggested is on the protobuf serialization side. But issue we are having is in the GtfsRealtime.cs class. Optional elements (properties in the class) such as bearing, speed, occupancy, etc. do not allow for null. Null should be a valid value (which should imply that we do not have a value for the element). For example, setting a default of bearing=0 when we do not have a value for bearing is incorrect as bearing=0 implies that the vehicle is pointing north.

We notice in GtfsRealtime.cs that the private variables are nullable while the corresponding public properties are not. As a result, it does not allow us to set any of the optional properties to null when we do not have a value for it.

For instance, below is a snippet from GtfsRealtime.cs

private OccupancyStatus? __pbn__occupancy_status;
        public OccupancyStatus occupancy_status
        {
            get { return __pbn__occupancy_status ?? OccupancyStatus.EMPTY; }
            set { __pbn__occupancy_status = value; }
        } 

Can it be changed to code below, so that the public property is nullable?

private OccupancyStatus? __pbn__occupancy_status;
        public OccupancyStatus? occupancy_status // make the property nullable
        {
            get { return __pbn__occupancy_status; } //return null when there is no value, not OccupancyStatus.EMPTY
            set { __pbn__occupancy_status = value; }
        } 

The same should be done for all optional properties. Please let us know if you can implement this change.

[barbeau]

@dmani We use an automated tool to generate the bindings, so I'm not sure this is an option at the code generation stage.

To make sure I understand your use case - you're trying to use this class as an internal data model and need to read and write values to this object (without serializing it to a protocol buffer representation)?

[barbeau]

From a quick search, the author of the library says in this SO post, answering the question of "why no nulls":

Simply, because the underlying protocol specification ("protobuf") has no concept of null and no way of expressing it. If I can't express it in the protocol, the best that can be supported is some kind of implicit "the absence of something means leave it alone which means it will probably be null".

And he's right, the protocol buffer format uses default values instead of null values, as described here:
https://developers.google.com/protocol-buffers/docs/proto#optional

Given this library is intended primarily as a (de)serialization library, you may need to create your own wrapper or data model classes if you want a more sophisticated data model internally for your application.

If you're worried about these values being serialized, which is the focus of this library, then I believe my original suggestion of setting RuntimeTypeModel.UseImplicitZeroDefaults to true may resolve this specific problem. Please give this a try and please re-open if it doesn't work as expected.