Decimal representation for optional mantissa with non-constant exponent

[RFrenkel]

Hi Don,
Which way is the right way to specify null-able decimal with not fixed exponent?
Do we specify only mantissa null value or the exponent also?

Will this work?
composite name="decimal" >
type name="mantissa" primitiveType="int64" presence="optional" nullValue="9223372036854775807" />
type name="exponent" primitiveType="int8" />

Or
composite name="decimal" >
type name="mantissa" primitiveType="int64" presence="optional" nullValue="9223372036854775807" />
type name="exponent" primitiveType="int8" presence="optional" nullValue="127" />

If there is no need to provide null value for exponent, then what should be the exponent value on the wire with null mantissa?

I looked up the spec and couldn't find examples of null-able decimals.

[donmendelson]

The spec currently says:

For optional fields, a special mantissa value is used to indicate that a field value is null.

For efficiency, the value of the exponent need not be checked by a decoder if the mantissa is null.

However, I don't like the fact that the value of the exponent is non-deterministic. In my opinion, all values should be deterministic. That is, given the same semantic values, two encoders should produce exactly the same buffer. Also, failing to overwrite previous values in a buffer could leak information and pose a security risk. So it should always write something. I would be in favor of changing the spec to say that the exponent should also be set null. Do others agree?

[kleihan]

Agree. Did you mean to write "However, I don't like the fact that the value of the exponent is non-deterministic."?

[donmendelson]

Thanks, Hanno. Corrected.

[RFrenkel]

Thank you, Don and Hanno!

So i take it the best practice is to set exponent also to null =127.

[donmendelson]

I suggest that we edit the spec to say that this is recommended practice.

(BTW the spec says that character arrays should be padded with nulls for the same reason--a deterministic output.)

[RFrenkel]

Great, thank you!