Julia: serialize the length of the model

[rcurtin]

While playing with the mlpack Julia bindings, I noticed that there is a bug in serialization that only happens in some niche cases.

Specifically, we serialize and deserialize the model directly to an IO stream, but we don't take the length of the serialized model into account. This means that at deserialization time, we eat everything in the stream---this screws up deserialization of anything that comes after the mlpack model.

Here's an example of a badly-behaved deserialization function:

function deserializeDecisionTreeModel(stream::IO)::DecisionTreeModel
          buffer = read(stream)
          DecisionTreeModel(ccall((:DeserializeDecisionTreeModelPtr, decision_treeLibrary), Ptr{Nothing}, (Ptr{UInt8}, UInt), Base.pointer(buffer), length(buffer)))
end

The solution here is, luckily, pretty easy: we just also serialize a UInt containing the length of the serialized model, and then deserialize that first to limit our read() to the appropriate length:

function deserializeDecisionTreeModel(stream::IO)::DecisionTreeModel
          buf_len = read(stream, UInt)
          buffer = read(stream, buf_len)
          DecisionTreeModel(ccall((:DeserializeDecisionTreeModelPtr, decision_treeLibrary), Ptr{Nothing}, (Ptr{UInt8}, UInt), Base.pointer(buffer), length(buffer)))
end

I added a test case for this situation also.

[zoq]

Looks good to me.

[mlpack-bot]

Second approval provided automatically after 24 hours. 👍