How does readUBitVar in reader.go work?

[gaultier]

Hello, I am an avid dotabuff user and I have been browsing the code of this project over the last few days. Big thanks for making it available to everyone! It is very readable as well (even without a Go background).

One point is not clear though: looking at the readUBitVar function (https://github.com/dotabuff/manta/blob/master/reader.go#L158), it seems to read a uint32 from an array of bytes. It is typically used to get the type of the message(s) inside a CDemoPacket.

Trying to make my own implementation of a dota 2 replay parser (in rust), I have used the equivalent of readVarUint32 to do this. It works well to get the type (EDemoCommand of type uint32) and size (uint32) of a demo command, but not to get the type of an embedded message in a CDemoPacket: it returns the wrong value.

So I wonder how this function works and what is the reasoning behind it (why not use readVarUint32) ?
This kind of low-level bit manipulation might be explained in some article or wikipedia page you could point me to? Is it a protobuf specific encoding?

What I have understood so far: the encoding is little-endian so the first 6 bits are in fact the last 6 bits in normal order. They contain information about the encoding of the number. We then read some more bits (a varying number depending on the encoding) to get the full number.

PS: I have looked at the python implementation (smoke) which does something similar but it is not clearer (https://github.com/skadistats/smoke/blob/a2954fbe2fa3936d64aee5b5567be294fef228e6/smoke/io/stream/entity.pyx#L11)

Thank you very much for your answer!

[jcoene]

A rust replay parser sounds cool! Do you plan to open source it?

readUBitVar and readVarUint32 read unsigned integers with different decoders. You'll need your own readUBitVar method with the same behavior of that in manta.

Here's the implementation in Java, from Clarity: https://github.com/skadistats/clarity/blob/master/src/main/java/skadistats/clarity/decoder/bitstream/BitStream.java#L153-L175

[gaultier]

Thank you, that's exactly what I was looking for! And yes I will open source it when it resembles something usable :)

[jcoene]

Sounds good!