Fix integer underflow and add additional size check

[otsmr]

This fixes #105. The check of the size was the wrong way as size must be greater then remaining otherwise there is an interger underflow of remaining. This adds also an additional check of res2 to ensure that this is never bigger then buf_len.

Please double check if this works, I am currently on vacation and I am not able to compile it to test it!