I recently posted a message on the MacVim discussion group about this issue, and found that this only happens with your version.
In short, I have found that using your fork with the Core Text renderer enabled causes the window to somehow draw square corners. This is remedied by making the scrollbars visible or using the regular NSTextView renderer.
I just thought I would mention this to you and see if you could make heads or tales of it!
There are even worse consequences actually. For instance, see the left sidebar in this screenshot:
I would love to see this fixed, but I have no time at all atm to look into MacVim.
Yeah, I'm really the one who needs to look into this but it is just kind of daunting and I've just started a new job so it is hard to find the time.