Fix blend mode assignment in GuiLayer rendering

Replaces direct assignment of blendMode with a call to compositeBlendMode to ensure correct alpha blending factors are used. Adds the compositeBlendMode helper function to adjust blend modes as needed.