I turned up to my brother's wedding in shorts and a t-shirt
Why would you show up in shorts and a t-shirt?
To get attention.
Personally, something like that wouldn't bother me. Your family probably minded, but again, I don't think it's an issue.
They thought it was disrespectful.
If they don't think it's okay to have shorts at the wedding then you should apologize for being underdressed.