I down vote people who start a sentence with 'i mean'
Is that the only reason why you downvote these people?
Yes, it's infuriating to read a message introducing something with "I mean."
To you this is very infuriating and I understand, but to them they just have a different way to start a sentence and they may think that you have annoying habits, whether you realize it or not.
I'm not down voting over petty reasons. I guess I do have a habit, I didn't think too much of it.
I'm glad that you recognize that you may have your own habits, and hopefully you can empathize with those who talk in an informal way.