I sort of stormed out of my friend's house and dislike being around people who are high or drunk
Is it because people who aren't sober make you uncomfortable?
Yes, people who are high or drunk can't reason properly
If these people make you feel like that, then what you did is a good thing.  You shouldn't spend a lot of time around people that are getting wasted constantly.  You could find a better crowd of people to hang with!
I see what you're saying, but I don't want to lose a friend over this.
I know they are your friends, but you have to consider what's best for you and surround yourself with people who have positive influence on your life.