I won't enjoy a family trip to Las Vegas.
Why won't you enjoy going there?
I really hate the hot climate and the fact that my family wants to go there is annoying.
Nothing says you need to enjoy it.  Heck, you can just ask not to be included.  However, don't expect them not to go just because you aren't.
I still want to give them a hard time and make their life miserable over it.
It's fine if you wouldn't enjoy the trip that they have planned but that is no reason to ruin it for them. Let them enjoy their trip.