I don't want to pay for a portion of the bachelor party
Does it have anything to do with that woman?
Yes, the lady is a close acquaintance and I don't want to hang out with her.
If you're not attending the party, you shouldn't be held responsible for paying for some of it. You are not a part of that party and enjoy nothing from it, so you shouldn't have to pay for any of it.
I just don't want to look like the jerk in this situation.
No one reasonable should have a problem of you not chipping in for the bachelor's party if you aren't attending. If they do have a problem with it, then remind them you aren't attending.