I feel like not answering someone that doesn't really want to talk with me
Why don't you want to answer them?
I'm fed up with listening to their shit
Do they really not want to talk with you? You should bring up this topic with them and have a conversation. You may be misreading things.
I'm sure they don't want to talk with me. I'll show them.
If that person doesn't want to talk to you. Why is that person calling you all the time? The views you have for that situation is skewed.