Do I really have to keep talking to my friend?
Why do you want to stop talking to them?
He is really annoying me most of the time lately
It is bad to stop talking to a friend but friends can grow apart. It is better to deal with it and end the friendship on a good note.
I don't want to stop. I just want to not give a shit anymore.
Then maybe you should talk to him. Tell him how you are feeling and see what he says.