I'm thinking of my friend trying to make me feel anxious
Is there a reason you're still thinking about it?
Yes I simply don't want him as a friend as long as he is trying to make me feel anxious for no reason.
Friends should be there for one another. If you don't feel like your friend is good for your mental health, maybe you should end that friendship.
No, I can't end it since he is the only friend that I have.
I understand that you are afraid of getting out of a toxic relationship, but you must do it for your health and sanity. There are lots of other people in the world for you to be friends with.