I basically want to tell my sister I don't consider her a friend
Do you really not consider her as a friend?
No, but I don't want to feel bad about it.
It is ok to have differing ideas about what a friendship is. You can talk to her and try to clarify your expectations and strengthen it.
Maybe it's good for me to have a "deal breaker" on what makes a friendship or not.
I am not sure what that means in this context. If you don't want to be friends, you can be honest with her, she's family. As long as she is still your sister, that is all that matters.