I'm not going to say I love you back
Do you not love them? Why don't you feel like saying it.
Because I think it's silly to say it back when I don't feel that way.
That makes sense. It's fine to feel weird about saying I love you back to someone if you don't feel the same way.
I thought you'd be angry at me for feeling that way.
Of course not. You shouldn't feel pressured to say something you don't believe in.