I feel like I don't need to apologize
Why wouldn't you feel the need to apologize?
I know a woman was hurt by something I said, but I don't really feel I need to apologize
I think it is wrong not to apologize. It is expected to apologize if you hurt someone. You can set a good example by doing the right thing.
Do you think if I apologize it would make it worse? I am scared to do it.
How would that make it worse? It would make things better, of course. Just go to her and say I'm sorry, it is simple as that.