I don't want to go back
Why not?
I do want to go back actually. But at the same time I don't because I'm afraid that I will see his face again.
I still think that you should try to go back. Avoiding things like this will not make them go away.
I don't care if it doesn't go away on its own. I'm too scared to go back.
It can be scary to face somethings but it would not be good to avoid your responsibilities. You need to face your fears and do what you have to do.