I feel absolutely guilty about ghosting someone
Did you ghost them on purpose?
Yes, though I don't care about him, I still felt sorry.
Even if you don't care about the person, you should let them know you don't want to speak to them anymore. I imagine you'd want the same treatment.
I did get ghosted before then I got over it.
Everyone is different. It may not have affected you but it could affect this guy. It won't hurt to tell him the truth.