I made fun of a cancer patient
Why would you make fun of anyone, little own a cancer patient? Did he make fun of you first?
Uh, the thing is that he did make fun of me first.
If someone makes fun of you then the normal reaction is to make fun of them back, but this was a cancer patient and they might not have much longer to live. Never do that even with revenge on your mind, the patient will be sad and have less support. They need support in order to get well again, and making fun of them is so not cool.
It was only because they started it, I thought they were inviting me to join.
It may have been so, but did he tell you that you can do such a thing? I think it's best to go ahead and apologize so there won't be any regrets. How about showing him kindness and love instead, you might be the friend he is looking for.