I don't want to go away with friends again after their rough behavior
What do you mean? What exactly did they do to you?
They were too rough and i'm a pretty fragile guy
If you don't want to go, you have every right to refuse. You shouldn't put up with rough behavior.
But i love them and want them to come back
You are a very loyal person, but sometimes your health matters a lot more than friendship! It's very okay to have time for yourself and recovering your physical and mental state, so you can be more ready to hang out with them next time.