Is it wrong that I'm thinking about avoiding one of my coworkers, and I'm a coward?
Is it because of their attitude?
He can be a kind guy, but he always exaggerates a lot
It's rude to avoid people, but this doesn't necessarily make you a coward.  If your coworker is generally a kind person, then you can probably let him know, gently, that he is exaggerating too much.
I'll try it. I didn't want to sound so out of line(tm)
Avoid people can seem rude and it would be obvious that you are doing so. It's great that you are willing to try another way.