I'm going to call an older man 'buddy'
Why would you call this man "buddy"?
I feel the word is cute and appropriate when referring to my father.
It's generally pretty rude to talk to older people in such an informal way, as they usually want to be treated with more respect.  However, if you are close, then it is fine to use nicknames as long as it isn't disrespectful.
Who cares? He called me buddy too so I'm just sharing my name.
Well I'm glad you two are becoming good friends!