I'm going to stand up for my grandpa after he made an off-colour joke to my sister in law
Was your sister in law offended by the joke?
Yes she was offended but it was only a joke
You might not have found it problematic, but if a joke is offensive to someone, you should not defend the person who told it. You should let the joke teller know that they are being disrespectful to others, and they should not tell off color jokes.
It's just a joke.
I understand that it was just for fun, and that your grandfather probably never meant to hurt anyone's feelings. But the fact is that your sister in law was offended, and you should acknowledge that. Don't criticize your grandfather, but just let him know that maybe jokes like that should be avoided.