I'm nervous about family dinner because I don't want to be the only person stuck talking to my grandfather
Um, isn't that a little rude? Why don't you want to talk to him?
I don't like his views on things.
It's okay not to agree with family members on things. Maybe you can discuss his views and how to avoid certain topics if you want to continue to have a relationship with him.
I'm not thinking about continuing a relationship. I just want to avoid talking tonight.
That's the wrong attitude to take towards your grandfather. Don't you think he really looks forward to talking to with you? You can make him happy and stay comfortable by steering the topic toward benign issues.