I'm hoping my invalid grandfather will die soon
That's very shocking. Why would you want someone die?
It's really selfish, I know. But I want to ignore him as much as possible.
You should not say things like that. It is hurtful and wrong to wish death on a family member.
But at the same time, I don't want to deal with his shit anymore.
If they are not causing harm or bothering you, just plainly ignoring them should be enough without them wishing them harm. I do hope you focus on your work or studies, rather than being obsessive about your grandfather specially if they haven't done anything wrong.