My best friend expects me to play with them whenever he wants. I'm not going to though.
If he's your best friend, why wouldn't you?
I want to choose when I play with him, not just be expected to play whenever he wants me to.
That seems logical. It's fine to want to do things when you want to.
It feels wrong not to play with him when he wants to though. He is just a little kid and wants to play when he can't do other things.
You are not a toy though. You are a person just like him who deserves to spend your free time how you want. He doesn't always need you whenever he wants to play, and it would be good for him to learn to entertain himself or make other friends too.