Even though I am supposed to pick up my brother from school, I don't think that I want to.
Seriously? What reason do you have for not wanting to pick up your own brother?
Because I was driving around the city without anywhere to go
If your brother needs a ride, you should pick him up and help out. He might have no other way to get home and be confused.
I can't always do this for him since the kid is always doing stupid shit.
You just said that you're already driving around and have nowhere to go, so why not do this favor for your brother. People will always expect you to help family out in times of need.