Can I take back my invitation to drive my roommate to the airport?
Why don't you want to take him any longer?
I don't like the guy very much
Regardless of how you now feel about the guy, you made an offer.  To back out of doing it now puts him in a major bind, and is pretty rude of you.  Follow through on what you promised to do.
I don't care, next week is more important than him.
You already made plans to take him so it is bad to back out now. With it being so close he might have a hard time finding another ride. You should be more kind to him.