My boss closed his office door because he wanted to get some work done but I walked in to ask him a question.
Really? Did you think he was bothered when you walked in like that?
Yes, but I left right after.
It's impolite to walk into your boss's office when his door is closed because that means he probably doesn't want to be disturbed. In the future you should knock and ask if you can come in before entering.
I'm only asking him a question.
Just because it was a simple question doesn't mean you can barge in like that. How did you think he felt when you showed up unannounced? I think you should apologize, he would understand if your question was really important back then!