My boss complained that I dressed to casually.
Will he fire you for that?
No they won't. But they need to let this go.
You should follow the rules of your job. If your boss thinks it's too casual then he is probably right.
I don't have to listen to my boss.
You should listen to your boss so you do not get fired. Just don't dress as casually from now on, you do not want to lose your job.