I find washing my hands in a public restroom to be more unsanitary than not washing them
Why do you think washing your hands in public is unsanitary?
The blow dryer just blows germs around.
That may be, but you should wash your hands. You don't want to spread diseases or get sick.
I'm not sure about that.
It's important to wash your hands after using the restroom. Most public restrooms have paper towels rather than blow dryers. But you can carry hand sanitizer if you're worried.