Should I go on a trip over my youngest daughter's birthday?
That sounds terrible. What do you think will happen if you're not there?
I just want to pay attention to my own needs for a while
You shouldn't go on a trip when your daughter is expecting you to be there for her birthday. Furthermore, you shouldn't be taking vacations without even asking your immediate family if they want to come, too.
It's just a one day trip and I've done this before many times.
One day isn't so bad, but why not let your family, especially your daughter know before hand. Have her present ready before you leave and let them know it's for something important. You could also call her the day of to still be present at her birthday.