I want to wear my wife's underwear when she is out of town
What compels you to want to wear your wife's underwear when she doesn't know about it?
I like sneaking around behind her back.
Not telling your wife about wearing her underwear is dishonest and wrong. Keeping secrets from your wife will destroy the trust in your relationship.
I think it's sweet. I get to do my own secret thing.
While it might be harmless depending on your relationship, I think it's wise to just be honest about it to her. How would she feel if she saw her underwear being used, she might get the wrong ideas about another woman.