I like to smell the pillow in our guest room because it smells feminine
You're kidding? Is this a sexual fetish?
It's just attractive. The pillow smells like fresh femaleness.
You're not hurting anyone, so whatever. Just don't get caught or explaining yourself will be awkward.
Does it matter?
It's fine if you know your limits and to keep it to fresh and new things, not used. It can make a lot of women uncomfortable if you think of things this way. So it's best to keep it to yourself.