I'm upset about receiving a celebratory gift for starting a new job.
Why would you be upset about getting a gift?
To me, it's just a regrettable waste of time and money.
I understand that you may not view a gift as a good use of money, but they got you a gift as a kind gesture. You should be grateful that they thought of you and wanted to do something nice. If you can't accept a gift, people will think you're a cold person who doesn't like other people.
Maybe you're right, I'm just being a little too much of a scrooge.
Sometimes it's good to be humble, but not that far! It's great you can acknowledge that someone appreciates you well even if you don't agree with how they do it.