I don't want to pay towards the birthday gift
Is there a specific reason why you don't want to pay?
Because the gift that I ordered with him before for his birthday was being with least quality least expensive that I ordered it from.
If it's a group gift and you want your name attached to it then you are expected to chip in. You shouldn't expect to be included in the gift if you don't want to put in money on it.
Can't someone else chip in?
That would be selfish. Is it really that important to have your name on a gift that you aren't chipping in for?