I left a kitten alone in freezing weather
Why did you leave the kitten alone in such weather?
It is not my job to save animals
You need to go back and get that kitten and save its life. You don't have to own it you can take it to a shelter and it can find a home.
Nah. It's the cats fault for getting itself into that situation.
The cat cannot be blamed for this situation. Some animals need the help of humans to survive. This kitten needs a home and you should call someone to pick it up if you cannot care for it.