I accidentally got $34 from my bank
What happened? Do you think it's your bank's fault?
During my mental breakdown, I accidentally got $34 in there.
Now that you're feeling better mentally, you should fix that mistake. Tell them you had a lapse is judgment and I'm sure it'll be fine.
Nah, I'm going to go on a shopping spree.
While there may not be any strong repercussions about not reporting what happened, you shouldn't take advantage of the situation and use the money that was accidentally put to your bank account. It maybe someone else's money and they might really need it. If the bank corrects the mistake, then you may be $34 short in the future.