Simplify keep_in_fp32_modules logic

[Cyrilvallez]

What does this PR do?

As per the title. It's easier to use a regex from the beginning compared to start with a list, then switch to regex near the end of the loading logic.
Also, make sure that only full layer names can be matched with the regex. Otherwise some old and bad layer names such as wo in T5 could be match against layers such as word_embedding, which contains wo.

Also, use the flag only when loading in fp16 or using a quantizer expecting it, as _keep_in_fp32_modules was introduced only to avoid issues when casting bf16 -> fp16. See #20287 for details. That is, the layers should not always be kept in fp32, despite the name of the flag... I added a detailed comment about that because it is not clear at all otherwise

[github-actions]

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. When it is ready for review, please click the Ready for review button (at the bottom of the PR page).

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ArthurZucker]

Nice! One thing that is missing: a check to enforce at init time, like the TP plan, that the layers actually exist in the model?
I think for Blip2 and T5 there might be weirdness, but at least show a warning !

[ArthurZucker]

Also some documentation is missing + maybe deprecate nion re pattern no?

[Cyrilvallez]

I just modified blip2 so that no model in the library can have the flag set to a non-existing layer (😮‍💨), so I don't think we actually need a check at init time - IMO it would clutter the code for something that would never happen. It can add it to be super super safe though!

[ArthurZucker]

LGTM, IMO from_pretrained should be checking the layer names to make sure typos are avoided for example