[Issue]: Obfuscated LLM Fallback in OpenAIWrapper

[WaelKarkoub]

Describe the issue

I was working on modalities and different transforms and realized the issue of not knowing which LLM configuration is being used before and after the hooks are called. For example, imagine an agent with image modality support where all your transforms assume the agent has this capability. What happens if the API request fails and the OpenAIWrapper switches to an LLM that doesn't support the image modality? Will the hooks run again? How will these agent capabilities adjust to the OpenAIWrapper in this scenario?

I believe @afourney previously raised this issue, and I'm facing it as well. I was thinking maybe the agent should select which LLM config to fall back to, rather than leaving this responsibility to the OpenAIWrapper.

Steps to reproduce

No response

Screenshots and logs

No response

Additional Information

No response