Add pod identities to Azure Key Vault provider

[JorTurFer]

Proposal

At this moment, the Azure Key Vault provider code supports pod identities but it's not documented anywhere. There is also a race condition in the way that is implemented because it uses the section podIdentity but scalers use that section too, so depending on scaler implementation will take secrets from the vault or use pod identity, we should redesign this, maybe placing that section inside the key vault block.
There is another problem because secret is required and we shouldn't require it if we support pod identity.

Use-Case

No response

Anything else?

No response

[v-shenoy]

I didn't take into account the race-condition. I thought if a TA had a Key Vault and pod identity, any Azure scalers would still work with the pod identity.

In my head the main use-case for this was, if a user had a non-Azure scaler in use such as RabbitMQ. We will use pod identity to access the key vault, but the scaler itself will use the secrets stored in the key vault.

[JorTurFer]

The use case how I discovered this was:
We have a Key Vault to store all the secrets and an identity with access to that Vault, when I tried to use the identity, the EventHub scaler jumps to pod identity in this switch, but the manage identity we use doesn0t have access to the eventhub or storage, only to the Key Vault.
We must use connection strings instead of manage identities in our resources for the exit plan, managed identities have vendor lock and for instance, we use kafka api to produce/consume events in the hub. I guess that there could be more people in our situation, and we can't assume that managed identities could be used for all. From apps pov, using or not a managed identity in KEDA doesn't change their behaviour and either the "architecture" of the application/secret management

[v-shenoy]

Sorry about the bug, but it's nice that you were able to discover it before some user did.

[JorTurFer]

Sorry about the bug, but it's nice that you were able to discover it before some user did.

No worries at all, my first contribution broke the rabbitmq scaler... it happens 😉😉
You do an awesome job ❤️
I guess end users haven't noticed it because it isn't documented, so maybe they don't know it supported that feature. I saw in the code when I was checking if it's supported or not