New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When using skills with user assigned managed identity , the msi endpoint fails with a 500 #6778
Comments
Hi @ceciliaavila @tracyboehrer would it be possible to triage this item? This is very high priority for our customers |
@jamesemann Was this previously working in production with MSI? |
Yes it was @tracyboehrer |
@jamesemann When did it stop? Any updates to the SDK version? |
@tracyboehrer i first spotted it yesterday as it is the first time I’ve provisioned a skill for a few weeks. No recent updates to the sdk version |
@tracyboehrer apologies I was away from the computer yesterday so couldn't confirm the exact version. I've checked and:
(I see the same behaviour in both) |
@jamesemann Ya. I wouldn't expect any changes between those two versions. Was trying to isolate if a major jump in SDK version had been made. Like from 4.18 to 4.22 or something larger. We will get setup to repro. Worth noting, we haven't made any explicit changes to this. But dependency changes can be wicked on occasion. |
@jamesemann How did you deploy the bots? One of us encounters a failure that matches your screenshot when they used AZ commands. I used the ARM Templates (and associated doc) and this sample appears to work normally. |
@tracyboehrer thank you - interesting info. I created them through the Azure portal - in our product we use the Arm templates (through a template spec) though and the properties look the same. Let me deploy a new set of resources for the sample using the Arm templates in the doc, and re-test. I'll report back as soon as possible |
@jamesemann We should be able to compare here too. |
@tracyboehrer unfortunately I'm getting the same error (HTTP 500 when requesting the managed identity token) after provisioning the resources using the templates in the bot builder repo. It seems to be a global problem for me. One thing I haven't tried is deploying to a different Azure subscription, so I'll try that next. I'll report back with the result of that. |
Update - same result using the arm templates on new subscription |
@tracyboehrer I've found the underlying error when we see the HTTP 500. It is visible in the managed identity sign-in logs in Azure AD/Entra
(I can share the activity details privately, if necessary) |
@jamesemann Still conferring with some about this. |
@tracyboehrer any luck on this? One question I did have was that the sample worked for you - did you use an existing managed identity or create a new one? One thing I have noticed is that this seems to be a problem only with recently created managed identities. We have a lot of existing managed identities for other customers and are not seeing the same behaviour. |
@jamesemann I created new ones, which is what the ARM templates do. I have confirmed that our support folks can't get the Skill to work. It remains a mystery why mine do. I have confirmed both Root and Skill are MSI. |
Version
Latest
Describe the bug
When using user assigned managed identity in azure app service, invoking a skill fails when requesting a token from the msi endpoint (
http://127.0.0.1:41120/msi/token/?api-version=2019-08-01&resource=<redacted>&client_id=<redacted>
) - which returns a HTTP 500 (with no detail) followed by the error below:To Reproduce
This is happening in our production system (where previously provisioned bots have been working), I have also reproduced it with a minimal repro on the bot builder skills sample
Clone this repo and open the
samples/csharp_dotnetcore/80.skills-simple-bot-to-bot
You will need to setup the following in Azure (using the standard User assigned managed identity stuff)
AZURE_CLIENT_ID
application setting to point to the root bot managed identity client id .AZURE_CLIENT_ID
application setting to point to the child bot managed identity client id .EchoSkillBot
(which will be deployed to the child webapp) toSimpleRootBot
(which will be deployed to the root webapp) toskill
, you will see the followingℹ️ If you look in the parent appinsights, you will see the error
Expected behavior
It should authenticate and communicate with the skill as expected.
Additional context
Please reach out if you need any further info, screen shares etc.
This is impacting a few of our customers, so is high priority for us. Thank you!
The text was updated successfully, but these errors were encountered: