DataProtection Keys folder doesn't sync on Azure Web Sites distribution slots

[tmm360]

Cloned from dotnet/aspnetcore#1582 by suggest of @guardrex. @blowdart

I'm running an Asp.Net Core RC2 application on Azure Web Sites, and I noted that every time that I switch staging and production environments, all users log off. So I've tried to test what happened, and I've discovered that on staging and production's "%HOME%\ASP.NET\DataProtection-Keys" folders the keys were different. Reading this page https://docs.asp.net/en/latest/security/data-protection/configuration/default-settings.html they should be synced.

I supposed that keys may be synced by slot, so when I switch slots the keys should keep under original environment. To test this I've created a new Azure web site with two new empty slots, but when I switch environments the "%HOME%\ASP.NET" folder follows its server. This implies that keys of staging are used under production, and keys of production are used under staging. I've discovered the origin of my issue.

Two options here: I didn't understand how DataProtection is synced between distribution slots, and in this case I'm asking you to tell me how I can solve this problem, or this is a bug and the sentence on documentation is wrong. What do you think? Thank you

[guardrex]

@blowdart If the answer is that the keys will follow the app as slots are swapped, then would something like this cover it ...

If the system is being hosted in Azure Web Sites, keys are persisted to the "%HOME%\ASP.NET\DataProtection-Keys" folder. This folder is backed by network storage and is synchronized across all machines hosting the application. This folder is created for each slot independently, so swapping an application's slot will move the application's keys with it. Keys are not protected at rest.

Also, it looks like they renamed the service slightly to "Azure Apps," "Azure App Service," or "Azure Web App." I thought "Azure Web Sites" was dropped at some point.

@tmm360 You should be able get some fine-grained control over this situation by using one of the other key ring providers, like Redis, Azure Blob Storage, or Azure Key Vault. Then, you could specify in startup which key store to use based on the status (staging/production) of the app.

[tmm360]

@guardrex Thank you, I've found the solution with the new 1.1 release. Because this I had closed the other issue.

[guardrex]

@tmm360 Excellent. 👍 Yeah, let's hold on a sec here to see if @blowdart thinks a statement there about the key ring going along with the slot would be a good add. He'll get back to us soon.

[blowdart]

We've ask azure for keys that flow between slots. It's on their backlog with no scheduled implementation time. It is a limitation of azure web apps. If you're wanting slots use blob storage, redis or implement a sql store suitable for your database schema.

[guardrex]

@blowdart ... then you want to leave the doc as it is? If so, @tmm360 can close.

[blowdart]

We should call out that keys don't flow over azure slots. I'm in meetings so I don't know what the doc says right now.

[guardrex]

I was a bit confused tho by the OP description of its behavior. I think I'm clear on it now.

The relevant section is (1) at https://github.com/aspnet/Docs/blob/master/aspnetcore/security/data-protection/configuration/default-settings.md#key-management

Helpful language might be something like ...

1. If the system is hosted in Azure Web Apps, keys are persisted to the "%HOME%\ASP.NET\DataProtection-Keys" folder. This folder is backed by network storage and is synchronized across all machines hosting the application. This folder supplies the key ring to all of the system's slots, so swapping an application's slot will not change the key ring used. If you desire slot-dependent key rings, use an external key ring provider, such as Azure Blob Storage, Azure Key Vault, a SQL store, or Redis cache. Keys are not protected at rest.

[blowdart]

Wrong way around :)

This folder supplies the key ring to all instances of an application in a single deployment slot. Seperate deployment slots such as Staging and Production will not share a keyring. When you swap between deployment slots, for example swapping staging to production, or using A/B testing any system using data protection will not be able to decrypt stored data using the keyring inside the previous slot. This will, for example, lead to users being logged out of an ASP.NET application which uses the standard ASP.NET cookie middleware, as it uses data protection to protect its cookies. If you desire slot-independent key rings, shared between multiple slots, use an external key ring provider, such as Azure Blob Storage, Azure Key Vault, a SQL store, or Redis cache.

Does that cover it @tmm360 ?

[tmm360]

Yes, it does! Thank you.

[guardrex]

Ah ... that's what I originally had in my first response. I guess I did understand the OP after all. 😄

[tmm360]

Yes, you did ;) Maybe my question was not very well exposed...

[blowdart]

This is the joy of reading issues in email :) @guardrex do you want to submit the doc update? We're still suffering from the effects of moving buildings

[guardrex]

@tmm360 Nah ... it's me. I'm a goofy green dinosaur! lol

@blowdart Yeah ... I'll take care of it right now. I'll use that text block you just provided and ping you on the PR.

@tmm360 No need to sweat closing this. The issue will auto-close when the PR goes in. Thanks for mentioning this. I'm just a community guy, but I'm sorry someone didn't catch your issue sooner. It was a good issue ... things are just crazy busy these days with .NET, and it slipped thru the cracks.

[tmm360]

Eheh, thank you!