Store UserSecretsId in MSBuild project file instead of project.json

[natemcmaster]

From @muratg on August 24, 2016 22:43

Slightly related #62.

Since project.json is going away, we'll need to resolve this soon.

@glennc @davidfowl @natemcmaster @Tratcher

Copied from original issue: aspnet/UserSecrets#96

[natemcmaster]

Just brainstorming: here are some options

• A special purpose setting file,usersecrets.json, in project root. Works the same as project.json does today. Ms.Ext.Config.UserSecrets extends the build target to ensure the file is copied to build folder.
• A general setting file, like appsettings.json (or web.config )
• A well-known MSBuild property <UserSecretId>XXXX-XXXX</UserSecretId>
  • Requires adding a target to generate a resource for config to read at runtime such as
    • an assembly attribute
    • a config file in app base dir
  • CLI tool
    • runs inside build: dotnet user-secrets becomes alias for dotnet msbuild /t:UserSecrets
    • otherwise, CLI tool hosts MSBuild and does a design time build to extract the property value. (See early implementation of something similar in dotnet-ef)

[natemcmaster]

From @glennc on August 30, 2016 6:4

msbuild seems the best option here. But why does there need to be a special file generated? Why can there not be a config source that reads from a csproj?

It is just data in the csproj.

[natemcmaster]

From @Tratcher on August 30, 2016 14:14

The csproj isn't published

[natemcmaster]

IMO keeping the id in a simple file such as usersecrets.json or .usersecretrc is the better option. Adding as a property in MSBuild is going to be more complicated and have little benefit

[natemcmaster]

From @davidfowl on August 30, 2016 16:44

What about appsettings.json and we pass the filename into the AddUserSecrets. It's a bit of a breaking change though but it seems like a cleaner approach

[natemcmaster]

Did a quick spike: here's what something like usersecrets.json would look like https://github.com/aspnet/UserSecrets/compare/namc/usersecrets.json.

I'm guessing some may complain "why the extra file in my project?". If users care, we could overload AddUserSecrets to allow specifying a path to appsettings.json or something else.

[natemcmaster]

Design from in-person discussion with @glennc and @davidfowl:

1. The value of userSecretId can be in any json file. Templates and tools will default to a file named appsettings.json. Users could configure it to 'project.json' if they still want

   {
      "userSecretId": "xxxx-xxxx"
   }

2. [assembly: UserSecretIdentifierFileNameAttribute("appsettings.json")] will be generated during build using a task imported from the Microsoft.Extensions.Configuration.UserSecrets nupkg. The value of the attribute is configured from a property in the csproj. This will be in templates. It will default to appsettings.json if not present.

   <PropertyGroup>
       <UserSecretIdentifierFile>appsettings.json</UserSecretIdentifierFile>
   </PropertyGroup>

3. Runtime: .AddUserSecrets() will look for an assembly attribute, Assembly.GetEntryAssembly().GetCustomAttribute<UserSecretIdentifierFileNameAttribute>().FileName and read userSecretId from the json file. As with other config files, we will assume the path is relative to ConfigurationBuilder.GetFileProvider().

4. Design-time tooling: VS can identify which file to use by evaluating the csproj. CLI can take a command line flag to point to a different JSON file.

[natemcmaster]

From @muratg on September 13, 2016 17:31

Will the attribute be a top-level attribute in the Json file (as it is today?)

[natemcmaster]

Yes. The JSON part of this doesn't change much. We are just going to remove the strict requirement that the JSON file be named project.json. (See #103)

[natemcmaster]

The plan of record is that .AddUserSecrets() will default to looking for the assembly attribute Microsoft.Extensions.Configuration.UserSecrets.UserSecretsIdAttribute on the entry assembly. For LTS users, the method will continue to fallback to PJ if the attribute is not available.

Most users will want this attribute to be auto-generated from the MSBuild property "UserSecretsId":

Usage:

    <PropertyGroup>
       <UserSecretsId>XYZ</UserSecretsId>
    </PropertyGroup>

Microsoft.Extensions.Configuration.UserSecrets 1.0.1 and 1.1 add support for this MSBuild prop and the assembly attribute. aspnet/Configuration#525

Microsoft.Extensions.SecretManager.Tools will be upgraded soon to understand csproj. #178

dotnet-migrate will automatically move the "userSecretsId" property in project.json to the csproj. dotnet/cli#4326

[hlubovac]

Before I decided to finally ask the question that follows, I looked at the source code to find references to what you're talking about. The problem that I'm trying to solve for myself is storing the userSecretsId outside of project.json, preferably totally abstracted away from the SecretManager.Tools package, so that it's [alternatively to its known locations] delivered to the tool via CLI arguments. I can't see that the tool code supports that (

DotNetTools/src/Microsoft.Extensions.SecretManager.Tools/Program.cs

Line 137 in 11bbd6d

var userSecretsId = ResolveUserSecretsId(options);

), although it doesn't seem to me that it'd be hard to implement (whatever your opinion may be related to conveniences of usage of that tool in that case; alluding to separation-of-concerns here).

The SecretManager's AddUserSecrets method already accepts userSecretsId as an alternative to assumption that that value lives in project.json, which is good. That enables me to store it where I want to (file, env-var, to mention a few). So, what's missing now it the Tools package to follow, as opposed to continue looking for it in project.json.

I couldn't find that asm attribute (UserSecretIdentifierFileNameAttribute) to even try pointing to another file, which makes me curious if what you're talking about made it to the repository or if it lives in another package (I don't see why it would, though).

Unrelated, but, IMO, this secret-identifier is a part of configuration, and it shouldn't even be discussed - beyond config-transformations - how to automate moving it via MSBuild: that value, along with associated "secrets" shouldn't cross business-environment boundaries anyway and they all should be defined manually, one time per machine-environment pair.

Thanks.

[natemcmaster]

@hlubovac This is a WIP. I've added --id <ID> to the CLI tool in the feature/msbuild branch. (see

DotNetTools/src/Microsoft.Extensions.SecretManager.Tools/Internal/CommandLineOptions.cs

Lines 42 to 45 in 27fc53e

	// the escape hatch if project evaluation fails, or if users want to alter a secret store other than the one
	// in the current project
	var optionId = app.Option("--id", "The user secret id to use.",
	CommandOptionType.SingleValue, inherited: true);

). UserSecretIdentifierFileNameAttribute was never implemented. We decided to go with the plan in my most recent comment: #169 (comment)

this secret-identifier is a part of configuration

The actual content of the ID doesn't matter for most people. But the problem we're trying to address is "where is it stored?" We chose the project file because it unifies the Visual Studio, CLI tools, and the runtime experience.

[hlubovac]

Thanks. Do you think it'll be long before that change makes it to master/nuget?

[natemcmaster]

I don't have a date. We're waiting on dotnet/cli 's MSBuild to stabilize. But, '--id' would be easy enough to add to the version on dev branch that I'd happily take a PR to added it now if it'll unblock you.

[hlubovac]

No, it's fine; I can wait. I'm experimenting anyway, with no particular plan in mind. I don't want to distract you from your current goal. Thank you, though.

[hlubovac]

A hack, as an alternative solution to storing this value in a separate file:

1. create a subdirectory within your project (e.g. "configuration")
2. create blah\project.json file; add userSecretsId value to it using established format
3. This and similar should then work:
   dotnet user-secrets list --project "[full-path-to]\configuration"

This can further be expanded [by app devs]:

1. Create more user-friendly tools to clear, list, remove and set secrets; e.g. these could be simple 4 batch files, 2 of which receive some input (I might do exactly this, and could share those here, if that's ok); place those batch files/tools in this directory for easy access by developer
2. Unrelated, but since this could become your app's config directory, all runtime config files could be there also (hint: ConfigurationBuilder.AddJsonFile)

---

By the way, just to make it clearer where I'm going with this: I started treating this system [of managing secrets] as more like per-user-non-source-controlled setting-management, with the plan on overriding whatever I have in my app's config file (e.g. appsettings.json) with those "secret" values. This is a one-time setup per dev-workstation, and also per shared server; nobody's going to be stepping on each other's toes and I don't have to have mysterious config transformations going on. Just sharing... :)

[natemcmaster]

One of the unintended consequences of this design #169 (comment) is aspnet/Configuration#543. Any design-time tooling that invokes 'Startup' is likely to break.

[natemcmaster]

The work for this is done. When released, the feature will be in the following packages:

Microsoft.Extensions.SecretManager.Tools (1.0.0-msbuild1-final)
Microsoft.Extensions.Configuration.UserSecrets (≥ 1.0.1)