Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

using appsettings.json + IConfiguration in Function App #4464

Closed
kemmis opened this issue May 17, 2019 · 37 comments
Closed

using appsettings.json + IConfiguration in Function App #4464

kemmis opened this issue May 17, 2019 · 37 comments
Labels
Needs: Attention 👋

Comments

@kemmis
Copy link

kemmis commented May 17, 2019

Is your question related to a specific version? If so, please specify:

Yes - the current version - that supports DI.

What language does your question apply to? (e.g. C#, JavaScript, Java, All)

C#

Question

In my asp.net core web apps I'm used to loading in appsettings.json and environment-specific appsettings.{environment}.json in on startup using the ConfigureAppConfiguration method on IWebHostBuilder. With the new FunctionsStartup & IFunctionsHostBuilder, how should one go about loading appsettings in, assuming they want to access the settings by having an IConfiguration injected into their Function using standard .net core DI functionality?

For example, what is the equivalent way of loading in appsettings.json now like I do here using IWebHostBuilder?

public static IWebHost BuildWebHost(string[] args)
{
    var host = WebHost.CreateDefaultBuilder(args)                                                           
        .ConfigureAppConfiguration((context, builder) =>
        {
            var tmpConfig = new ConfigurationBuilder()
                .AddJsonFile("appsettings.json")
                .Build();

            var environmentName = tmpConfig["Environment"];

            builder.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
                .AddJsonFile($"appsettings.{environmentName}.json", optional: true, reloadOnChange: true);
            
        })
        .UseStartup<Startup>()
        .Build();

    return host;
}
@kemmis
Copy link
Author

kemmis commented May 21, 2019 •
edited
Loading

FYI - here is what I ended up doing... I call this in my Startup.Configure() method:

static class IFunctionsHostBuilderConfigurationExtensions
{
    public static IFunctionsHostBuilder AddAppSettingsToConfiguration(this IFunctionsHostBuilder builder)
    {
        var currentDirectory = "/home/site/wwwroot";
        bool isLocal = string.IsNullOrEmpty(Environment.GetEnvironmentVariable("WEBSITE_INSTANCE_ID"));
        if (isLocal)
        {
            currentDirectory = Environment.CurrentDirectory;
        }

        var tmpConfig = new ConfigurationBuilder()
            .SetBasePath(currentDirectory)
            .AddJsonFile("appsettings.json")
            .Build();

        var environmentName = tmpConfig["Environment"];

        var configurationBuilder = new ConfigurationBuilder();

        var descriptor = builder.Services.FirstOrDefault(d => d.ServiceType == typeof(IConfiguration));
        if (descriptor?.ImplementationInstance is IConfiguration configRoot)
        {
            configurationBuilder.AddConfiguration(configRoot);
        }

        var configuration = configurationBuilder.SetBasePath(currentDirectory)
            .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
            .AddJsonFile($"appsettings.{environmentName}.json", optional: true, reloadOnChange: true)
            .AddAzureKeyVault()
            .Build();

        builder.Services.Replace(ServiceDescriptor.Singleton(typeof(IConfiguration), configuration));

        return builder;
    }
}

UPDATE: Had to add logic to access the actual appsettings.json location when deployed in Azure. It would run fine locally, but wasn't able to find the appsettings files when it was deployed.

@miladghafoori
Copy link

I have the same issue as well. In the context of a Function, an ExecutionContext object is injected from which I can access FunctionAppDirectory . It would be really helpful if a similar object is made available in the FunctionsStartup so that all configuration and DI can be done in the startup.

@Arash-Sabet
Copy link

It's a great idea to bake this approach in FunctionsStartup as @miladghafoori said.

@espray espray mentioned this issue Jun 1, 2019
Closed
@espray espray mentioned this issue Jun 18, 2019
Open
@KalyanChanumolu-MSFT
Copy link

A Visual Studio project template similar to the asp.net core web/api app with the startup.cs and appsettings.json put together would be of great help than having to create these manually.

This was referenced Jun 19, 2019
Closed
Closed
@yangyadi1993
Copy link

Another thought is that we could make a wrapper class of IConfiguration or IConfigurationRoot, in that way we could separate the logic of application settings and Azure function settings. In the dependency injection, we could only inject the wrapper class as a new type. We don't need to replace.

@gzuber gzuber added this to the Triaged milestone Jul 15, 2019
@gzuber
Copy link
Member

gzuber commented Jul 15, 2019

Sorry for the delayed response here. Currently, we don't have an accepted pattern/any guidance on accomplishing this. The workaround provided by @kemmis will work if needed. In the future, we do have plans to allow loading in configuration files out of the box without the need for custom code.

@gzuber gzuber closed this as completed Jul 15, 2019
@KalyanChanumolu-MSFT
Copy link

@gzuber Please provide an Issue# for us to follow and know when it is available

@kemmis
Copy link
Author

kemmis commented Jul 17, 2019

FYI - I updated my example above so that it should find the correct appsettings.json file when running in Azure. Using Environment.CurrentDirectory was only working when I ran the function locally, but not in Azure.

@PureKrome
Copy link

@kemmis nice update. I'd also add an extra explanation as to the differences between localhost and azure settings/file locations (e.g. in the bin folder vs not in the bin folder but a level above [which is different to how netcore apps now work]).

Also, maybe a note about making sure the appsettings.json, etc files will need to be manually 'copy always' or 'copy when newer', to make sure they get pushed up.

nice code!

@sandeepiiit
Copy link

sandeepiiit commented Jul 17, 2019 •
edited
Loading

I have done something similar to get the correct directory based on environment. I make use of AZURE_FUNCTIONS_ENVIRONMENT, AzureWebJobsScriptRoot & HOME environment variables to achieve the same.

public class Startup : FunctionsStartup
{
    public override void Configure(IFunctionsHostBuilder builder)
    {
        // Get the path to the folder that has appsettings.json and other files.
        // Note that there is a better way to get this path: ExecutionContext.FunctionAppDirectory when running inside a function. But we don't have access to the ExecutionContext here.
        // Functions team should improve this in future. It will hopefully expose FunctionAppDirectory through some other way or env variable.
        string basePath = IsDevelopmentEnvironment() ?
            Environment.GetEnvironmentVariable("AzureWebJobsScriptRoot") :
            $"{Environment.GetEnvironmentVariable("HOME")}\\site\\wwwroot";

        var config = new ConfigurationBuilder()
            .SetBasePath(basePath)
            .AddJsonFile("appsettings.json", optional: false, reloadOnChange: false)  // common settings go here.
            .AddJsonFile($"appsettings.{Environment.GetEnvironmentVariable("AZURE_FUNCTIONS_ENVIRONMENT")}.json", optional: false, reloadOnChange: false)  // environment specific settings go here
            .AddJsonFile("local.settings.json", optional: true, reloadOnChange: false)  // secrets go here. This file is excluded from source control.
            .AddEnvironmentVariables()
            .Build();

        builder.Services.AddSingleton<IConfiguration>(config);
    }

    public bool IsDevelopmentEnvironment()
    {
        return "Development".Equals(Environment.GetEnvironmentVariable("AZURE_FUNCTIONS_ENVIRONMENT"), StringComparison.OrdinalIgnoreCase);
    }
}

Update: The above code creates a new IConfiguration and ends up replacing the originally registered IConfiguration. So, I would recommend follow kemmis's pattern which takes care of adding the originally registered IConfiguration into the newly created one. However, feel free to incorporate the environment variables suggested above.

@fabiocav
Copy link
Member

I want to leave a warning here when using this approach. There's a reason this wasn't exposed with the startup. The work to make this happen is trivial in the host, but we can't perform that work in isolation.

While this works and the host will happily load configuration, if you add things like connection strings and some other trigger configuration to sources outside of the ones currently supported out of the box, key infrastructure components will not work as designed. O key example is the scale controller when running in consumption. If the controller is unable to locate configuration in the known sources, your Function App may not be activated and/or scaled. This is less of a concern in dedicated environments, but I wanted to make sure others were aware of potential problems with this and one of the key reasons why this is currently a runtime limitation.

@PureKrome
Copy link

like connection strings and some other trigger configuration to sources outside of the ones currently supported out of the box

@fabiocav can you please elaborate on this point with some examples. I don't fully understand, please 😊

@fabiocav
Copy link
Member

Sure... some infrastructure pieces, particularly in the consumption model, need to monitor trigger sources and some configuration options that control the Function App behavior. One example of such component is the scale controller.

In order to activate and scale your application, the scale controller monitors the event source to understand when work is available and when demand increases or decreases. Using Service Bus as an example; the scale controller will monitor topics or queues used by Service Bus triggers, inspecting the queue/topic lengths and reacting based on that information. To accomplish this task, that component (which runs as part of the App Service infrastructure) needs the connection string for each Service Bus trigger setup and, today, it knows how to get that information from the sources we support. Any configuration coming from other providers/sources is not visible to the infrastructure outside of the runtime.

Hope the above is sufficient to clarify what I stated above, but let me know if you have any more questions.

@PureKrome
Copy link

OK - that does help me understand (and I hope, others also :) )

So to clarify ...

Azure Functions has some hardcoded conventions for configuration with respect to the built in 'stuff' like triggers. This convention is to use hosts.json (docs ref) for application settings (like queue settings or logging settings) and local.settings.json for app settings and connection strings that are used when running locally (live uses the portal to set these things).

But .. if we have our own business logic settings we could leverage our own custom settings files, like appsettings.json / appsettings.<environment>.json. If we choose to do this, then we need to understand that these files are for our own business logic and not for azure function settings stuff AND they'll require some custom code to be parsed/used in the azure functions c# code.

Finally, when using the portal to set settings (via the functions Application Settings section), this ends up adding theses key/values to Environmental Variables, which get used by the Functions infrastructure.

ref:
image

Okay .. was that about it?

@mbusila
Copy link

mbusila commented Jul 25, 2019

To accomplish this task, that component (which runs as part of the App Service infrastructure) needs the connection string for each Service Bus trigger setup and, today, it knows how to get that information from the sources we support.

@fabiocav can you enumerate the list of sources supported? Is Azure Key Vault part of it?

@kemmis
Copy link
Author

kemmis commented Jul 25, 2019 via email

@espray espray mentioned this issue Jul 28, 2019
Closed
@brettsam brettsam mentioned this issue Aug 5, 2019
Closed
@scottluskcis
Copy link

based on the example from @kemmis I've added some extension methods to a repo to approach this is a similar way

check out this repo

using extensions methods file like @kemmis in repo at IFunctionsHostBuilderConfigurationsExtensions.cs

If you look at Startup.cs you will see I'm calling an extension method and passing Func<T, TResult> to apply settings to IConfigurationBuilder

Example:

            builder.AddConfiguration((configBuilder) =>
            {
                var envName = Environment.GetEnvironmentVariable("ENVIRONMENT_NAME");

                // use IConfigurationBuilder like you typically would
                var configuration = configBuilder
                    .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
                    .AddJsonFile($"appsettings.{envName}.json", true, true)
                    .TryAddAzureKeyVault(Environment.GetEnvironmentVariable("VAULT_NAME"))
                    .AddEnvironmentVariables()
                    .Build();
                
                return configuration;
            });

thanks @kemmis for the insights, if anyone has additional thoughts would love to hear any feedback. great to see DI in Azure Functions now, eager to see it continue to evolve.

@lukasvosyka
Copy link

Hi,

I had a similar issue with this, but I think passing the IConfiguration around as DI injected something is not a good idea in general. Rather than that, I would suggest to go for IOptions injected setting models.

You can then use this for example together with the Configuration in the Functions portal. Then you just build a local config that you do not register within the DI pipeline, but you can use it to configure your options.

public override void Configure(IFunctionsHostBuilder builder)
{
	ConfigureServices(builder.Services);
}

private void ConfigureServices(IServiceCollection services)
{
	var localConfig = new ConfigurationBuilder()
		.AddEnvironmentVariables()
		.Build();

	services.Configure<ImageServiceSettings>(localConfig.GetSection("ImagesServiceSettings"));
	services.AddTransient<ImageResizeService>();
}

You can surely also use an appsettings.json file, if you want to.

The service would then get something like:

public class ImageResizeService
{
	private ILogger _logger;
	private ImageServiceSettings _settings;

	public ImageResizeService(ILogger<ImageResizeService> logger, IOptions<ImageServiceSettings> settings)
	{
		_logger = logger;
		_settings = settings.Value;
	}
       
        // some code
}

Hope it helps someone. The important part is to not register the IConfiguration with the local configuration as also you host.json stuff (and everything @fabiocav mentioned) gets overwritten.

@jedmonsanto
Copy link

i tried this @lukasvosyka and in my application all value is null,

Startup
image

Function Trigger
image

Azure Portal
image

But it works fine in my local.

@lukasvosyka
Copy link

@jedmonsanto Maybe your BasePath is set wrong for Azure functions, when deployed. Try the suggested way as described above. Something like


 var currentDirectory = "/home/site/wwwroot";
        bool isLocal = string.IsNullOrEmpty(Environment.GetEnvironmentVariable("WEBSITE_INSTANCE_ID"));
        if (isLocal)
        {
            currentDirectory = Environment.CurrentDirectory;
        }

var tmpConfig = new ConfigurationBuilder()
            .SetBasePath(currentDirectory)
            .AddJsonFile("appsettings.json")
            .Build();

@jedmonsanto
Copy link

@lukasvosyka no luck, what exactly the basepath in azure function app?
i've seen this below:
image

@joaoantunes
Copy link

joaoantunes commented Sep 25, 2019 •
edited
Loading

I've used the above approach to set a specific JSON file for configurations: appsettings.json and it works locally. In order to work on azure, I've set the base path using this:

private static string GetCurrentDirectory(IFunctionsHostBuilder builder)
{
	ExecutionContextOptions executionContextOptions = builder.Services.BuildServiceProvider()
		.GetService<IOptions<ExecutionContextOptions>>().Value;
	return executionContextOptions.AppDirectory;
}

But when I try to use the function on azure it says:

Error:

The function runtime is unable to start. Microsoft.Extensions.Configuration.FileExtensions: The configuration file 'appsettings.json' was not found and is not optional. The physical path is 'D:\home\site\wwwroot\appsettings.json'.
Session Id: fe4614558cdf459d94749746b8cfbc6b

Probably appsettings.json is not publishing to azure function app. The method that I'm using to deploy is just the "Publish" from VS2019. How can put the config file on: D:\home\site\wwwroot\appsettings.json? Can I only do this by using ARM Templates?

@janus007
Copy link

janus007 commented Oct 1, 2019

Don't use appsettings.json , use Azure App Configuration instead. IMHO much better and easy to configure.

@kemmis
Copy link
Author

kemmis commented Oct 1, 2019 via email

@kemmis
Copy link
Author

kemmis commented Oct 2, 2019 via email

@janus007
Copy link

janus007 commented Oct 2, 2019

Yeah.. I know the battles as well. Sometimes things just are the way they are...

@joaoantunes
Copy link

Ok regarding my issue on the top, I solved it by adding to .csproj where I was doing the publish, the following information:

<ItemGroup>
  <None Update="appsettings.json">
    <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    <CopyToPublishDirectory>Always</CopyToPublishDirectory>
  </None>
</ItemGroup>

Now everytime I publish the azure function it sends the appsettings.json also!

@MikeGriffinReborn
Copy link

MikeGriffinReborn commented Oct 11, 2019 •
edited
Loading

This is what I did, it works just fine, locally or on Azure. I actually set the "Copy to Output Directory" property on my local.settings.json in Visual Studio to "Copy if newer". The GetCustomSettingsPath() method ensure it's runs on azure or locally. This is a very easy approach that works fine.

public static partial class MyFunctionClass
{
    static internal IConfigurationRoot Config = null;

    static MyFunctionClass()
    {
        Config = new ConfigurationBuilder()
            .SetBasePath(GetCustomSettingsPath())
            .AddJsonFile("local.settings.json", optional: true, reloadOnChange: true)
            .AddEnvironmentVariables()
            .Build();
    }

    // This is for Portal Support and requires no Auth token
    [FunctionName("IsAlive")]
    public static async Task<HttpResponseMessage> IsAlive() 
    {
        // a function method
        return null;
    }

    private static string GetCustomSettingsPath()
    {
        string path = "";

        // This works on Azure or when running locally ...
        string home = Environment.GetEnvironmentVariable("HOME");
        if (home != null)
        {
            // We're on Azure
            path = Path.Combine(home, "site", "wwwroot");
        }
        else
        {
            // Running locally
            path = new Uri(Assembly.GetExecutingAssembly().CodeBase).LocalPath;
            path = Path.GetDirectoryName(path);
            DirectoryInfo parentDir = Directory.GetParent(path);
            path = parentDir.FullName;
        }

        return path;
    }
}

@FabienColoignier
Copy link

FabienColoignier commented Oct 16, 2019 •
edited
Loading

You can use this piece of code in your startup file.
I've just tested it today for my project and it works on both cloud and local

var executioncontextoptions = builder.Services.BuildServiceProvider()
    .GetService<IOptions<ExecutionContextOptions>>().Value;
var currentDirectory = executioncontextoptions.AppDirectory;

var config = new ConfigurationBuilder()
   .SetBasePath(currentDirectory)
   .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
   .AddEnvironmentVariables()
   .Build();

@joeyeng
Copy link

joeyeng commented Oct 23, 2019 •
edited
Loading

Another thought is that we could make a wrapper class of IConfiguration or IConfigurationRoot, in that way we could separate the logic of application settings and Azure function settings. In the dependency injection, we could only inject the wrapper class as a new type. We don't need to replace.

Is this an option or is the actual act of loading the appsettings file with the ConfigurationBuilder what causes the issues @fabiocav mentioned? Wasn't clear if that's the issue or it's adding the resulting IConfiguration to the service collection for DI. My assumption, is the latter, but can someone confirm?

@SaebAmini
Copy link

Brilliant thinking @FabienColoignier! thanks for sharing that.

@martinfletcher
Copy link

I know that this issue is closed, but it is the one that keeps hitting Google result 1 - So i will put it here!

@lukasvosyka - Not sure if you still have this problem?

I have been experiencing the same issue trying to add custom configuration for a functions v2 app via DI - As you reported, the host.json variable values get overwritten when registering a custom IConfiguration...

The below will ensure that all configurations are kept:

public void ConfigureServices(IServiceCollection services)
{
    var providers = new List<IConfigurationProvider>();

    foreach(var descriptor in services.Where(descriptor => descriptor.ServiceType == typeof(IConfiguration)).ToList())
    {
        var existingConfiguration = descriptor.ImplementationInstance as IConfigurationRoot;

        if(existingConfiguration is null)
        {
            continue;
        }

        providers.AddRange(existingConfiguration.Providers);

        services.Remove(descriptor);
    }

    var configuration = new ConfigurationBuilder();

    // Add custom configuration to builder

    providers.AddRange(configuration.Build().Providers);

    services.AddSingleton<IConfiguration>(new ConfigurationRoot(providers));
}

@kijujjav
Copy link

@martinfletcher , Would you mind sharing your Startup.cs file. I am facing the same problem where after injecting IConfiguration all the host.json values are lost.

@martinfletcher
Copy link

@kijujjav - Adding the above should be all you need. Can you post your startup file and I can take a look to see what might be your issue?

@kijujjav
Copy link

kijujjav commented Nov 15, 2019 •
edited
Loading 

Its working with below code. Thanks @martinfletcher 

 public class Startup : FunctionsStartup
    {
        /// <summary>
        /// This method is is for depedency injection of config files
        /// </summary>
        /// <param name="builder"></param>
        public override void Configure(IFunctionsHostBuilder `builder)`
        { 
            ConfigureServices(builder.Services);
        }

        public void ConfigureServices(IServiceCollection services)
        {
            var providers = new List<IConfigurationProvider>();

            foreach (var descriptor in services.Where(descriptor => descriptor.ServiceType == typeof(IConfiguration)).ToList())
            {
                var existingConfiguration = descriptor.ImplementationInstance as IConfigurationRoot;
                if (existingConfiguration is null)
                {
                    continue;
                }
                providers.AddRange(existingConfiguration.Providers);
                services.Remove(descriptor);
            }

            var executioncontextoptions = services.BuildServiceProvider()
                .GetService<IOptions<ExecutionContextOptions>>().Value;
            var currentDirectory = executioncontextoptions.AppDirectory;

            var config = new ConfigurationBuilder()
                .SetBasePath(currentDirectory)
                .AddJsonFile("appSettings.json", optional: false, reloadOnChange: true)
                .AddEnvironmentVariables();

            providers.AddRange(config.Build().Providers);

            services.AddSingleton<IConfiguration>(new ConfigurationRoot(providers));
        }
    }

@Leon99
Copy link

Leon99 commented Nov 27, 2019

@gzuber why is this closed if you're planning to implement it and there is no other issue open to track it?

@ghost ghost locked as resolved and limited conversation to collaborators Dec 31, 2019
@jeffhollan
Copy link

Just came across this - hopefully this issue was resolved via this feature that allows loading in configuration sources: https://docs.microsoft.com/en-us/azure/azure-functions/functions-dotnet-dependency-injection#customizing-configuration-sources

Feel free to open a new issue if there are capabilities this is missing (sometimes hard to have mega-issues that span a few stuff, so keeping closed to focus on still remaining work needed)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Needs: Attention 👋
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests