Reload-on-change for the Custom EF Provider

[philip-reed]

Would be curious if there was a way to implement the reload-on-change functionality for the custom EF provider. Perhaps using SqlNotifications?

---

Document details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 0eb4d762-3910-2c57-8741-3306c15537b6
• Version Independent ID: ba2254a0-ea8e-9b28-e94b-edd0fc58c043
• Content: Configuration in ASP.NET Core
• Content Source: aspnetcore/fundamentals/configuration/index.md
• Product: aspnet-core
• GitHub Login: @guardrex
• Microsoft Alias: riande

[guardrex]

Hello @philip-reed ... Since that approach is used to maintain caches, I don't see why something couldn't be worked out.

The engineers might consider it to be too much of an edge case to show. I'll ask shortly; but first ...

Can you describe your scenario in more detail ... how do you plan to use this scenario in apps?

[philip-reed]

Hi @guardrex,

It is exactly as you state. Most of our apps live in web farms, but pull their configuration from a central SQL server. We currently have a WCF service that caches the data, and the apps use this to obtain their configuration, but this is ageing now and calling WCF from planned ASPNET Core apps just doesn't feel... appropriate. So I am experimenting with alternatives.

I've got the example custom (EF) configuration provider working, but had to switch from EntityFramework to plain old SqlDataReaders for the SqlDependency bits, not sure if I could have done it with EF? Either way I haven't got the notifications though yet so might not have something set up right. Plan was to give each of the new apps the custom configuration provider to allow them to pull in their configuration. Or create a new Web Api that other apps call to get their settings , still experimenting with ideas.

Had a look at Redis, but I don't think we could get that implemented in a reasonable time frame, and company is still shy of putting stuff on Azure so those options are out for now (I think).

[guardrex]

• SqlClient: Add query notification support to SqlClient corefx#20708
• EF Core
  • Not here ... How to: Can I listen for Changes in the underlying MS SQL Tables? efcore#8516
  • This is interesting ... How should I listen for entities being added/deleted/modified in ef core? efcore#5970
  • Lifecycle Hooks (but looks like a post-3.0 thing) ... Events and interception (aka lifecycle hooks) efcore#626

Pinging @HaoK and @Tratcher for advice on your scenario and to ask about this scenario for the EF Core Provider in our configuration topic.

[HaoK]

I would guess this isn't possible with EF until they address the lifecycle hooks issue, but adding @ajcvickers and @divega to confirm its not possible to get callbacks when something changes in the DbContext...

[guardrex]

... and @ajcvickers does this approach still work ... dotnet/efcore#5970 (comment) ... as a temp workaround perhaps for @philip-reed.

[divega]

There is no equivalent abstraction for something like SqlDependency at the EF Core level. Specifically, there is no API on EF Core that will tell you that objects in the database have changed.

I don't think dotnet/efcore#5970 was about that either. That issue was about listening to change tracking events on a specific DbContext. That is something we have in EF Core 2.1 (look for Tracked And StateChanged events on ChangeTracker), but nothing to do with changes in the database.

All that said, if you are using EF Core to store data on a SQL Server database, you should be able to use SqlDependency. The main roadblock I can think of is that there is no easy way to obtain the SQL generated for a LINQ query in EF Core, but you should be able to write your own SQL and make it work.

[guardrex]

That issue was about listening to change tracking events on a specific DbContext.

Thanks @divega ... looks like @philip-reed's scenario is that changes to the dB are on a different context (i.e., webfarm scenario), so his scenario won't permit that particular approach to work.

@HaoK It doesn't look like we have an actionable issue for the docs at this time. Do you concur? If so, we can either backlog this (if you think the EF Core provider should ... one day ... get a reload-on-change update with lifecycle hooks if/when they become available) or close this issue. Which way do you want to go?

[HaoK]

Closing seems fine, this doesn't look like something that will be possible to do anytime soon.

[guardrex]

Thanks for asking @philip-reed ... it's not actionable for the docs at this time. I hope the discussion has helped.

My hunch/hope is that posting to SO or a support chat, such as Slack or Gitter, will lead to helpful info. The devs on the ASP.NET Core Slack may have worked similar scenarios and have some war stories on implementations. If you're not already on that Slack, the self-signup is here 👉 http://tattoocoder.com/aspnet-slack-sign-up/ ... post your scenario in the #general chat or even just a link to this issue. Gitter is over here 👉 https://gitter.im/aspnet/Home

[philip-reed]

Thanks for your input everyone, very helpful. Didn't think this would get so much attention, but grateful that it has.

Thanks again.

[philip-reed]

Hopefully resurrecting this isn't an issue.

I have this working now using SqlDependency, but I'm not sure how to make sure that SqlDependency.Stop() gets called when the app shuts down.

Can anyone provide guidance on how I might do this in a custom configuration provider?

[guardrex]

Hello @philip-reed ... Ask about that on a support forum, such as Stack Overflow, or a support chat, such as Gitter or Slack. You probably won't get an answer here. There's just a handful of doc authors closely monitoring issues on this repo and working on docs issues.

[pratishthap]

Is there no other alternative to using sqldependency for getting notified of databasechanges ? How can i achive this by pull rather than push?How to i implement itokenchange.Onchanges for this approach

[philip-reed]

@pratishthap I got something working that you might find useful.

https://stackoverflow.com/q/54072129/2064829

Only issue is that SqlDependanct.Stop() never gets called.

[pratishthap]

@pratishthap I got something working that you might find useful.

https://stackoverflow.com/q/54072129/2064829

Only issue is that SqlDependanct.Stop() never gets called.

Hey Philip that for the reply. I went through this article. But i want to implement it without using SQLDependency as my application deals with both sql and Oracle. So i was hopng if there is an answer which can be used by combining ITokenChanges.OnChanges and IOptions<> and using customprovider

[Tratcher]

@pratishthap IChangeToken supports a pull model via the HasChagned property. Set ActiveChangeCallbacks = false and then you don't need to implement RegisterChangeCallback. Here's an example.

[pratishthap]

Hi

@pratishthap IChangeToken supports a pull model via the HasChagned property. Set ActiveChangeCallbacks = false and then you don't need to implement RegisterChangeCallback. Here's an example.

Hi.Thanks for your reply. One major problem i ahve facing is how to register and configure the customdatabaseprovider on startup correctly. I am using an extensions class via which i am adding the AddCustomDbConfirProvider to the IConfigurationBuilder in the startup constructor. But that config file gets lost from the IConfiguration reference once the application goes past the starup. So none of my controllers and access the customconfigprovider to access the values. How do i correctly configure and register so that the application can have access to it

[Tratcher]

Add it in Program.Main by calling ConfigureAppConfiguration on IWebHostBuilder.

[pratishthap]

Add it in Program.Main by calling ConfigureAppConfiguration on IWebHostBuilder.

I think calling it from either Startup or Program.Main() will result in the same. What i wantt is how do i see the values of the changed or current values and display it. On the first load i injected the @configuration to get to my customEFprovider and and can access values from Data . But the configuration does not have customEFPprovider after having past the startup.

[Tratcher]

Please show the code for how you're adding it in Startup.

[pratishthap]

Please show the code for how you're adding it in Startup.

public Startup(IConfiguration configuration, IHostingEnvironment env)
        {
 var builder = new ConfigurationBuilder()
                       .AddEFConfiguration(dbcontext);
 Configuration = builder.Build();
}

 public static class EntityFrameworkExtensions
    {
        public static IConfigurationBuilder AddEFConfiguration(
            this IConfigurationBuilder builder,dbcontext)
        {
            return builder.Add(new EFConfigurationSource(_dbcontext));
        }
    }

[Tratcher]

Yes, that config will only be available in Startup. How were you consuming it?

If you want to change the IConfiguration instance that's available in DI you need to do that on the WebHostBuilder in Program.cs.

[pratishthap]

Yes, that config will only be available in Startup. How were you consuming it?

If you want to change the IConfiguration instance that's available in DI you need to do that on the WebHostBuilder in Program.cs.

So inorder to get IConfiguration values available in application controllers we need to do that in Program.cs. Do we need to do anything else for consuming the Data for it?I mean do we need to use IOptions somewhere for the same?

[Tratcher]

That depends on your usage scenarios. Where do you consume the config? That's what needs to pull for changes.

[pratishthap]

That depends on your usage scenarios. Where do you consume the config? That's what needs to pull for changes.

public static void Main(string[] args)
{
var configuration = new ConfigurationBuilder()
.AddEFConfiguration(dbcontext)
.Build();
CreateWebHostBuilder(args,configuration).Run();
}

public static IWebHost CreateWebHostBuilder(string[] args, IConfiguration config) =>
WebHost.CreateDefaultBuilder(args)
.UseConfiguration(config)
.UseStartup()
.Build()
;

Now this is the way i am using the customconfigprovider inside main() of program.cs .Now i just want to verify i can see efprovider whenever i inject @IConfiguration
like this - @Configuration.Providers - this should show me the efconfigurationprovider after it goes past startup. But in the @IConfiguration i do not see that provider

[Tratcher]

Replace UseConfiguration with ConfigureAppConfiguration. There's a known issue with UseConfiguration where it copies they keys and then discards the providers.

[pratishthap]

Replace UseConfiguration with ConfigureAppConfiguration. There's a known issue with UseConfiguration where it copies they keys and then discards the providers.

thanks that got me to where i can see my provider getting registered. Also is there not possibilty that instead of pulling, the changetoken itself gets triggered when database changes.

public EFConfigurationProvider(DBContex dbContext)
{
_dbContext = dbContext;
ChangeToken.OnChange(
() => GetReloadToken(), // listener to token change
() =>
{
this.Load();
});

Why does this not getting triggered on something changing on dbcontext?

[Tratcher]

OnChange is just a wrapper for RegisterChangeCallback.

[pratishthap]

OnChange is just a wrapper for RegisterChangeCallback.

ok so we need to use some kind of poling mechanism inorder to get the database updates.

[pratishthap]

OnChange is just a wrapper for RegisterChangeCallback.

SO if i understand the flow correctly -
We need
1)Custom db config provider which can reference itokenchanges and changes haschanges accordingly.
2)we need extensions method for adding dbconfiguration in main()
3)custom dbprovider

Where does the poling class come in between them?I was trying to understand the fileprovider you sent.Was trying to understand how the poling class will be called and from where?

[Tratcher]

What do you have reading from IConfiguration? Options?

[pratishthap]

Currently i have not associated IOptions with IConfiguration for the same. How do i use poling and IOptions for changes to dbcontext

[Tratcher]

@HaoK ?

[pratishthap]

any updates please?

[Alik2015]

Any updates, new features for this in net 5?

[Tratcher]

Comments on closed issues are not tracked, please open a new issue with the details for your scenario.