Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query plan caching doesn't work with Query Filter #175

Closed
adamlepkowski opened this issue Jun 23, 2017 · 8 comments

Comments

@adamlepkowski
Copy link

commented Jun 23, 2017

Hi, I found a serious bug which can introduce performance issues because when Query Filter feature is used query plan caching is not working.
I made a basic test to verify it and code is available in this repository: https://github.com/adamlepkowski/EFPlusIssuesInvestigations.
To show a problem I defined three tests:

  1. two queries without query filter - for the second query plan is retrieved from the cache
  2. two queries with global filter - for the second query 2 times query compilation is executed
  3. two queries with instance filter - for the second query 4 times query compilation is executed
    In those test, I execute queries in different database contexts to emulate Session per request pattern.
    image

I think that problem is in two places:

  1. Value for UniqueKey field from a BaseQueryFilterInterceptor class is generated. When I temporary established some static value then for test number 3, I still had 2 query compilations so this is the reason why I think that two places introduce problems.
    image

  2. I didn't have time to find the second place.

/Adam

@JonathanMagnan JonathanMagnan self-assigned this Jun 25, 2017
@adamlepkowski

This comment has been minimized.

Copy link
Author

commented Jun 28, 2017

Hi Jonathan, any progress? Potentially I can find some free time for further investigation, but I would like to know your opinion on this subject.

@JonathanMagnan

This comment has been minimized.

Copy link
Member

commented Jun 28, 2017

Hello @adamlepkowski ,

Sorry for the delay, time is currently missing again!

Because a filter is used, we re-use the execution plan only within the same instance of the DbContext.

It could be possible for Global Filter to re-use the execution plan already compiled but impossible for instance filter since the filter can be changed from an instance to another. Because we cannot get access to methods that cache or retrieve from the cache the execution plan, the perfect solution cannot be used.

They have the double query compilation when using a filter since the first compilation is used as a hook and will never be really compiled. It will use the result from the query generated using all filters instead.

Let me know if that's a little bit more clear.

Best Regards,

Jonathan

@adamlepkowski

This comment has been minimized.

Copy link
Author

commented Jul 8, 2017

Hi @JonathanMagnan

Sorry for the delay but I was extremely busy.

Thanks for your answer the subject is much more clearer now. I will do some performance tests with a global filter, and if results are not acceptable, then I will probably abandon a filter module (currently performance is important for me).

/Adam

@rmaziarka

This comment has been minimized.

Copy link

commented Aug 3, 2017

Hi guys, we finally switched your EF+ filters with EntityFramework.DynamicFilters - https://github.com/jcachat/EntityFramework.DynamicFilters

In this library such dynamic caching works well and you can disable and enable filters without further problems. I think that you can investigate how they achieved such functionality :)

@JonathanMagnan

This comment has been minimized.

Copy link
Member

commented Aug 3, 2017

Hello @rmaziarka ,

Don't worry, I already investigate this library a lot and know exactly how is doing it ;)

The problem is our library is cursed by having more flexibility and support filtering with any LINQ method including: Take, Skip, OrderBy, etc...

I believe the right thing to do is having both features in the future. One working like DynamicFilter when only Where clause is supported and one where all LINQ methods are supported.

Best Regards,

Jonathan

@kf-radoslawmaziarka

This comment has been minimized.

Copy link

commented Aug 3, 2017

@JonathanMagnan You mean that in DynamicFilters other clauses doesn't work?

@JonathanMagnan

This comment has been minimized.

Copy link
Member

commented Aug 3, 2017

Hello @kf-radoslawmaziarka ,

Exactly. Dynamic Filter works perfectly with Where clause.

However, you cannot use a method such as Include, Take, Skip unless I'm wrong.

Best Regards,

Jonathan

@adamlepkowski

This comment has been minimized.

Copy link
Author

commented Aug 3, 2017

Hi both,

In our case, all scenarios are supported, and we don't need anything else. I checked it very carefully including analysis of EntityFramework.DynamicFilters source code.

We just need to filter entities implementing some specific interface by property value. The filter must work if we make a query on an entity (dbContext.EntityWithInterface) that implement the interface or when such entity is eager loaded (dbContext.NotFilteredEntity.Include(x=>x.EntityWithInterface)).

Your library provides much more functionality like default filtering and include filtering, but in our case, we don't need to use it.

P.S we didn't resign from all your modules because I think that your library belongs to "must have" category when EF is used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
4 participants
You can’t perform that action at this time.