New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EF Core doesn't lazy-load virtual navigation properties #3312

Closed
bitcrazed opened this Issue Oct 3, 2015 · 47 comments

Comments

Projects
None yet
@bitcrazed

bitcrazed commented Oct 3, 2015

Issue: EF7 may not be lazy-loading all entities in a graph of related entities, despite all entities' navigation properties being virtual:

I built a basic blog atop EF6. It behaves as expected and correctly models many-to-many relationships - in this case, the fact that posts may have zero or more authors.

I then rewrote this basic blog atop EF7.

This is where I found out that EF7 doesn't currently support modelling many-to-many relationships (see #1368) and requires one to manually create and handle the join entity itself.

So, I added the PostAuthor type to the app's model.

Aside: Many-to-many is going to be supported for RTW, right? This is a pretty essential feature

  +----------+          +----------+          +-------------+          +--------+
  |          | 1      n |          | 1      n |             | n      1 |        |
  |   Blog   | -------- |   Post   | -------- | PostAuthors | -------- | Author |
  |          |          |          |          |             |          |        |
  +----------+          +----------+          +-------------+          +--------+

When querying the model, I was found that EF wasn't populating the Blog's Posts collection requiring me to .Include(blog => b.Posts) and then didn't populate the Post's PostAuthors collection, requiring me to .ThenInclude(p=>p.PostAuthors). Imagine my surprise, then, when I also found that I had to .ThenInclude(pa=>pa.Authors). The net result? This:

 var blogs = ctx
        .Blogs
        .Include(b => b.Posts)
        .ThenInclude(p => p.PostAuthors)
        .ThenInclude(pa => pa.Author)
         .ToList();

My expectation is that since all the entities' navigation properties are virtual (Post, PostAuthor and Author ) that they'd be lazy-loaded on demand.

What am I doing wrong here?

TIA,

@bitcrazed bitcrazed changed the title from EF7 doesn't hydrate virtual navigation properties to EF7 doesn't lazy-load virtual navigation properties Oct 3, 2015

@HenryGarle

This comment has been minimized.

Show comment
Hide comment
@HenryGarle

HenryGarle Oct 4, 2015

The short answer is that EF7 does not currently support automatic lazy loading in the same way that EF6 did. I've seen it discussed in a few places and if anyone is better informed feel free to correct me but as far as I am aware it is fairly far down their priority list and if some posts I have read are to be believed, that's if they choose to implement it at all.

HenryGarle commented Oct 4, 2015

The short answer is that EF7 does not currently support automatic lazy loading in the same way that EF6 did. I've seen it discussed in a few places and if anyone is better informed feel free to correct me but as far as I am aware it is fairly far down their priority list and if some posts I have read are to be believed, that's if they choose to implement it at all.

@divega

This comment has been minimized.

Show comment
Hide comment
@divega

divega Oct 5, 2015

Member

@HenryGarle is right. There is no automatic lazy loading support in EF7, and we don't have a clear outlook on its priority. Lazy loading can make data access easier to deal with but it has pitfalls such as potentially producing very inefficient query patterns that cause many more database roundtrips than necessary. By comparison eager loading (i.e. the Include() methods), and using projections or multiple queries to load a graph of objects can be more cumbersome but give you finer control.

Member

divega commented Oct 5, 2015

@HenryGarle is right. There is no automatic lazy loading support in EF7, and we don't have a clear outlook on its priority. Lazy loading can make data access easier to deal with but it has pitfalls such as potentially producing very inefficient query patterns that cause many more database roundtrips than necessary. By comparison eager loading (i.e. the Include() methods), and using projections or multiple queries to load a graph of objects can be more cumbersome but give you finer control.

@bitcrazed

This comment has been minimized.

Show comment
Hide comment
@bitcrazed

bitcrazed Oct 5, 2015

Wow. I am surprised - while far from perfect, removing Lazy Loading is a pretty major change that I've not seen explicitly stated in any of the docs, nor in the ASPNET/Announcements repo, no mention of it in the Guiding Principles, etc. Can you point me at any docs where this is stated?

@divega: You state that EF7 doesn't support lazy loading, but in #3318 you state that "... and if lazy loading is enabled it will work the same for collection and reference properties." which indicates that LazyLoading can be enabled, no?

bitcrazed commented Oct 5, 2015

Wow. I am surprised - while far from perfect, removing Lazy Loading is a pretty major change that I've not seen explicitly stated in any of the docs, nor in the ASPNET/Announcements repo, no mention of it in the Guiding Principles, etc. Can you point me at any docs where this is stated?

@divega: You state that EF7 doesn't support lazy loading, but in #3318 you state that "... and if lazy loading is enabled it will work the same for collection and reference properties." which indicates that LazyLoading can be enabled, no?

@divega

This comment has been minimized.

Show comment
Hide comment
@divega

divega Oct 5, 2015

Member

Edited that last sentence to make it clearer that I was referring to EF6 behaviors.

Member

divega commented Oct 5, 2015

Edited that last sentence to make it clearer that I was referring to EF6 behaviors.

@bitcrazed

This comment has been minimized.

Show comment
Hide comment
@bitcrazed

bitcrazed Oct 5, 2015

Thanks.

Assume there's currently no statement on the removal of Lazy Loading? Any chance the EF team could add Announcements and/or document breaking changes like these - it'd be a huge help to those of us trying to get started (and help others do so) on the new bits?

bitcrazed commented Oct 5, 2015

Thanks.

Assume there's currently no statement on the removal of Lazy Loading? Any chance the EF team could add Announcements and/or document breaking changes like these - it'd be a huge help to those of us trying to get started (and help others do so) on the new bits?

@natemcmaster

This comment has been minimized.

Show comment
Hide comment
@natemcmaster

natemcmaster Oct 5, 2015

Member

@bitcrazed Thanks for the feedback. Since we are still pre-release, there is no documentation on what is a "breaking" change from EF6 to EF7. As EF7's features stabilize, we will write more docs. I've added aspnet/EntityFramework.Docs#40 to help capture this feedback.

Member

natemcmaster commented Oct 5, 2015

@bitcrazed Thanks for the feedback. Since we are still pre-release, there is no documentation on what is a "breaking" change from EF6 to EF7. As EF7's features stabilize, we will write more docs. I've added aspnet/EntityFramework.Docs#40 to help capture this feedback.

@rowanmiller

This comment has been minimized.

Show comment
Hide comment
@rowanmiller

rowanmiller Oct 5, 2015

Member

Agreed, we will have some "Should I be using EF7" documentation and the "What's new in EF7" and "What's removed in EF7" content will be a large part of that.

Member

rowanmiller commented Oct 5, 2015

Agreed, we will have some "Should I be using EF7" documentation and the "What's new in EF7" and "What's removed in EF7" content will be a large part of that.

@bitcrazed

This comment has been minimized.

Show comment
Hide comment
@bitcrazed

bitcrazed Oct 5, 2015

@natemcmaster Many thanks.

I know it's early days for the EF team but when developing in the open like this, timely release of breaking changes vs. a previous version is pretty important.

I think that official guidance on issues like this (along with no automatic many-to-many support) will be crucially important for many who need to decide between using EF7 or EF<=6 for new projects that may be shipping in the next 6-8 months.

I'll be blogging on subjects like this as I find them anyhow, but an official position would be most useful.

bitcrazed commented Oct 5, 2015

@natemcmaster Many thanks.

I know it's early days for the EF team but when developing in the open like this, timely release of breaking changes vs. a previous version is pretty important.

I think that official guidance on issues like this (along with no automatic many-to-many support) will be crucially important for many who need to decide between using EF7 or EF<=6 for new projects that may be shipping in the next 6-8 months.

I'll be blogging on subjects like this as I find them anyhow, but an official position would be most useful.

@bitcrazed

This comment has been minimized.

Show comment
Hide comment
@bitcrazed

bitcrazed Oct 5, 2015

@rowanmiller For me, this is just another example why I think this project should have been called something other than EntityFramework - because it's not EF. While EF7 retains some vestiges of EF concepts, I think it's sufficiently (so very) different from EF that it'd be easier to explain what it is if it wasn't called EF!

bitcrazed commented Oct 5, 2015

@rowanmiller For me, this is just another example why I think this project should have been called something other than EntityFramework - because it's not EF. While EF7 retains some vestiges of EF concepts, I think it's sufficiently (so very) different from EF that it'd be easier to explain what it is if it wasn't called EF!

@rowanmiller rowanmiller closed this Oct 5, 2015

@JeffGillin

This comment has been minimized.

Show comment
Hide comment
@JeffGillin

JeffGillin Dec 4, 2015

@divega, @rowanmiller
First off, great job so far with EF7. The team is doing an excellent job. I know a major re-write of an ORM can't be easy - to say the least.
Regarding Lazy Loading, I feel it is absolutely essential. Here's why.
A lot of us have domain logic coded into model classes and use the repository pattern to fetch entities. In this case, the domain model class expects the navigation property to have values that match what's in the database (whether they are loaded on-demand or eager loaded). Otherwise, for a particular operation in the domain model class, you are 'assuming' that the property was pre-loaded. This is a dangerous assumption. If it wasn't pre-loaded, then in the case of a Collection, it's Count will be 0, and your logic is assuming it's zero in the database - but it's not - it just wasn't loaded.
Thus, in order to work properly, you need to now coordinate each type of action in your application with a proper fetch strategy from your repository. I'm working with EF7 now, and whenever I load something from a repository, I pass in an enum value that represents a "eager loading strategy" enum parameter so that I can determine which navigation properties need to be included, and add then include those dynamically to the Linq-to-entities query. This works - but the necessary coordination between repository fetching and domain model action is definitely brittle. The lazy loading is a nice safe haven to ensure it will always be loaded when some logic is expecting it will be. EF6 had the ability to disable lazy loading. I'm going to suggest going the opposite direction with EF7. Make it a feature that is disabled by default but can be enabled via an option in the 'DbContextOptionsBuilder'.
One other note along the same lines, is that, with a clearly defined separation of layers (where domain model has no knowledge of EF), I'm finding it difficult to deal with collections in a way that works as I would expect. Let's say a have a method in a model class that removes some child entities based on some filter. If I remove an entity from a ICollection navigation property, I would expect that this would automatically set a 'deleted' state on that child entity, and a 'SaveChanges' on the tracked parent entity will know to delete that item from the child table.
Since we want the domain model to be persistence ignorant, we don't want any references to EF in the domain model assembly, and thus can't directly set a deleted state with EF methods. In this type of architecture, it's necessary that this removal is tracked (again - maybe an optional setting or something - but without it - my architecture, of keeping logic in domain model classes, no longer works and it's necessary to start putting some of this logic in the service(application) layer which is undesirable because I want to write automated tests against the domain model to test it's logic without having any db involved in the process.
Is this something that's possible to do? If so, is it something that is intended to be included at some point?

JeffGillin commented Dec 4, 2015

@divega, @rowanmiller
First off, great job so far with EF7. The team is doing an excellent job. I know a major re-write of an ORM can't be easy - to say the least.
Regarding Lazy Loading, I feel it is absolutely essential. Here's why.
A lot of us have domain logic coded into model classes and use the repository pattern to fetch entities. In this case, the domain model class expects the navigation property to have values that match what's in the database (whether they are loaded on-demand or eager loaded). Otherwise, for a particular operation in the domain model class, you are 'assuming' that the property was pre-loaded. This is a dangerous assumption. If it wasn't pre-loaded, then in the case of a Collection, it's Count will be 0, and your logic is assuming it's zero in the database - but it's not - it just wasn't loaded.
Thus, in order to work properly, you need to now coordinate each type of action in your application with a proper fetch strategy from your repository. I'm working with EF7 now, and whenever I load something from a repository, I pass in an enum value that represents a "eager loading strategy" enum parameter so that I can determine which navigation properties need to be included, and add then include those dynamically to the Linq-to-entities query. This works - but the necessary coordination between repository fetching and domain model action is definitely brittle. The lazy loading is a nice safe haven to ensure it will always be loaded when some logic is expecting it will be. EF6 had the ability to disable lazy loading. I'm going to suggest going the opposite direction with EF7. Make it a feature that is disabled by default but can be enabled via an option in the 'DbContextOptionsBuilder'.
One other note along the same lines, is that, with a clearly defined separation of layers (where domain model has no knowledge of EF), I'm finding it difficult to deal with collections in a way that works as I would expect. Let's say a have a method in a model class that removes some child entities based on some filter. If I remove an entity from a ICollection navigation property, I would expect that this would automatically set a 'deleted' state on that child entity, and a 'SaveChanges' on the tracked parent entity will know to delete that item from the child table.
Since we want the domain model to be persistence ignorant, we don't want any references to EF in the domain model assembly, and thus can't directly set a deleted state with EF methods. In this type of architecture, it's necessary that this removal is tracked (again - maybe an optional setting or something - but without it - my architecture, of keeping logic in domain model classes, no longer works and it's necessary to start putting some of this logic in the service(application) layer which is undesirable because I want to write automated tests against the domain model to test it's logic without having any db involved in the process.
Is this something that's possible to do? If so, is it something that is intended to be included at some point?

@rowanmiller

This comment has been minimized.

Show comment
Hide comment
@rowanmiller

rowanmiller Dec 7, 2015

Member

@JeffGillin - here is the backlog item tracking Lazy Loading #3797

Member

rowanmiller commented Dec 7, 2015

@JeffGillin - here is the backlog item tracking Lazy Loading #3797

@SimonOrdo

This comment has been minimized.

Show comment
Hide comment
@SimonOrdo

SimonOrdo Dec 15, 2015

I'm using Identity and EF7. I have extended my ApplicationUser class with a navigation property like so:

public class ApplicationUser : IdentityUser
{
          public virtual PersonalConfigData MyConfig { get; set; }

          [ForeignKey("ID")]
          public long MyConfigID { get; set; }
}

In the account controller, the user is loaded via the Identity framework like so:

var user = await _userManager.FindByNameAsync(model.Username);

How would you get the additional data to load? I thought lazy loading would take care of this.

SimonOrdo commented Dec 15, 2015

I'm using Identity and EF7. I have extended my ApplicationUser class with a navigation property like so:

public class ApplicationUser : IdentityUser
{
          public virtual PersonalConfigData MyConfig { get; set; }

          [ForeignKey("ID")]
          public long MyConfigID { get; set; }
}

In the account controller, the user is loaded via the Identity framework like so:

var user = await _userManager.FindByNameAsync(model.Username);

How would you get the additional data to load? I thought lazy loading would take care of this.

@rowanmiller

This comment has been minimized.

Show comment
Hide comment
@rowanmiller

rowanmiller Dec 15, 2015

Member

Lazy loading is not implemented in EF7 (backlog item is #3797).

When you query, you can use the Include method to bring in related data:

context.Users.Include(u => u.PersonalConfigData).Where(u => u.Username == username).Single()
Member

rowanmiller commented Dec 15, 2015

Lazy loading is not implemented in EF7 (backlog item is #3797).

When you query, you can use the Include method to bring in related data:

context.Users.Include(u => u.PersonalConfigData).Where(u => u.Username == username).Single()
@SimonOrdo

This comment has been minimized.

Show comment
Hide comment
@SimonOrdo

SimonOrdo Dec 15, 2015

Understood @rowanmiller , but that does mean I have to query the entire object twice, in this case, right?

First time, the Identity _userManager does the query. Then, if it comes back gtg, I need to query the object AGAIN to load the nav property, if I'm understanding you correctly.

Is there also currently no way to hydrate specific properties on an object? Like "LoadProperty", for instance? Seems that without lazy loading, that would be a minimum requirement, considering I might not always have control over the initial query.

[Edit]

Nevermind. I guess I'm a bit ignorant on how objects are tracked. It looks like that line of code you sent adds the additional property data to the originally queried object, not just a newly returned one.

SimonOrdo commented Dec 15, 2015

Understood @rowanmiller , but that does mean I have to query the entire object twice, in this case, right?

First time, the Identity _userManager does the query. Then, if it comes back gtg, I need to query the object AGAIN to load the nav property, if I'm understanding you correctly.

Is there also currently no way to hydrate specific properties on an object? Like "LoadProperty", for instance? Seems that without lazy loading, that would be a minimum requirement, considering I might not always have control over the initial query.

[Edit]

Nevermind. I guess I'm a bit ignorant on how objects are tracked. It looks like that line of code you sent adds the additional property data to the originally queried object, not just a newly returned one.

@rowanmiller

This comment has been minimized.

Show comment
Hide comment
@rowanmiller

rowanmiller Dec 15, 2015

Member

Correct, or if you just query for the related object then EF will also join things up in-memory for you.

context.PersonalConfigData.Where(c => c.Id == user.MyConfigId).Single()
Member

rowanmiller commented Dec 15, 2015

Correct, or if you just query for the related object then EF will also join things up in-memory for you.

context.PersonalConfigData.Where(c => c.Id == user.MyConfigId).Single()
@dwdickens

This comment has been minimized.

Show comment
Hide comment
@dwdickens

dwdickens Jan 12, 2016

Thanks @rowanmiller this is very helpful.

I'm having trouble with using an include to bring in the related data.

var _gifts = _context.Gifts.Include(g => g.Image).Where(g => g.Registry.ApplicationUser.Id == usrId)

throws the error: Unable to cast object of type 'System.Linq.Expressions.FieldExpression' to type 'System.Linq.Expressions.ParameterExpression'

I don't understand this error - my code seems to be the same as that above.

If I remove the Include it works, but I then need to have: await _context.Images.ToListAsync(); to load the Images (which is bad because this loads all images, not just those for the selected gifts).

How can I just load the Images selected in _gifts?

The models are:

public class Gift
{
    public int GiftID { get; set; }
    public virtual Registry Registry { get; set; }
    public virtual Image Image { get; set; }

}

public class Image
{
    public int ImageID { get; set; }
    public byte[] Content { get; set; }
}

dwdickens commented Jan 12, 2016

Thanks @rowanmiller this is very helpful.

I'm having trouble with using an include to bring in the related data.

var _gifts = _context.Gifts.Include(g => g.Image).Where(g => g.Registry.ApplicationUser.Id == usrId)

throws the error: Unable to cast object of type 'System.Linq.Expressions.FieldExpression' to type 'System.Linq.Expressions.ParameterExpression'

I don't understand this error - my code seems to be the same as that above.

If I remove the Include it works, but I then need to have: await _context.Images.ToListAsync(); to load the Images (which is bad because this loads all images, not just those for the selected gifts).

How can I just load the Images selected in _gifts?

The models are:

public class Gift
{
    public int GiftID { get; set; }
    public virtual Registry Registry { get; set; }
    public virtual Image Image { get; set; }

}

public class Image
{
    public int ImageID { get; set; }
    public byte[] Content { get; set; }
}
@NullVoxPopuli

This comment has been minimized.

Show comment
Hide comment
@NullVoxPopuli

NullVoxPopuli Jan 14, 2016

Hey, I asked this question on Stack Overflow: http://stackoverflow.com/questions/34790533/how-do-i-simplify-the-access-of-a-has-many-relationship-with-the-entity-framewor

It seems that implementing lazy loading would solve the root of my problem.

But If there was a dynamic work around to achieve something similar, I'd be up for that as well.

//This is what I have so far

        public object RelationshipFor(string relationship)
        {
            using (var db = User.DbContext())
            {
                var myTable = (DbSet<DatabaseModels.User>)db.Send(RelationshipName);
                var myInclude = myTable.Include(i => i.Send(relationship));
                var meWithRelationship = myInclude.First(MatchesIdFor);
                return meWithRelationship.Send(relationship);
            }
        }


        public bool MatchesIdFor(object item)
        {
            return (long)item.Send(IdColumn) == Id;
        }

but I get the error:

Unable to cast object of type 'System.Linq.Expressions.MethodCallExpressionN' to type 'System.Linq.Expressions.MemberExpression'.

NullVoxPopuli commented Jan 14, 2016

Hey, I asked this question on Stack Overflow: http://stackoverflow.com/questions/34790533/how-do-i-simplify-the-access-of-a-has-many-relationship-with-the-entity-framewor

It seems that implementing lazy loading would solve the root of my problem.

But If there was a dynamic work around to achieve something similar, I'd be up for that as well.

//This is what I have so far

        public object RelationshipFor(string relationship)
        {
            using (var db = User.DbContext())
            {
                var myTable = (DbSet<DatabaseModels.User>)db.Send(RelationshipName);
                var myInclude = myTable.Include(i => i.Send(relationship));
                var meWithRelationship = myInclude.First(MatchesIdFor);
                return meWithRelationship.Send(relationship);
            }
        }


        public bool MatchesIdFor(object item)
        {
            return (long)item.Send(IdColumn) == Id;
        }

but I get the error:

Unable to cast object of type 'System.Linq.Expressions.MethodCallExpressionN' to type 'System.Linq.Expressions.MemberExpression'.
@NullVoxPopuli

This comment has been minimized.

Show comment
Hide comment
@NullVoxPopuli

NullVoxPopuli Jan 14, 2016

Also, is there a way to dynamically cast an object of a DbSet?

NullVoxPopuli commented Jan 14, 2016

Also, is there a way to dynamically cast an object of a DbSet?

@rowanmiller

This comment has been minimized.

Show comment
Hide comment
@rowanmiller

rowanmiller Jan 14, 2016

Member

@dwdickens and @NullVoxPopuli the bug being tracked by this issue has been resolved. Please open new issues for the problems you are encountering.

Member

rowanmiller commented Jan 14, 2016

@dwdickens and @NullVoxPopuli the bug being tracked by this issue has been resolved. Please open new issues for the problems you are encountering.

@rowanmiller

This comment has been minimized.

Show comment
Hide comment
@rowanmiller

rowanmiller Jan 15, 2016

Member

@dwdickens when you open a new issue can you include a full code listing (or project) that we can use to reproduce the issue. There is probably a specific pattern in your model that is causing the error to occur.

Member

rowanmiller commented Jan 15, 2016

@dwdickens when you open a new issue can you include a full code listing (or project) that we can use to reproduce the issue. There is probably a specific pattern in your model that is causing the error to occur.

@makarti

This comment has been minimized.

Show comment
Hide comment
@makarti

makarti Mar 2, 2016

This code

categories .Include(c => c.Products).ThenInclude(p => p.Reviews.).ThenInclude(r => r.Author) .Include(c => c.Products).ThenInclude(p => p.Supplier);

in sql query genarate two joins "Products". Is it possible optimize this expression?

makarti commented Mar 2, 2016

This code

categories .Include(c => c.Products).ThenInclude(p => p.Reviews.).ThenInclude(r => r.Author) .Include(c => c.Products).ThenInclude(p => p.Supplier);

in sql query genarate two joins "Products". Is it possible optimize this expression?

@rowanmiller

This comment has been minimized.

Show comment
Hide comment
@rowanmiller

rowanmiller Mar 2, 2016

Member

@makarti we have #3419 to look at consolidating joins in scenario like this.

Member

rowanmiller commented Mar 2, 2016

@makarti we have #3419 to look at consolidating joins in scenario like this.

@christophedemey

This comment has been minimized.

Show comment
Hide comment
@christophedemey

christophedemey Mar 6, 2016

I am also missing lazy loading in EF7 👎

christophedemey commented Mar 6, 2016

I am also missing lazy loading in EF7 👎

@gdoron

This comment has been minimized.

Show comment
Hide comment
@gdoron

gdoron Mar 6, 2016

@christophedemey , why?
This proxy-lazy loading pattern caused people to hit the Database multiple times and possibly N+1 times.
Now you must think before you query what relationships do you need, and you will never hit the DB multiple times.
That's not lazy loading, it's lazy programming... 😆
The lazy loading that is quite clever but not frequently used is "Lazy loading properties", which means that unless you eagerly include (pre-configured) properties, they will stay null.

It's helpful mainly when querying for entities that have lengthy string properties.
So unless you actually need that property, it will remain null, just like with Navigation Properties.

gdoron commented Mar 6, 2016

@christophedemey , why?
This proxy-lazy loading pattern caused people to hit the Database multiple times and possibly N+1 times.
Now you must think before you query what relationships do you need, and you will never hit the DB multiple times.
That's not lazy loading, it's lazy programming... 😆
The lazy loading that is quite clever but not frequently used is "Lazy loading properties", which means that unless you eagerly include (pre-configured) properties, they will stay null.

It's helpful mainly when querying for entities that have lengthy string properties.
So unless you actually need that property, it will remain null, just like with Navigation Properties.

@dwdickens

This comment has been minimized.

Show comment
Hide comment
@dwdickens

dwdickens Mar 6, 2016

I appreciate the performance reasons for discouraging lazy loading.

Developing an app now with EF, it gets confusing if a bug is due to bad db data or a failure to properly load - null is ambiguous.

Why doesn't EF throw an error if you try to access data that hasn't been loaded?

On 7 Mar 2016, at 6:05 AM, Doron Grinzaig notifications@github.com wrote:

@christophedemey , why?
This proxy-lazy loading pattern caused people to hit the Database multiple times and possibly N+1 times.
Now you must think before you query what relationships do you need, and you will never hit the DB multiple times.
That's not lazy loading, it's lazy programming...
The lazy loading that is quite clever but not frequently used is "Lazy loading properties", which means that unless you eagerly include (pre-configured) properties, they will stay null.

It's helpful mainly when querying for entities that have lengthy string properties.
So unless you actually need that property, it will remain null, just like with Navigation Properties.


Reply to this email directly or view it on GitHub.

dwdickens commented Mar 6, 2016

I appreciate the performance reasons for discouraging lazy loading.

Developing an app now with EF, it gets confusing if a bug is due to bad db data or a failure to properly load - null is ambiguous.

Why doesn't EF throw an error if you try to access data that hasn't been loaded?

On 7 Mar 2016, at 6:05 AM, Doron Grinzaig notifications@github.com wrote:

@christophedemey , why?
This proxy-lazy loading pattern caused people to hit the Database multiple times and possibly N+1 times.
Now you must think before you query what relationships do you need, and you will never hit the DB multiple times.
That's not lazy loading, it's lazy programming...
The lazy loading that is quite clever but not frequently used is "Lazy loading properties", which means that unless you eagerly include (pre-configured) properties, they will stay null.

It's helpful mainly when querying for entities that have lengthy string properties.
So unless you actually need that property, it will remain null, just like with Navigation Properties.


Reply to this email directly or view it on GitHub.

@christophedemey

This comment has been minimized.

Show comment
Hide comment
@christophedemey

christophedemey Mar 8, 2016

@gdoron if you put it this way you are definitly right !

christophedemey commented Mar 8, 2016

@gdoron if you put it this way you are definitly right !

@gdoron

This comment has been minimized.

Show comment
Hide comment
@gdoron

gdoron Mar 8, 2016

@dwdickens, you will get a null reference exception if you don't load the navigation property (and you didn't init the property in the ctor), and an empty list if you did, so it is already easy to understand.

I'm happy with the current implementing o navigation properties in EF core.

gdoron commented Mar 8, 2016

@dwdickens, you will get a null reference exception if you don't load the navigation property (and you didn't init the property in the ctor), and an empty list if you did, so it is already easy to understand.

I'm happy with the current implementing o navigation properties in EF core.

@willotheblessed

This comment has been minimized.

Show comment
Hide comment
@willotheblessed

willotheblessed Mar 11, 2016

@rowanmiller I had the same problem as he did, but my issue isn't solved yet. When I try to reference the "context" I am using ApplicationDBContext and though it has an option called "Users" That doesn't include a method called "Include"

I'm using ASP5, MVC6. So I either need a different method, or a way to get the right context so I can use this:

context.Users.Include(u=> u.PersonalConfigData).Where(u => u.Username == username).Single()

willotheblessed commented Mar 11, 2016

@rowanmiller I had the same problem as he did, but my issue isn't solved yet. When I try to reference the "context" I am using ApplicationDBContext and though it has an option called "Users" That doesn't include a method called "Include"

I'm using ASP5, MVC6. So I either need a different method, or a way to get the right context so I can use this:

context.Users.Include(u=> u.PersonalConfigData).Where(u => u.Username == username).Single()

@rowanmiller

This comment has been minimized.

Show comment
Hide comment
@rowanmiller

rowanmiller Mar 11, 2016

Member

@willotheblessed make sure you have a using for the Microsoft.Data.Entity namespace, Include is an extension method in that namespace.

Member

rowanmiller commented Mar 11, 2016

@willotheblessed make sure you have a using for the Microsoft.Data.Entity namespace, Include is an extension method in that namespace.

@willotheblessed

This comment has been minimized.

Show comment
Hide comment
@willotheblessed

willotheblessed commented Mar 11, 2016

@rowanmiller

That did it! Thanks!

@SimonOrdo

This comment has been minimized.

Show comment
Hide comment
@SimonOrdo

SimonOrdo Mar 14, 2016

@rowanmiller Could I ask you a tangentially related question? I noticed VS doesn't offer hints to add missing using namespace statements for extension methods the way it does for other references (right click on red-lined code -> Quick Action).

It seems like with this version of ASP.NET there's more extension method stuff going on, so it's been much more noticeable lately.

Could you suggest an appropriate place for me to submit that feature request?

SimonOrdo commented Mar 14, 2016

@rowanmiller Could I ask you a tangentially related question? I noticed VS doesn't offer hints to add missing using namespace statements for extension methods the way it does for other references (right click on red-lined code -> Quick Action).

It seems like with this version of ASP.NET there's more extension method stuff going on, so it's been much more noticeable lately.

Could you suggest an appropriate place for me to submit that feature request?

@rowanmiller

This comment has been minimized.

Show comment
Hide comment
@rowanmiller

rowanmiller Mar 21, 2016

Member

@SimonOrdo you can submit VS ideas at http://connect.microsoft.com/VisualStudio

That said, quick actions does offer to add namespaces for extension methods for me:

quickactions

Member

rowanmiller commented Mar 21, 2016

@SimonOrdo you can submit VS ideas at http://connect.microsoft.com/VisualStudio

That said, quick actions does offer to add namespaces for extension methods for me:

quickactions

@rahulsahay19

This comment has been minimized.

Show comment
Hide comment
@rahulsahay19

rahulsahay19 Mar 27, 2016

Here is a scenario.
//Movie Class
public class Movie
{
public int Id { get; set; }
public string MovieName { get; set; }
public string DirectorName { get; set; }
public string ReleaseYear { get; set; }
public virtual ICollection Reviews { get; set; }
}

//MovieReview
public class MovieReview
{
public int Id { get; set; }
public string ReviewerName { get; set; }
public string ReviewerComments { get; set; }
public int ReviewerRating { get; set; }
public int MovieId { get; set; }
}

//MovieViewModel
public class MovieViewModel
{
public int Id { get; set; }
public string MovieName { get; set; }
public string DirectorName { get; set; }
public string ReleaseYear { get; set; }
public int NoOfReviews { get; set; }
}

//Now, with below Get query, I am expected to get NoOfReviews as well. But, its always returning 0 as shown below in the screenshot
// GET api/movies
[HttpGet("api/movies")]
public IQueryable Get()
{
var model = UOW.Movies.GetAll().OrderByDescending(m => m.Reviews.Count())
.Select(m => new MovieViewModel
{
Id = m.Id,
MovieName = m.MovieName,
DirectorName = m.DirectorName,
ReleaseYear = m.ReleaseYear,
NoOfReviews = m.Reviews.Count
});
return model;
}
70th

rahulsahay19 commented Mar 27, 2016

Here is a scenario.
//Movie Class
public class Movie
{
public int Id { get; set; }
public string MovieName { get; set; }
public string DirectorName { get; set; }
public string ReleaseYear { get; set; }
public virtual ICollection Reviews { get; set; }
}

//MovieReview
public class MovieReview
{
public int Id { get; set; }
public string ReviewerName { get; set; }
public string ReviewerComments { get; set; }
public int ReviewerRating { get; set; }
public int MovieId { get; set; }
}

//MovieViewModel
public class MovieViewModel
{
public int Id { get; set; }
public string MovieName { get; set; }
public string DirectorName { get; set; }
public string ReleaseYear { get; set; }
public int NoOfReviews { get; set; }
}

//Now, with below Get query, I am expected to get NoOfReviews as well. But, its always returning 0 as shown below in the screenshot
// GET api/movies
[HttpGet("api/movies")]
public IQueryable Get()
{
var model = UOW.Movies.GetAll().OrderByDescending(m => m.Reviews.Count())
.Select(m => new MovieViewModel
{
Id = m.Id,
MovieName = m.MovieName,
DirectorName = m.DirectorName,
ReleaseYear = m.ReleaseYear,
NoOfReviews = m.Reviews.Count
});
return model;
}
70th

@rahulsahay19

This comment has been minimized.

Show comment
Hide comment
@rahulsahay19

rahulsahay19 Mar 27, 2016

here movie class is having public virtual ICollection Reviews { get; set; } . Don't know why it is not getting printed properly.

rahulsahay19 commented Mar 27, 2016

here movie class is having public virtual ICollection Reviews { get; set; } . Don't know why it is not getting printed properly.

@rahulsahay19

This comment has been minimized.

Show comment
Hide comment
@rahulsahay19

rahulsahay19 Mar 27, 2016

Attaching Movie class.
71th

rahulsahay19 commented Mar 27, 2016

Attaching Movie class.
71th

@gdoron

This comment has been minimized.

Show comment
Hide comment
@gdoron

gdoron Mar 27, 2016

@rahulsahay19

var model = UOW.Movies.Include(x=> x.Reviews).GetAll().OrderByDescending(m => m.Reviews.Count())

gdoron commented Mar 27, 2016

@rahulsahay19

var model = UOW.Movies.Include(x=> x.Reviews).GetAll().OrderByDescending(m => m.Reviews.Count())
@rahulsahay19

This comment has been minimized.

Show comment
Hide comment
@rahulsahay19

rahulsahay19 Mar 27, 2016

This is breaking with the following exception.
72nd

however, when i am writing the below piece
var model = UOW.Movies.GetAll().Include(x => x.Reviews).OrderByDescending(m => m.Reviews.Count);
it is giving below error.
73rd

rahulsahay19 commented Mar 27, 2016

This is breaking with the following exception.
72nd

however, when i am writing the below piece
var model = UOW.Movies.GetAll().Include(x => x.Reviews).OrderByDescending(m => m.Reviews.Count);
it is giving below error.
73rd

@rahulsahay19

This comment has been minimized.

Show comment
Hide comment
@rahulsahay19

rahulsahay19 Mar 27, 2016

My Mistake done. Here is the completed code

var model = UOW.Movies.GetAll().Include(x => x.Reviews).OrderByDescending(m => m.Reviews.Count)
.Select(m => new MovieViewModel
{
Id = m.Id,
MovieName = m.MovieName,
DirectorName = m.DirectorName,
ReleaseYear = m.ReleaseYear,
NoOfReviews = m.Reviews.Count
});

rahulsahay19 commented Mar 27, 2016

My Mistake done. Here is the completed code

var model = UOW.Movies.GetAll().Include(x => x.Reviews).OrderByDescending(m => m.Reviews.Count)
.Select(m => new MovieViewModel
{
Id = m.Id,
MovieName = m.MovieName,
DirectorName = m.DirectorName,
ReleaseYear = m.ReleaseYear,
NoOfReviews = m.Reviews.Count
});

@joehoeller

This comment has been minimized.

Show comment
Hide comment
@joehoeller

joehoeller Mar 30, 2017

If using .NET Core and Entity Framework Core, and .Include(x => x.TableName ) is not returning relationships (from principal table to dependent table), or only returning one row of data, FIX HERE:

[Stackoverflow Answer]((https://stackoverflow.com/questions/43127957/include-not-working-in-net-core-returns-one-parent)

Also, in Startup.cs make sure you have this at the top:

using Microsoft.EntityFrameworkCore;
using Newtonsoft.Json;
using Project_Name_Here.Models;

*For older .NET you can use the [JsonIgnore] attribute as well.

joehoeller commented Mar 30, 2017

If using .NET Core and Entity Framework Core, and .Include(x => x.TableName ) is not returning relationships (from principal table to dependent table), or only returning one row of data, FIX HERE:

[Stackoverflow Answer]((https://stackoverflow.com/questions/43127957/include-not-working-in-net-core-returns-one-parent)

Also, in Startup.cs make sure you have this at the top:

using Microsoft.EntityFrameworkCore;
using Newtonsoft.Json;
using Project_Name_Here.Models;

*For older .NET you can use the [JsonIgnore] attribute as well.

@hmdhasani

This comment has been minimized.

Show comment
Hide comment
@hmdhasani

hmdhasani Jul 29, 2017

@joehoeller The Stackoverflow page is removed!

hmdhasani commented Jul 29, 2017

@joehoeller The Stackoverflow page is removed!

@gdoron

This comment has been minimized.

Show comment
Hide comment
@gdoron

gdoron commented Jul 29, 2017

@hmdhasani there you go.
image

@joehoeller

This comment has been minimized.

Show comment
Hide comment
@joehoeller

joehoeller Sep 19, 2017

The jerks at Stack removed it bc they have serious control issues, even though I answered the ques, see here: https://www.linkedin.com/pulse/include-working-net-core-only-returns-one-parent-entity-joe-hoeller-

joehoeller commented Sep 19, 2017

The jerks at Stack removed it bc they have serious control issues, even though I answered the ques, see here: https://www.linkedin.com/pulse/include-working-net-core-only-returns-one-parent-entity-joe-hoeller-

@joehoeller

This comment has been minimized.

Show comment
Hide comment
@joehoeller

joehoeller Sep 19, 2017

Also, Im not sure that this will help, but try placing "app.UseResponseBuffering();" inside public void Configure located in Startup.cs.

joehoeller commented Sep 19, 2017

Also, Im not sure that this will help, but try placing "app.UseResponseBuffering();" inside public void Configure located in Startup.cs.

@MartinDawson

This comment has been minimized.

Show comment
Hide comment
@MartinDawson

MartinDawson Sep 20, 2017

@joehoeller They closed it because that's not how you answer your own question. You first post the question and then answer it otherwise it's confusing to future readers. If you had actually read the guidelines for posting before posting you would have known this and not got upset.

MartinDawson commented Sep 20, 2017

@joehoeller They closed it because that's not how you answer your own question. You first post the question and then answer it otherwise it's confusing to future readers. If you had actually read the guidelines for posting before posting you would have known this and not got upset.

@shpsyte

This comment has been minimized.

Show comment
Hide comment
@shpsyte

shpsyte Jan 26, 2018

@joehoeller not work form me

Iam using .NET Core 2.0 and Entity Framework Core, and .Include(x => x.TableName ) returning one row of data

i try:
in Startup.cs

services .AddMvc() .AddJsonOptions(options => { options.SerializerSettings.ReferenceLoopHandling = ReferenceLoopHandling.Ignore; });

Exists another alternative ?

shpsyte commented Jan 26, 2018

@joehoeller not work form me

Iam using .NET Core 2.0 and Entity Framework Core, and .Include(x => x.TableName ) returning one row of data

i try:
in Startup.cs

services .AddMvc() .AddJsonOptions(options => { options.SerializerSettings.ReferenceLoopHandling = ReferenceLoopHandling.Ignore; });

Exists another alternative ?

@divega divega changed the title from EF7 doesn't lazy-load virtual navigation properties to EF Core doesn't lazy-load virtual navigation properties Feb 26, 2018

@lukemason76

This comment has been minimized.

Show comment
Hide comment
@lukemason76

lukemason76 Apr 23, 2018

The GroupEditModel is part of a CRUD for editing a Group.

First the object is declared and annotated
[BindProperty]
public Group Group { get; set; }

Then in OnGetAsync() it is filled.
Group = await _db.Groups
.Include(g => g.Tier)
.Include(g => g.GroupType)
.Include(g => g.Parent)
.Include(g => g.Organisation)
.FirstOrDefaultAsync(g => g.Id == id.Value);

Then the object comes back in the OnPostAsync(). Here the object is filled by the binder, but the Navigation properties are null. How do I get these filled by the binder, or fill them easily after that occurs?

lukemason76 commented Apr 23, 2018

The GroupEditModel is part of a CRUD for editing a Group.

First the object is declared and annotated
[BindProperty]
public Group Group { get; set; }

Then in OnGetAsync() it is filled.
Group = await _db.Groups
.Include(g => g.Tier)
.Include(g => g.GroupType)
.Include(g => g.Parent)
.Include(g => g.Organisation)
.FirstOrDefaultAsync(g => g.Id == id.Value);

Then the object comes back in the OnPostAsync(). Here the object is filled by the binder, but the Navigation properties are null. How do I get these filled by the binder, or fill them easily after that occurs?

@smitpatel

This comment has been minimized.

Show comment
Hide comment
@smitpatel

smitpatel Apr 23, 2018

Contributor

@lukemason76 - Did you enable lazy loading proxies? EF Core does not lazy load virtual navigations by default. See https://docs.microsoft.com/en-us/ef/core/querying/related-data#lazy-loading

If you are still having issues after following the documentation, please file a new issue with detailed repro steps.

Contributor

smitpatel commented Apr 23, 2018

@lukemason76 - Did you enable lazy loading proxies? EF Core does not lazy load virtual navigations by default. See https://docs.microsoft.com/en-us/ef/core/querying/related-data#lazy-loading

If you are still having issues after following the documentation, please file a new issue with detailed repro steps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment