NullReferenceException when derived types have explicit FK property (doesn't happen if FK prop is created as shadow by convention)

[MatthewMWR]

Exception:

System.NullReferenceException was unhandled
  HResult=-2147467261
  Message=Object reference not set to an instance of an object.
  Source=EntityFramework.Core
  StackTrace:
       at Microsoft.Data.Entity.Metadata.Internal.ClrAccessorSource`1.Create(PropertyInfo property)
       at Microsoft.Data.Entity.Metadata.Internal.ClrAccessorSource`1.<GetAccessor>b__3_0(Tuple`2 k)
       at Microsoft.Data.Entity.Internal.ThreadSafeDictionaryCache`2.GetOrAdd(TKey key, Func`2 factory)
       at Microsoft.Data.Entity.Metadata.Internal.ClrAccessorSource`1.GetAccessor(Type declaringType, String propertyName)
       at Microsoft.Data.Entity.Metadata.Internal.ClrAccessorSource`1.GetAccessor(IPropertyBase property)
       at Microsoft.Data.Entity.ChangeTracking.Internal.EntityEntryMetadataServices.ReadValue(Object entity, IPropertyBase propertyBase)
       at Microsoft.Data.Entity.ChangeTracking.Internal.InternalEntityEntry.ReadPropertyValue(IPropertyBase propertyBase)
       at Microsoft.Data.Entity.ChangeTracking.Internal.InternalEntityEntry.get_Item(IPropertyBase propertyBase)
       at Microsoft.Data.Entity.ChangeTracking.Internal.InternalEntityEntry.set_Item(IPropertyBase propertyBase, Object value)
       at Microsoft.Data.Entity.ChangeTracking.Internal.NavigationFixer.SetForeignKeyValue(IForeignKey foreignKey, InternalEntityEntry dependentEntry, IReadOnlyList`1 principalValues)
       at Microsoft.Data.Entity.ChangeTracking.Internal.NavigationFixer.NavigationCollectionChangedAction(InternalEntityEntry entry, INavigation navigation, IEnumerable`1 added, IEnumerable`1 removed)
       at Microsoft.Data.Entity.ChangeTracking.Internal.NavigationFixer.InitialFixup(InternalEntityEntry entry)
       at Microsoft.Data.Entity.ChangeTracking.Internal.NavigationFixer.<>c__DisplayClass14_0.<StateChanged>b__0()
       at Microsoft.Data.Entity.ChangeTracking.Internal.NavigationFixer.PerformFixup(Action fixupAction)
       at Microsoft.Data.Entity.ChangeTracking.Internal.NavigationFixer.StateChanged(InternalEntityEntry entry, EntityState oldState)
       at Microsoft.Data.Entity.ChangeTracking.Internal.InternalEntityEntryNotifier.<>c__DisplayClass7_0.<StateChanged>b__0(IEntityStateListener l)
       at Microsoft.Data.Entity.ChangeTracking.Internal.InternalEntityEntryNotifier.Dispatch(Action`1 action)
       at Microsoft.Data.Entity.ChangeTracking.Internal.InternalEntityEntryNotifier.StateChanged(InternalEntityEntry entry, EntityState oldState)
       at Microsoft.Data.Entity.ChangeTracking.Internal.InternalEntityEntry.SetEntityState(EntityState oldState, EntityState newState, Boolean acceptChanges)
       at Microsoft.Data.Entity.ChangeTracking.Internal.InternalEntityEntry.SetEntityState(EntityState entityState, Boolean acceptChanges)
       at Microsoft.Data.Entity.ChangeTracking.Internal.EntityGraphAttacher.PaintAction(InternalEntityEntryGraphNode n)
       at Microsoft.Data.Entity.ChangeTracking.Internal.EntityEntryGraphIterator.TraverseGraph[TNode](TNode node, Func`2 handleNode)
       at Microsoft.Data.Entity.ChangeTracking.Internal.EntityGraphAttacher.AttachGraph(InternalEntityEntry rootEntry, EntityState entityState)
       at Microsoft.Data.Entity.DbContext.SetEntityState(InternalEntityEntry entry, EntityState entityState, GraphBehavior behavior)
       at Microsoft.Data.Entity.DbContext.SetEntityState[TEntity](TEntity entity, EntityState entityState, GraphBehavior behavior)
       at Microsoft.Data.Entity.DbContext.Add[TEntity](TEntity entity, GraphBehavior behavior)
       at ConsoleApplication2.Program.Main(String[] args) in C:\Users\matt\Documents\Visual Studio 2015\Projects\ConsoleApplication2\ConsoleApplication2\Program.cs:line 27
       at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
       at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
       at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
       at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
       at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading.ThreadHelper.ThreadStart()
  InnerException: 

Repro:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Threading;
using Microsoft.Data.Entity;

namespace ConsoleApplication2
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            //  build hub/spoke object graph
            var toolboxInstance = new Toolbox() {Title = "This toolbox is red"};
            var hammerInstance = new Hammer() {Description = "My favorite hammer"};
            var wrenchInstance = new Wrench() {Description = "My least favorite wrench"};
            toolboxInstance.Tools.Add(hammerInstance);
            toolboxInstance.Tools.Add(wrenchInstance);

            using (var dbc = new ToolsContext())
            {
                dbc.Database.EnsureCreated();
                dbc.Database.EnsureDeleted();
                dbc.Database.EnsureCreated();

                dbc.Add(toolboxInstance);
                dbc.Add(hammerInstance);
                dbc.Add(wrenchInstance);
                Console.WriteLine(dbc.SaveChanges());
            }

            using (var dbc = new ToolsContext())
            {
                //  Exception does not happen if you remove the .Include() from this
                //  but you don't get the subordinate objects back
                var toolboxesBackFromDb = dbc.Toolboxes.Include(x => x.Tools);
                Console.WriteLine(toolboxesBackFromDb.Count());
                var toolbox1 = toolboxesBackFromDb.AsEnumerable().FirstOrDefault();
                Console.WriteLine(toolbox1.Tools.Any());
            }
            Thread.Sleep(10000);
        }
    }

    public class ToolsContext : DbContext
    {
        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            optionsBuilder.UseSqlServer(@"server=(localdb)\MsSqlLocalDB;database=HubSpokeExperiment");
        }
        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            //  using reflection here to pick up new tool types as they are created
            //  rather than maintaining static list
            foreach (var t in Assembly.GetExecutingAssembly().DefinedTypes.Where(t => t.IsSubclassOf(typeof(ToolBase))))
            {
                var eb = modelBuilder.Entity(t.AsType());
                eb.HasKey("Id");
            }
        }
        public DbSet<Toolbox> Toolboxes { get; set; } 
        public DbSet<ToolBase> Tools { get; set; }
    }

    public class Toolbox
    {
        public Toolbox()
        {
            Tools = new List<ToolBase>();
        }
        public int Id { get; set; }
        public string Title { get; set; }
        public ICollection<ToolBase> Tools { get; set; }
    }

    public abstract class ToolBase
    {
        public string Description { get; set; }
        public Toolbox Toolbox { get; set; }
        public int Id { get; set; }
    }

    public class Hammer : ToolBase
    {
        public string SomethingAboutHammers { get; set; }
        public int ToolboxId { get; set; }
    }

    public class Wrench : ToolBase
    {
        public string SomethingAboutWrenches { get; set; }
        public int ToolboxId { get; set; }
    }
}

[natemcmaster]

@MatthewMWR what is the build number of EF?

[MatthewMWR]

Should be beta8 downloaded yesterday. Will confirm shortly.

[MatthewMWR]

File properties on EntityFramework.SqlServer.dll shows:
File version: 7.0.0.11009
Product version: 7.0.0-beta8-15964

NuGet shows:

PM> get-package

Id                                  Versions                                 ProjectName                                           
--                                  --------                                 -----------                                           
EntityFramework.Core                {7.0.0-beta8}                            ConsoleApplication2                                   
EntityFramework.Relational          {7.0.0-beta8}                            ConsoleApplication2                                   
EntityFramework.SqlServer           {7.0.0-beta8}                            ConsoleApplication2                                   
Ix-Async                            {1.2.5}                                  ConsoleApplication2                                   
Microsoft.Framework.Caching.Abst... {1.0.0-beta8}                            ConsoleApplication2                                   
Microsoft.Framework.Caching.Memory  {1.0.0-beta8}                            ConsoleApplication2                                   
Microsoft.Framework.Configuration   {1.0.0-beta8}                            ConsoleApplication2                                   
Microsoft.Framework.Configuratio... {1.0.0-beta8}                            ConsoleApplication2                                   
Microsoft.Framework.Configuratio... {1.0.0-beta8}                            ConsoleApplication2                                   
Microsoft.Framework.DependencyIn... {1.0.0-beta8}                            ConsoleApplication2                                   
Microsoft.Framework.DependencyIn... {1.0.0-beta8}                            ConsoleApplication2                                   
Microsoft.Framework.Logging         {1.0.0-beta8}                            ConsoleApplication2                                   
Microsoft.Framework.Logging.Abst... {1.0.0-beta8}                            ConsoleApplication2                                   
Microsoft.Framework.OptionsModel    {1.0.0-beta8}                            ConsoleApplication2                                   
Microsoft.Framework.Primitives      {1.0.0-beta8}                            ConsoleApplication2                                   
Remotion.Linq                       {2.0.0-beta-002}                         ConsoleApplication2                                   
System.Collections.Immutable        {1.1.36}                                 ConsoleApplication2

[natemcmaster]

@MatthewMWR Possible fixed in #3478. Can you verify this bug still occurs with build >= 16047? (this will be an rc1 release).

[MatthewMWR]

Sure. Is there a nightly build I can download or do I need to get set up to build locally?

[natemcmaster]

@MatthewMWR If you are unfamiliar with nightly builds, you should know there is some added instability of nightly builds due to rapid churn. Nightly builds can be by using this package feed: http://myget.org/gallery/aspnetvnext (Note there have been many renames of packages since beta8, including aspnet/Announcements#85)

[MatthewMWR]

The latest prerelease I am seeing on MyGet is 7.0.0-rc1-15945.
Is >= 16047 hiding somewhere?

[ErikEJ]

@natemcmaster looks like aspvnext has stalled, maybe better to point people to aspnetcidev ?

[natemcmaster]

@ErikEJ aspvnext intentionally lags (for testing) and is more stable than aspnetcidev.

That said .... @MatthewMWR http://myget.org/gallery/aspnetcidev

[MatthewMWR]

So far all attempts at installing from aspnetcidev fail with "Failed to add reference to 'System.Collections'."
I'm trying different NuGet tricks-- uninstall, reinstall, re-ordering and such, but so far no luck. Might there be some hidden dependency that NuGet doesn't understand (maybe need a visual studio update or fx update?).

[MatthewMWR]

Forgive my ignorance for package management and fx versions... but it appears installing very recent EntityFramework.Core from http://myget.org/gallery/aspnetcidev is blocked because the EntityFramework.Core nupkg wants to add a project reference to System.Collections. At the same time it appears to be impossible to add such a reference (even manually) in a 4.5.x project because the vNext nuget packages for System.Collections don't appear to include a 4.5.x DLL. I imagine this means "use the framework version", but in 4.5.x there is no System.Collections assembly. The related classes appear to be implemented in mscorlib.dll.

Maybe I am doing it wrong, or maybe this is a new bug.

In any case I know you guys have a lot on your plate. If the NullReference bug looks important I'm eager to help repro so I'd appreciate help with this problem. If my original bug doesn't look important for what you are working on now I can wait.

[natemcmaster]

@MatthewMWR this particular bug investigation make take a while as we have some urgent deadlines upon us. Any help you can give us with this particular bug is much appreciated. Nightly builds are unstable and not always compatible with released version of VS tooling. Make sure you have the latest NuGet extension on use the dnx command line if/when that fails.

[MatthewMWR]

Sure thing. I will update if I can get set up with >= 16047.
In the meantime the above repro is a complete copy/paste console app so if you get the chance consider trying against your latest.

[smitpatel]

@MatthewMWR - Since the latest codebase of EF has TPH scenario fully implemented, while trying to run above repro code, model uses TPH hence defining keys on derived types is not possible.

[rowanmiller]

By design (see previous comment from @smitpatel). Support for TPC is tracked by #3170