Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'Sequence contains no matching element' scaffolding database tables #1584

Closed
RotateAt60MPH opened this issue Dec 9, 2021 · 10 comments
Closed

Comments

@RotateAt60MPH
Copy link

RotateAt60MPH commented Dec 9, 2021

Steps to reproduce

dotnet ef dbcontext scaffold "connect string" "Pomelo.EntityFrameworkCore.MySql" --verbose

The issue

Throws an exception after not finding design-time services.

Using project 'd:\Client\olab\devel\repos\OLab4\OLab4-api\Service\OLabWebAPI.csproj'.
Using startup project 'd:\Client\olab\devel\repos\OLab4\OLab4-api\Service\OLabWebAPI.csproj'.
Writing 'd:\Client\olab\devel\repos\OLab4\OLab4-api\Service\obj\OLabWebAPI.csproj.EntityFrameworkCore.targets'...
dotnet msbuild /target:GetEFProjectMetadata /property:EFProjectMetadataFile=C:\Users\corey\AppData\Local\Temp\tmp3401.tmp /verbosity:quiet /nologo d:\Client\olab\devel\repos\OLab4\OLab4-api\Service\OLabWebAPI.csproj
Writing 'd:\Client\olab\devel\repos\OLab4\OLab4-api\Service\obj\OLabWebAPI.csproj.EntityFrameworkCore.targets'...
dotnet msbuild /target:GetEFProjectMetadata /property:EFProjectMetadataFile=C:\Users\corey\AppData\Local\Temp\tmp35D7.tmp /verbosity:quiet /nologo d:\Client\olab\devel\repos\OLab4\OLab4-api\Service\OLabWebAPI.csproj
Build started...
dotnet build d:\Client\olab\devel\repos\OLab4\OLab4-api\Service\OLabWebAPI.csproj /verbosity:quiet /nologo

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.93
Build succeeded.
dotnet exec --depsfile d:\Client\olab\devel\repos\OLab4\OLab4-api\Service\bin\Debug\netcoreapp3.1\OLabWebAPI.deps.json --additionalprobingpath C:\Users\corey\.nuget\packages --runtimeconfig d:\Client\olab\devel\repos\OLab4\OLab4-api\Service\bin\Debug\netcoreapp3.1\OLabWebAPI.runtimeconfig.json C:\Users\corey\.dotnet\tools\.store\dotnet-ef\5.0.0\dotnet-ef\5.0.0\tools\netcoreapp3.1\any\tools\netcoreapp2.0\any\ef.dll dbcontext scaffold server=diskstation.local;uid=entrada_dev;pwd=b012db9d53B!;database=dev_olab Pomelo.EntityFrameworkCore.MySql --assembly d:\Client\olab\devel\repos\OLab4\OLab4-api\Service\bin\Debug\netcoreapp3.1\OLabWebAPI.dll --startup-assembly d:\Client\olab\devel\repos\OLab4\OLab4-api\Service\bin\Debug\netcoreapp3.1\OLabWebAPI.dll --project-dir d:\Client\olab\devel\repos\OLab4\OLab4-api\Service\ --language C# --working-dir d:\Client\olab\devel\repos\OLab4\OLab4-api\Service --verbose --root-namespace OLabWebAPI
Using assembly 'OLabWebAPI'.
Using startup assembly 'OLabWebAPI'.
Using application base 'd:\Client\olab\devel\repos\OLab4\OLab4-api\Service\bin\Debug\netcoreapp3.1'.
Using working directory 'd:\Client\olab\devel\repos\OLab4\OLab4-api\Service'.
Using root namespace 'OLabWebAPI'.
Using project directory 'd:\Client\olab\devel\repos\OLab4\OLab4-api\Service\'.
Remaining arguments: .
Finding design-time services for provider 'Pomelo.EntityFrameworkCore.MySql'...
Using design-time services from provider 'Pomelo.EntityFrameworkCore.MySql'.
Finding design-time services referenced by assembly 'OLabWebAPI'.
No referenced design-time services were found.
Finding IDesignTimeServices implementations in assembly 'OLabWebAPI'...
No design-time services were found.
System.InvalidOperationException: Sequence contains no matching element
   at System.Linq.ThrowHelper.ThrowNoMatchException()
   at System.Linq.Enumerable.Single[TSource](IEnumerable`1 source, Func`2 predicate)
   at Pomelo.EntityFrameworkCore.MySql.Scaffolding.Internal.MySqlDatabaseModelFactory.GetConstraints(DbConnection connection, IReadOnlyList`1 tables)
   at Pomelo.EntityFrameworkCore.MySql.Scaffolding.Internal.MySqlDatabaseModelFactory.GetTables(DbConnection connection, Func`3 filter)
   at Pomelo.EntityFrameworkCore.MySql.Scaffolding.Internal.MySqlDatabaseModelFactory.Create(DbConnection connection, DatabaseModelFactoryOptions options)
   at Pomelo.EntityFrameworkCore.MySql.Scaffolding.Internal.MySqlDatabaseModelFactory.Create(String connectionString, DatabaseModelFactoryOptions options)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.ReverseEngineerScaffolder.ScaffoldModel(String connectionString, DatabaseModelFactoryOptions databaseOptions, ModelReverseEngineerOptions modelOptions, ModelCodeGenerationOptions codeOptions)
   at Microsoft.EntityFrameworkCore.Design.Internal.DatabaseOperations.ScaffoldContext(String provider, String connectionString, String outputDir, String outputContextDir, String dbContextClassName, IEnumerable`1 schemas, IEnumerable`1 tables, Boolean useDataAnnotations, Boolean overwriteFiles, Boolean useDatabaseNames)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.ScaffoldContextImpl(String provider, String connectionString, String outputDir, String outputDbContextDir, String dbContextClassName, IEnumerable`1 schemaFilters, IEnumerable`1 tableFilters, Boolean useDataAnnotations, Boolean overwriteFiles, Boolean useDatabaseNames)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.ScaffoldContext.<>c__DisplayClass0_0.<.ctor>b__0()
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.<>c__DisplayClass3_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.Execute(Action action)
Sequence contains no matching element

Further technical details

MySQL version: 10.3.29 MariaDB
Operating system: Windows 10
Pomelo.EntityFrameworkCore.MySql version: 3.2.7

Other details about my project setup:

Solution made up of main 'Services' project, with a dependancy to a 'Data' classlib project that contains the DBContext.

Ran scaffolding in main .EXE project source directory (OLabWebAPI.csproj). DBContext is in dependant project called 'Data'. Tried running the scaffolding in Data.csproj directory and same result.

I tried adding the following to the project that has my DBContext, to no avail:

using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Design;

// this is only used for scaffolding
public class DesignTimeOLabDBContext  : IDesignTimeDbContextFactory<OLabDBContext>
{
  public OLabDBContext CreateDbContext(string[] args)
  {
    var optionsBuilder = new DbContextOptionsBuilder<OLabDBContext>();
    // pass your design time connection string here
    optionsBuilder.UseMySql("connect string");
    return new OLabDBContext(optionsBuilder.Options);
  }
}
@mguinness
Copy link
Collaborator

Probably related to PR #1364 that hasn't been backported to 3.2-maint branch yet.

@RotateAt60MPH
Copy link
Author

RotateAt60MPH commented Dec 9, 2021

I'm not so sure of that unless there's other relevant infrastructure being addressed in that PR. Given the --verbose logs, it's not even getting to processing the DBContext to get tables, etc. - it simply cannot find a design-time service.

So a question I might pose is what interface is being queried in the project's assembly?

Early on in my project, I did manage to get this to work with an older 3.2.x pomelo. Back then, I'm pretty sure I had everything in a single assembly- definitely without the need to add the IDesignTimeDbContextFactory implementation above.

If the DBContext is (now) in a classlib could that cause any additional grief? Even if I run the scaffolding on the main DLL or the underlying, dependant, classlib assembly?

@mguinness
Copy link
Collaborator

mguinness commented Dec 9, 2021

Given the --verbose logs, it's not even getting to processing the DBContext to get tables, etc.

The stack trace shows it's hitting GetConstraints() method in MySqlDatabaseModelFactory class where the bug occurs.

@RotateAt60MPH
Copy link
Author

RotateAt60MPH commented Dec 9, 2021

Ah, I see what you are talking about. Could you not also interpret that the exception is because it didn't have a design-time service in place when it ran GetContraints? Is the design-time service needed, or is it just a info message in the log?

Could this problem just be some unforseen oddity in the DBContext definition?

Might be worth it to strip down the DBContext, to say: one table, and start building it up until it fails? If it fails with one simple table with no constraints, that could be useful as it would lead to an environment issue.

Thanks for your help.

@mguinness
Copy link
Collaborator

I'm pretty certain the exception is caused by that bug. Check for tables with foreign keys that reference a table in another schema. Otherwise try building the 3.2-maint branch with the fix from the aforementioned PR or upgrading to version 5.0.3 and retry.

@RotateAt60MPH
Copy link
Author

In this case I only have one schema. I'll will try and incrementally strip down the DBContext and see if I can narrow it down to an offending entity. I'll report what I find.

@mguinness
Copy link
Collaborator

mguinness commented Dec 10, 2021

Use the General Query Log to see the last SQL statement executed by the scaffold command before the exception and confim that REFERENCED_TABLE_NAME exists.

private const string GetConstraintsQuery = @"SELECT
`CONSTRAINT_NAME`,
`TABLE_NAME`,
`REFERENCED_TABLE_NAME`,
GROUP_CONCAT(CONCAT_WS('|', `COLUMN_NAME`, `REFERENCED_COLUMN_NAME`) ORDER BY `ORDINAL_POSITION` SEPARATOR ',') AS PAIRED_COLUMNS,
(SELECT `DELETE_RULE` FROM `INFORMATION_SCHEMA`.`REFERENTIAL_CONSTRAINTS` WHERE `REFERENTIAL_CONSTRAINTS`.`CONSTRAINT_NAME` = `KEY_COLUMN_USAGE`.`CONSTRAINT_NAME` AND `REFERENTIAL_CONSTRAINTS`.`CONSTRAINT_SCHEMA` = `KEY_COLUMN_USAGE`.`CONSTRAINT_SCHEMA`) AS `DELETE_RULE`
FROM `INFORMATION_SCHEMA`.`KEY_COLUMN_USAGE`
WHERE `TABLE_SCHEMA` = '{0}'
AND `TABLE_NAME` = '{1}'
AND `CONSTRAINT_NAME` <> 'PRIMARY'
AND `REFERENCED_TABLE_NAME` IS NOT NULL
GROUP BY `CONSTRAINT_SCHEMA`,
`CONSTRAINT_NAME`,
`TABLE_NAME`,
`REFERENCED_TABLE_NAME`;";

@lauxjpn
Copy link
Collaborator

lauxjpn commented Dec 10, 2021

@RotateAt60MPH The No design-time services were found. message is just an informational message that your project did not explicitly define any design-time services. Getting this message with the --verbose flag is normal, as most projects do not explicitly implement their own design-time services. So the message is completely unrelated to your issue.


However, to easily debug the issue (or even implement a workaround later), you can actually just add a custom design time service. Add the CustomMySqlDesignTimeServices and CustomMySqlDatabaseModelFactory classes from the following code to your project:

Program.cs
using System;
using System.Collections.Generic;
using System.Data.Common;
using System.Diagnostics;
using System.Linq;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Design;
using Microsoft.EntityFrameworkCore.Diagnostics;
using Microsoft.EntityFrameworkCore.Scaffolding;
using Microsoft.EntityFrameworkCore.Scaffolding.Metadata;
using Microsoft.EntityFrameworkCore.Storage;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Pomelo.EntityFrameworkCore.MySql.Design.Internal;
using Pomelo.EntityFrameworkCore.MySql.Infrastructure.Internal;
using Pomelo.EntityFrameworkCore.MySql.Scaffolding.Internal;

namespace IssueConsoleTemplate
{
    public class CustomMySqlDesignTimeServices : IDesignTimeServices
    {
        private readonly MySqlDesignTimeServices _originalMySqlDesignTimeServices;

        public CustomMySqlDesignTimeServices()
        {
            _originalMySqlDesignTimeServices = new MySqlDesignTimeServices();
        }

        public virtual void ConfigureDesignTimeServices(IServiceCollection serviceCollection)
        {
            _originalMySqlDesignTimeServices.ConfigureDesignTimeServices(serviceCollection);

            serviceCollection.AddScoped<IDatabaseModelFactory, CustomMySqlDatabaseModelFactory>();
        }
    }

    public class CustomMySqlDatabaseModelFactory : MySqlDatabaseModelFactory
    {
        private readonly IDiagnosticsLogger<DbLoggerCategory.Scaffolding> _logger;
        
        public CustomMySqlDatabaseModelFactory(
            IDiagnosticsLogger<DbLoggerCategory.Scaffolding> logger,
            IRelationalTypeMappingSource typeMappingSource,
            IMySqlOptions options)
            : base(
                logger,
                typeMappingSource,
                options)
        {
            _logger = logger;
        }
        
        private const string GetConstraintsQuery = @"SELECT
 	`CONSTRAINT_NAME`,
 	`TABLE_NAME`,
 	`REFERENCED_TABLE_NAME`,
 	GROUP_CONCAT(CONCAT_WS('|', `COLUMN_NAME`, `REFERENCED_COLUMN_NAME`) ORDER BY `ORDINAL_POSITION` SEPARATOR ',') AS PAIRED_COLUMNS,
 	(SELECT `DELETE_RULE` FROM `INFORMATION_SCHEMA`.`REFERENTIAL_CONSTRAINTS` WHERE `REFERENTIAL_CONSTRAINTS`.`CONSTRAINT_NAME` = `KEY_COLUMN_USAGE`.`CONSTRAINT_NAME` AND `REFERENTIAL_CONSTRAINTS`.`CONSTRAINT_SCHEMA` = `KEY_COLUMN_USAGE`.`CONSTRAINT_SCHEMA`) AS `DELETE_RULE`
 FROM `INFORMATION_SCHEMA`.`KEY_COLUMN_USAGE`
 WHERE `TABLE_SCHEMA` = '{0}'
 		AND `TABLE_NAME` = '{1}'
 		AND `CONSTRAINT_NAME` <> 'PRIMARY'
        AND `REFERENCED_TABLE_NAME` IS NOT NULL
        GROUP BY `CONSTRAINT_SCHEMA`,
        `CONSTRAINT_NAME`,
        `TABLE_NAME`,
        `REFERENCED_TABLE_NAME`;";

        protected virtual void GetConstraints(
            DbConnection connection,
            IReadOnlyList<DatabaseTable> tables)
        {
            Debugger.Launch(); // <-- launch the debugger

            foreach (var table in tables)
            {
                using (var command = connection.CreateCommand())
                {
                    command.CommandText = string.Format(GetConstraintsQuery, connection.Database, table.Name);
                    using (var reader = command.ExecuteReader())
                    {
                        while (reader.Read())
                        {
                            var referencedTableName = reader.GetString(2);
                            var referencedTable = tables.FirstOrDefault(t => t.Name == referencedTableName);
                            if (referencedTable == null)
                            {
                                // On operation systems with insensitive file name handling, the saved reference table name might have a
                                // different casing than the actual table name. (#1017)
                                // In the unlikely event that there are multiple tables with the same spelling, differing only in casing,
                                // we can't be certain which is the right match, so rather fail to be safe.
                                referencedTable = tables.Single(t => string.Equals(t.Name, referencedTableName, StringComparison.OrdinalIgnoreCase));
                            }
                            if (referencedTable != null)
                            {
                                var fkInfo = new DatabaseForeignKey {Name = reader.GetString(0), OnDelete = ConvertToReferentialAction(reader.GetString(4)), Table = table, PrincipalTable = referencedTable};
                                foreach (var pair in reader.GetString(3).Split(','))
                                {
                                    fkInfo.Columns.Add(table.Columns.Single(y =>
                                        string.Equals(y.Name, pair.Split('|')[0], StringComparison.OrdinalIgnoreCase)));
                                    fkInfo.PrincipalColumns.Add(fkInfo.PrincipalTable.Columns.Single(y =>
                                        string.Equals(y.Name, pair.Split('|')[1], StringComparison.OrdinalIgnoreCase)));
                                }

                                table.ForeignKeys.Add(fkInfo);
                            }
                            else
                            {
                                _logger.Logger.LogWarning($"Referenced table `{referencedTableName}` is not in dictionary.");
                            }
                        }
                    }
                }
            }
        }
    }

    internal static class Program
    {
        private static void Main()
        {
        }
    }
}

The CustomMySqlDatabaseModelFactory.GetConstraints() method is the one that the original 3.2.7 implementation uses (with the Debugger.Launch(); line added).

Just scaffold again and the JIT debugger should pop up. You can then just debug the method (or replace tables.Single() with tables.SingleOrDefault()).

@mguinness mguinness changed the title 'No design-time services found' scaffolding database tables 'Sequence contains no matching element' scaffolding database tables Dec 10, 2021
@lauxjpn
Copy link
Collaborator

lauxjpn commented Feb 12, 2022

@RotateAt60MPH What is the status of this issue? Where you able to debug it further with the information from my previous post?

@lauxjpn
Copy link
Collaborator

lauxjpn commented Feb 21, 2022

@RotateAt60MPH Since we have not heard back from you, we will close this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants