dotnet ef6 (Migrations commands on projects without Visual Studio)

[bricelam]

In EF Core, we introduced dotnet ef which enabled Migrations commands (Add-Migration, Update-Database, etc.) outside of Visual Studio and on other platforms like macOS and Linux.

As part of #231, we introduced the underlying ef6 command which is cross-platform but works directly with assemblies instead of project files (csproj, vbproj, etc). For example, the equivalent of Update-Database would be...

dotnet exec \
  --depsfile ./bin/Debug/netcoreapp3.0/MyApp.deps.json \
  --runtimeconfig ./bin/Debug/netcoreapp3.0/MyApp.runtimeconfig.json \
  ~/.nuget/packages/entityframework/6.3.0/tools/netcoreapp3.0/any/ef6.dll \
  --assembly ./bin/Debug/netcoreapp3.0/MyApp.dll \
  database update

Obviously, this isn't very user friendly compared to a simple dotnet ef6 database update, but it is possible. You can also get the exact command to run by adding -Verbose to the PMC commands since they are also backed by this same underlying command.

Is this enough or should we introduce dotnet ef6? How many EF6 projects will be developed without using Visual Studio?

[bricelam]

@mrahhal created a dotnet ef6 command a while ago. It may also just work with 6.3.0...

[mrahhal]

I do continue to support dotnet ef6 now but it has fallen a bit behind on sdk compatibility (it still works with the latest versions though, it's just that I haven't reviewed it for new sdk versions in a long time). I'm willing to contribute on this if it means integrating the tool in the built in experience of 6.3.

And to add my own experience, we use the cli commands everywhere even though we mainly develop in VS. Also from what I've seen most people prefer to use it instead of any other alternative.

[marchy]

We absolutely need migrations working from the command line (ie: dotnet ef6) but @bricelam please don't count on this 'just working' without the team's official support

@mrahhal has been fantastic at supporting it, but we've had troubles multiple times, on multiple machines and on multiple OSes (Mac + Win) to get it to work with different .NET versions (link). It has actually stalled us from upgrading to ASP.NET Core 2.2 and some of the more recent .NET Core versions, and have been stuck on ASP.NET Core 2.1.0 and .NET Core 2.1.503 for months, because we can't keep burning dev resources to try to troubleshoot basic database migration needs.

@mrahhal we do owe you a further investigation once we can get back to another consideration of a .NET version upgrade – likely some time after the official 3.0 support this autumn.

[mrahhal]

@marchy I agree. I did what worked for me, but there are edge cases (I try to fix them whenever I find one, but seems like there are some that I can't even reproduce).

As I said in my earlier comment: "I'm willing to contribute on this if it means integrating the tool in the built in experience of 6.3." Meaning getting full support, I would love for this to get that by the team. It most likely will just work in the most basic setup when 6.3 comes out, but it'll be far from dependable in its current state. (Of course, even if that means the team working on a new tool from scratch, I'll be happy just getting a robust cli tool for EF6 😅 )

[bricelam]

@marchy I think you misread the description. A cross-platform command-line experience will work in 6.3.0 without this.

[marchy]

@bricelam and is this going to be as human-friendly as dotnet ef6 database update?

The ticket description above seems to indicate this isn't the case (ie: you'll need tooling to generate this command rather than being able to invoke it as a user from the command line sanely)

[bricelam]

@mrahhal If you're interested, the work is mostly just copying the dotnet-ef project and replacing core with 6. 😉

Otherwise, I'm sure I'll get around to it soon. Just maybe not for the 6.3.0 release; since I also have some EF Core 3.0.0 work on my plate which ships at the same time.

[bricelam]

@marchy Per the description:

Obviously, this isn't very user friendly compared to a simple dotnet ef6 database update

[bricelam]

@marchy Are you developing without Visual Studio? The Update-Database command (and friends) will work with .NET Core 3.0 projects in 6.3.0. So you really only need dotnet ef6 if you don't use VS.

[bricelam]

Ah, you just said you use Rider. They have first-class UI for EF6 Migrations. I suspect they'll update it for 6.3.0.

[marchy]

We are indeed (Rider). When you say only need dotnet ef6 are you referring to @mrahhal's Migrator.EF6 library or the upcoming officially supported one? I guess that's what keeps confusing the thread.

Much looking forward to trying this out in the next preview – and will provide feedback on our success.
Thanks @bricelam!

[bricelam]

Ah, sorry. When I say dotnet ef6, I mean a Microsoft-published-and-supported version.

[marchy]

@bricelam one clarification above here, we don't use the Rider UI for migrations (and have never been successful at getting them to reliably work). We use pure command line from the terminal, on both Windows and Mac.

I do hope this is a supported scenario as anything else forces or depends on external tooling – which in never the reliable way to go. We're living in a cross-platform world, so we can't depend on VS/Rider/etc.

Pure command line is the (only) official way to go.

[cunnpole]

I'd also like to see this to make it easier to run and diagnose migrations during install scripts. We have several projects that access the same database so relying on a single one of them to do the migrations isn't optimal. Doing this via script is much safer

[bedej]

(Commenting per request on the EF core3/ef6 preview 8 release announcement blog post)
Personally I would not consider ef6 on .NET Core to be "done" unless the pure and simple form of these migration commands is available. The general confusion caused by some of the EF Core 1 & 2 implementation decisions (e.g. client side evaluation and associated performance problems), which seem to have been reset in EF Core 3, has left me hesitant when evaluating adoption of EF components. It would be helpful to know that Microsoft and the EF team is fully committed to support EF6 on .NET Core end to end. Finishing the tooling in a complete way is IMO part of that effort and will help restore confidence.

[ajcvickers]

@bedej What's your development platform? (operating system, IDE, etc.)

[bedej]

MacOS + VSCode is our primary development environment. Currently having to use a VM with Windows + VS for some parts of the stack due to being blocked by lack of fully released version of EF6 on .NET Core (can’t use EF Core due to aforementioned issues).

Excited for this to be resolved when final release occurs soon! Hopefully full tooling will be a part of that release.

[tocsoft]

Seems a shame that dotnet-ef couldn't just detect the correct entity framework version in play and execute the command against the correct ef backend underneath for you (even if it has to call dotnet build in the process to get hold of the latest dlls etc). Would be much better user experience instead of adding in a whole new tool with its additional cognitive load of a having to know/learn another new tool.

[marchy]

Guys we need this. Blocking out .NET Core 3.0 migration – which has otherwise passed all other hurdles.

We CAN'T innovate without updating our models/DB. This isn't an 'enhancement' it's a must have / deal-breaker. What's the team's plan for it? (we will be stuck on .NET Core 2.2 until this is addressed)

[ErikEJ]

@marchy VS migrations commands works as always- what is blocking you?

[marchy]

@ErikEJ VS migrations commands?
This issue is about command-line tooling (ie: dotnet ef6 migrations list etc.), which doesn't work and never has.

The community has been using @mrahhal's Migrator.EF6 tool to work around it the entire .NET Core 1.x/2.x journey – but it no longer works on .NET Core 3.0, and this gap is blocking us from adopting .NET Core 3.0.

We don't use VS. We've been exclusively Rider, and have been stuck running Windows VMs on Macs anytime we touch any projects that touch Entity Framework (the pre-3.0 state of affairs), which we've been using the command-line tooling to manage EF 6 migrations, and always did even back in the VS days, for scripting and otherwise.

[marchy]

@bricelam @divega Brice what's the status on the official Microsoft dotnet ef6 command-line tool?
Is there a work item tracking this? My assumption is this ticket was tracking it, but I could be mis-understanding.

.NET Core 3.0 is here and we still don't have this deal-breaking component in.

As @bedej mentioned, the EF 6 .NET Core port can't be considered 'done' without it. It forces us to still be stuck on Windows/Visual Studio, which is LITERALLY against the .NET Core goals/mission/initiative.

Someone needs to pull the chord on this.

[ErikEJ]

Then maybe fix Migrator.net to work with EF Core 3.0?

[marchy]

@ErikEJ this has nothing to do with Migrator.EF6.

This is a basic functionality of making ANY mutations to your data model, and Microsoft shouldn't be relying on third parties to deliver mission-critical parts of their own frameworks/platforms.

We literally can't add a column to our domain models without spinning up a Windows VM, developing in a duplicated repo/IDE and have been in this state of affairs for the past 2+ years. We've tried everything from buying multiple machines to deal with this, running boot partitions, and now slow and resource-eating VMs. We've exerted all sorts of needless engineering efforts for this, from splitting our servers into needlessly-isolated micro-services to reduce the impact of how many projects have to be stuck on .NET Framework/Windows environments, to forking and debugging 3rd party stop-gap solutions like Migrator.EF6 – let alone adopting and then fully-downgrading from EF Core due to the ridiculous unreadiness on that front that's been the case along this whole journey (#side-vent).

The point is that .NET Core is ALL about running in non-Windows environments (read the mission/purpose) – literally 'cross-platform' and 'command-line tools' are two of the main stated goals lol

Quit defending the complete ball-drop on this. This needs to be resolved ASAP and without expecting the community to do Microsoft's own internal development. Your defense/work-around mindset for this is only feeding the problem.

[mrahhal]

With the help of @marchy I'm working on a version of Migrator.EF6 that'll support .NET Core 3.0 here: mrahhal/Migrator.EF6#67
Already have a prototype fully working, but faced some problems mainly unrelated to the tooling itself and more related to some other broken things in the Preview build of 3.0.

This is not to say that Migrator.EF6 is a feature-complete alternative for an official tool. Only doing this as an ad hoc solution. Would love to see this officially supported.

[penfold]

I'm loving EF6.3 - but the migrations are just doing my head in.

Really common scenario (migrating from .net 4.7 to .net core 3 but need to retain EF6 as EF Core hasn't got all the features I need).

So I have a solution with ASP Net Core (settings in appsettings.json) with separate assembly containing EF6.3 DBContext - but I just can't fathom out how to get migrations working.

dotnet ef6.dll migrations list --assembly MyCompany.DataAccess.dll --connection-string "XXX" --connection-provider System.Data.SqlClient

But it can't find the config for the provider factory.

Any hints?

[bedej]

Has anyone been able to get this to work via the complex command line listed in the original post of this issue? @penfold did you solve the issue you were seeing?

I haven't tried yet, so don't have any specific problems, I just want to know if people have actually been able to do this before I embark down the road...

[slask]

I'm loving EF6.3 - but the migrations are just doing my head in.

Really common scenario (migrating from .net 4.7 to .net core 3 but need to retain EF6 as EF Core hasn't got all the features I need).

So I have a solution with ASP Net Core (settings in appsettings.json) with separate assembly containing EF6.3 DBContext - but I just can't fathom out how to get migrations working.

dotnet ef6.dll migrations list --assembly MyCompany.DataAccess.dll --connection-string "XXX" --connection-provider System.Data.SqlClient

But it can't find the config for the provider factory.

Any hints?

Having the same issue. created separate github issue #1534

[penfold]

Has anyone been able to get this to work via the complex command line listed in the original post of this issue? @penfold did you solve the issue you were seeing?

I haven't tried yet, so don't have any specific problems, I just want to know if people have actually been able to do this before I embark down the road...

@bedej @slask I found that I was able to get migrations working by:

• make all library assemblies .net core 3.x and not .net standard
• the package manager commands work in visual studio but you need to use the connection string and provider name arguments. As I don't have a web/app.config.
• I also am using the DbConfigurationType attribute on the dbcontext class. This configuration class also sets the default provider factory type.

This would be so much better if EF could pull settings from appSettings.json.

[marchy]

@penfold unfortunately not all of us use Visual Studio / Windows. .NET Core is SUPPOSED to be cross-platform, and it's insane that something as mission critical us mutating your database is not supported.

@bricelam this ignored issue has turned into a blocking fire – and was not resolved in the recent EF 6.4 despite being raised as a critical issue in July.

Please have the team resolve urgently. Being blocked on mutating your database and/or not being able to upgrade to .NET Core 3.x is NOT an ignorable issue.

[mark-feeney-sage]

Has anyone been able to get this to work via the complex command line listed in the original post of this issue? @penfold did you solve the issue you were seeing?

I haven't tried yet, so don't have any specific problems, I just want to know if people have actually been able to do this before I embark down the road...

Not exactly as quoted. Instead, this seemed to work:

dotnet exec \
 --depsfile ./MyProject/bin/Debug/netcoreapp3.1/MyProject.deps.json \
 --runtimeconfig ./MyProject/bin/Debug/netcoreapp3.1/MyProject.runtimeconfig.json \
 ~/.nuget/packages/entityframework/6.4.0/tools/netcoreapp3.0/any/ef6.dll \
 database update --assembly ./MyProject/bin/Debug/netcoreapp3.1/MyProject.dll

[JamesReate]

Anybody find how to add a migration with the command line?
here is what I tried:

  --depsfile ./bin/Debug/netcoreapp3.1/MyProject.Data.deps.json \
  --runtimeconfig ./bin/Debug/netcoreapp3.1/MyProject.Data.runtimeconfig.json \
  ~/.nuget/packages/entityframework/6.4.0/tools/netcoreapp3.0/any/ef6.dll \
  migrations add InitialNextgen \
  --assembly ./bin/Debug/netcoreapp3.1/MyProject.Data.dll \
  --language cs \
  --data-dir ./ \
  --migrations-config MyProject.Data.Migrations.Configuration \
  --connection-string "Server=tcp:127.0.0.1,1433;Database=MyProjectDb;Persist Security Info=False;User id=sa;Password=Your_password123;MultipleActiveResultSets=True;Encrypt=False;TrustServerCertificate=False;" \
  --connection-provider System.Data.SqlClient

But I get the error:

System.ArgumentNullException: Value cannot be null. (Parameter 'path1')
   at System.IO.Path.Combine(String path1, String path2)
   at System.Data.Entity.Tools.Commands.ProjectCommandBase.WriteMigration(ScaffoldedMigration scaffoldedMigration, Boolean rescaffolding, Boolean force, String name) in /_/src/ef6/Commands/ProjectCommandBase.cs:line 114

I'm trying to understand the source code but I can't figure out where it is setting the Directory property of the ScaffoldedMigration object. The problem is that Directory is coming through null. The project is netcoreapp3.1 and I'm running the command from the project root.

[BigApple1988]

Anybody find how to add a migration with the command line?

I wrote simple .bat file to be able to run migrations without such pain.
I figured out that ef6.dll should be in the same directory as assembly which contains DbContext definition

@echo off

pushd .
cd
if "%~1"=="" (goto noargument) else goto begin
:noargument
@echo Migration Name should be passed as parameter
pause
goto done
:begin
dotnet publish DbContextProject\DbContextProject.csproj
cd DbContextProject/bin/Debug/netstandard2.1/publish
mkdir migration
xcopy /y . .\migration
cd migration
xcopy /y "%USERPROFILE%\.nuget\packages\entityframework\6.4.0\tools\netcoreapp3.0\any" .
xcopy /y "%USERPROFILE%\.nuget\packages\entityframework\6.4.0\lib\netstandard2.1" .
xcopy /y "%USERPROFILE%\.nuget\packages\system.componentmodel.annotations\4.3.0\lib\netstandard1.4\System.ComponentModel.Annotations.dll" .
dotnet ef6.dll migrations add %1 --assembly DbContextProject.dll --project-dir ../../../../../../DbContextProject --config ../../../../../../AppProject/App.config
:done
popd
exit /B /1

maybe it's not the perfect solution but it works for me :)

[romanryasne2]

Any updates?

[marchy]

@bricelam @ajcvickers What's the team's priority on this?

Has this been decided as a "won't do"? (meaning we are forced to update to EF Core to have database migrations again?)

Please note there are still several things not ready in EF Core (such as TPT) that are only slated before such a migration would even be viable - and that's assuming the myriad of querying bugs will be resolved, which after the most recent 'complete rewrite' of the query engine in 3.0 does not instil much confidence that they will.

Silence is not an acceptable response to this fire you've left us a whole year with burning.

[marchy]

Guys what's the update on this?

We've been forced into an attempted side-by-side .NET Core 3.1 upgrade and the framework is still far from production-ready – extremely basic example: link

We need to get unblocked on migrating our EF6 database again.
This issue is CRITICAL and has left us without the ability to mutate our models/database for over a year, while EF Core continues to be nowhere near production ready.

@bricelam the radio silence is not helping. Please share the team's rationale for why this hasn't been prioritized.

[bricelam]

I'll defer to @ajcvickers. My primary focus is on EF Core Migrations for the .NET 5.0 wave.

But also, "blocked" and "critical" seem like a bit of an exaggeration to me. The Visual Studio PMC commands (Add-Migration, Update-Database, etc.) work on .NET Core projects. You can also use the ugly (but functional) method described in the description of this issue. If you truly are blocked from (and not just inconvenienced by) using migrations, please clarify what issues you are hitting.

[marchy]

@bricelam thanks for chiming in.

As I have repeatedly stated we do not run on Windows / Visual Studio. The whole point of .NET CORE is that it's supposed to be cross-platform, and thus why we use it from our Mac/Rider environment. Rider did not add the support for EF6 migrations (given Microsoft's own indifference I imagine they did not want to hunt for workarounds themselves and were waiting on you guys to provide official support). "Visual Studio" (ie: Xamarin MonoDevelop) on Mac has no support for EF migrations as far as we can tell. Therefore we HAVE been fully blocked.

We even tried spinning up VMs so we can perform the migrations from Windows + Visual Studio, but as of the latest .NET Core 3.1 update we could not get VS migrations to work with our project (v 15.1.4-47270). We were previously using @mrahhal's Migrator.EF6 for .NET Core 1.x-2.x but neither @mrahal or I could get it to work for 3.x – and thus pushed off the 3.x upgrade until we were forced due to other environment dependencies.

This is the error we get within VS/Windows on a simple 'dotnet ef6 migrations list' command:

Unhandled Exception:
 
System.ComponentModel.Win32Exception: The system cannot find the file specified
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at Microsoft.DotNet.Cli.Utils.Command.Execute()
   at Migrator.EF6.Tools.Program.Dispatch(String[] args)
   at Migrator.EF6.Tools.Program.Main(String[] args)

I don't know what happened but at some point there are enough hacks that the whole thing breaks down – and after leaving this known and unsolved problem for over a year this seems to have been that point.

You honestly can't just rely on the community (Migrator.EF6) for something as critical and mission-critical as UPDATING YOUR MODELS/DATABASE. I will continue to use the words 'blocking' and 'critical' Brice because this is what not being able to touch your database means for a business, and the lack of the team's understanding of why this ought to be a P1 is astonishing.

[marchy]

PS: Your primary focus should have been on your existing, production-grade customers who are running stable and proven systems (ie: EF6) and making sure they can evolve through the .NET Core transition until EF Core is ready (MAYBE as of EFC 5 - though I wouldn't put my money on it before EFC 6).

Instead you've left us in the dust, had us burn endless cycles in dealing with preventable livability issues such as the above, and most of all left an EXTREMELY bad taste in our mouths as a result of everything that EFC 1.x, 2.x and 3.x has been.

#rantover

[mignatko]

Anybody find how to add a migration with the command line?
here is what I tried:

  --depsfile ./bin/Debug/netcoreapp3.1/MyProject.Data.deps.json \
  --runtimeconfig ./bin/Debug/netcoreapp3.1/MyProject.Data.runtimeconfig.json \
  ~/.nuget/packages/entityframework/6.4.0/tools/netcoreapp3.0/any/ef6.dll \
  migrations add InitialNextgen \
  --assembly ./bin/Debug/netcoreapp3.1/MyProject.Data.dll \
  --language cs \
  --data-dir ./ \
  --migrations-config MyProject.Data.Migrations.Configuration \
  --connection-string "Server=tcp:127.0.0.1,1433;Database=MyProjectDb;Persist Security Info=False;User id=sa;Password=Your_password123;MultipleActiveResultSets=True;Encrypt=False;TrustServerCertificate=False;" \
  --connection-provider System.Data.SqlClient

But I get the error:

System.ArgumentNullException: Value cannot be null. (Parameter 'path1')
   at System.IO.Path.Combine(String path1, String path2)
   at System.Data.Entity.Tools.Commands.ProjectCommandBase.WriteMigration(ScaffoldedMigration scaffoldedMigration, Boolean rescaffolding, Boolean force, String name) in /_/src/ef6/Commands/ProjectCommandBase.cs:line 114

I'm trying to understand the source code but I can't figure out where it is setting the Directory property of the ScaffoldedMigration object. The problem is that Directory is coming through null. The project is netcoreapp3.1 and I'm running the command from the project root.

I had the same problem and figured out that path1 is mapped to the --project-dir switch. And after I set it to point to directory of project which contains my migrations, it started to work.

Hope this will help somebody.

[ajcvickers]

This issue has been closed because EF6 is no longer being actively developed. We are instead focusing on stability of the codebase, which means we will only make changes to address security issues. See the repo README for more information.