EF Core 5 support

[timmi-on-rails]

.NET 5 is coming and the release is planned for November 2020.
At the same time EF Core 5 will be released.
Today I can already try the preview builds of .NET 5 and EF Core 5 via NuGet.
But I am unable to use the Pomelo.EntityFrameworkCore.MySql adapter, since there are breaking changes in EF Core 5.
I couldn't find a build of the adapter targeting EF Core 5 Preview.

Do you have plans for that and what is the progress?
I saw there is a 5.0-wip branch.

[lauxjpn]

An EF Core 5 preview compatible Pomelo release is planned for the second half of June.

[timmi-on-rails]

Any news? On what branch are you developing the .net 5.0 upgrade?
Maybe I am able to contribute, to speed things up.
5.0-wip branch hasn't been touched since February.

[mguinness]

The current effort is focused on adding spatial and JSON support to 3.x and then afterwards upgrading will be looked at. Based on previous version upgrades it's a significant amount of work which will take several months based on the limited developer resources.

[ansarizafar]

An EF Core 5 preview compatible Pomelo release is planned for the second half of June.

@lauxjpn June is almost over. Can we expect An EF Core 5 preview compatible release this month.

[timmi-on-rails]

Based on version 3.1.1 I got an upgrade to EF Core 5 preview "working".
I have inserted some records and ran a select statement successfully.

I still have to run the tests and I need help at 2-3 places.
Just wanted to inform you, that it is possible to get a "compiling" version in couple hours.
Probably faster, when knowing EF Core + database provider api.

EF Core 5 does not require net5.0, only netstandard2.1.
Ofcourse this can still change, because it is preview state, still.

[lauxjpn]

We will publish an official preview after the JSON support release, of which I have a prototype running at the moment, that still needs a bit of polishing.
So expect an EF Core 5.0 preview compatible release until about 2020-07-15.

@ansarizafar, @timmi-on-rails If you have time, please share with us the reasons, why you already need a EF Core 5.0 compatible release. Maybe what you need can be done with the current release as well.

[timmi-on-rails]

@lauxjpn I am currently working on a project, which is still in an early phase and the first release is planned for end of 2020, so I wanted to update the project dependencies, e.g. efcore to the preview of 5.0, because this will be the up-to-date version, when the project is released.
The reason is to test and find incompatibilities as early as possible.
I haven't checked if any new efcore 5 features are useful for the project.
Just planning ahead to stay updated.

[Simonl9l]

I believe there is a general hope, across the system developer community, that the teams developing common libraries that are essential components, that in context of the overall developer ecosystem, are this time more broadly lined up with the DotNet Core 5.0 release schedule than they were with the 3.0 release schedule.

Also there are numerous issues on the various .Net Core and more importantly in the case on the EF Core GitHub repo that have a "Fixed in 5.0" conclusion.

Although 3.1 is supposed to the current LTS release, many of the solutions are rhetorically involving breaking changes or are too complex to be made backward compatible with 3.1, so are not even made candidates for Long Term Support.

As such the develop community are eager to get to use and experiment with the new features, and more specifically, if they have a MySQL dependency, need a Pomelo EF Core 5.0 compatible driver (beta) to even make that use/experiment possible.

We all await with bated breath, and any progress in this regard is greatly appreciated!

[lauxjpn]

This finally just got unblocked by #1102 (through #1157 and #1158).
Expect something working by the end of the week.

[mguinness]

The current effort is focused on adding spatial and JSON support to 3.x and then afterwards upgrading will be looked at.

Work on both of these features is now complete, thanks to Laurents for all his hard work!

[IvanZheng]

@lauxjpn When and where can I get a preview release to try on EF Core 5?

[jparadnikas]

Hey! Is there a preview release for this? I need to try it, query splitting is essential for me

[lauxjpn]

I have already upgraded Pomelo in the way that it compiles with .NET 5. I am currently upgrading the tests, so that they compile as well. Once that is done (probably tomorrow), we will release a first alpha (on our nightly build feed). Once the tests all run, we will release an official preview (probably around the end of the week). We will then introduce a couple of breaking changes and features. So code depending on alpha or preview releases will break (mostly at compile-time and not very significantly).

[onurh]

@lauxjpn Is there any change on scheduled time for alpha version?

[lauxjpn]

@onurh There is an alpha prerelease in our nightly build feed. I can also update it with the latest version in a couple of hours. In this latest version, there are still about 100 of the 15,000 tests not working yet, but this is unlikely to be significant. I am actively working on them, to get a version out with all tests running as soon as possible.

[zaneclaes]

Amazing. I now have Pomelo working w/ dotnet5, EFCore5, Identity Server 4, and MySqlConnector 1.0.1.

I had to recreate my databases (couldn't get the IS4 migration to work properly, but no big deal). I have tested all the core identity flows (creating an account, verifying, etc.). But the really nice part is that many-to-many tables without explicit mappings are now working, which saved me a TON of boilerplate and headache 🎉

[ezealv]

Amazing. I now have Pomelo working w/ dotnet5, EFCore5, Identity Server 4, and MySqlConnector 1.0.1.

I had to recreate my databases (couldn't get the IS4 migration to work properly, but no big deal). I have tested all the core identity flows (creating an account, verifying, etc.). But the really nice part is that many-to-many tables without explicit mappings are now working, which saved me a TON of boilerplate and headache 🎉

---

How do you make Pomelo work with EF Core 5? Because Pomelo 5 hasn't come out yet.

[lauxjpn]

How do you make Pomelo work with EF Core 5? Because Pomelo 5 hasn't come out yet.

@ezealv Use the latest prerelease from our nightly build feed.
The latest build 5.0.0-alpha.1.20513.5 was just released 2 hours ago.

[lauxjpn]

Sounds good, could that include 6.0 7.0? So we who would like to help could know, which ones is thought of being really needed for next version, so the release for net core 6.0 could be ready faster? Like almost at the time when net 6.0 is ready.

I would generally implement new features for the new major release and fixes of issues as a servicing release on the latest stable branch. So it makes sense to update the outstanding features then to the 6.0.0 milestone.

Usually we only plan as far as the next major release. So something is either planned for the next service release, the next minor release, the next major release or in the backlog (so there would be no planning for 7.0.0 at the moment, but only 6.0.0).

I agree that we should define a feature cut-off at least a month before the next major EF Core version release date (so around 2021-09-30), so we can deliver on time. We would then increase the milestone version on the features that didn't made the cut. (As for the current issues, I will go over it once the GA release is out.)

While we had plans for this last year as well, we plan to target the EF Core previews once 5.0.0 GA is out, so we can release on time with the EF Core 6 when it goes GA.

(Last year, we basically missed that due to limited time on my part, and the promise for spatial, Microsoft JSON and Newtonsoft JSON support within the 3.x release cycle, which where extensive and time consuming. I think it was important that we got this done for 3.x, because it is the last version that is compatible with the .NET Framework.)

[Zwergenpunk]

Honest question: Is that project goind to die or will you maintain it for dotnet 6?

It is not going to die. It is healthy and active. We will start targeting EF Core 6, as soon as our 5 GA release is out.

That's good news.
Just to clarify, I did not want to criticize you or the work you're doing. I just had the feeling that there is a possibility that your priorities had shifted too much (which would be fine, just good to know)

I did not excpect this to blow up, but I'm happy that alot of questions could be answered.

Thanks for your work.

[lauxjpn]

Just to clarify, I did not want to criticize you or the work you're doing. I just had the feeling that there is a possibility that your priorities had shifted too much (which would be fine, just good to know)

We are basically half a year later than we wanted to be with our GA release, so it is a legitimate question. But as long as there are pull requests being merged, the project is probably healthy.

Personally, I don't think it is going anywhere soon. It has a much stronger community than Oracle's implementation, is faster, more versatile, contains many major features that are not part of Oracle's implementation, is the only provider that supports MariaDB and has a quick reaction time to issues posted here or on StackOverflow.

So no worries.

[mc0re]

Regarding the upcoming beta release, it will be the one with the breaking changes, where we made the DbContext option CharSet() obsolete...

...the second (much smaller) breaking change, that is about the case in-/sensitive comparison translations

There is one more breaking change that I hit - removal of support for provider-specific functions from In-memory database provider. Now when running against InMemory, things like Hex() throw. See #1355 for details.

[lauxjpn]

There is one more breaking change that I hit - removal of support for provider-specific functions from In-memory database provider. Now when running against InMemory, things like Hex() throw. See #1355 for details.

For a list of all general EF Core 5.0 breaking changes, see Breaking changes in EF Core 5.0.

[lauxjpn]

I expect it's push today. The 5.0.0-beta.1 will be out about 24 hours after that.

PR #1385 has been pushed now.

[aminzar-access]

Hi, I still can't see Beta version in releases, is it out already?

[lauxjpn]

Hi, I still can't see Beta version in releases, is it out already?

As said:

The 5.0.0-beta.1 will be out about 24 hours after [the push].

So, expect tomorrow (after we have merged the PR). I did also update the Schedule & Roadmap accordingly.

[lukas-kamleitner]

Will support for JSON operations on MySQL will be part of the 5.0.0 release?
In the current version 3.2.5 EF.Functions.JsonExtract seems not to be implemented

[mguinness]

Have you tried using it in 5.0? Look at JsonPocoQueryTestBase and also see #14 (comment). If you cannot get it working it's best to open a new issue rather than discuss it here.

Pomelo.EntityFrameworkCore.MySql/src/EFCore.MySql/Query/ExpressionTranslators/Internal/MySqlJsonDbFunctionsTranslator.cs

Lines 84 to 92 in d48240a

	nameof(MySqlJsonDbFunctionsExtensions.JsonExtract)
	=> _sqlExpressionFactory.NullableFunction(
	"JSON_EXTRACT",
	Array.Empty<SqlExpression>()
	.Append(Json(args[0]))
	.Concat(DeconstructParamsArray(args[1])),
	method.ReturnType,
	_sqlExpressionFactory.FindMapping(method.ReturnType, "json"),
	false),

As previously mentioned if anyone would like to help with documentation (including JSON support) that would be welcomed.

[lukas-kamleitner]

Already got it working, thanks. Sorry for not opening a new ticket for the question.

[lauxjpn]

Release 5.0.0-beta.1 is now available on nuget.org.

As stated above, it contains breaking changes about how Pomelo handles the default character set (also see #1385).

[lauxjpn]

We will release 5.0.0-beta.2 on Sunday Monday Tuesday (today), that we expect to contain the last changes before the GA release on Wednesday.

Notably, there were two issues related to scaffolding (#1393 and #1394), that will be fixed in it, the ability to precisely control how charsets and collations are being inherited at every level (#1402 and #1405) and support for an optimized default collation for char-based Guid properties/columns (#1403 and #1404).

(As always, take a look at the milestone for everything related.)

[lauxjpn]

Release 5.0.0-beta.2 is now available on nuget.org.

If you need the exact charset behavior as in previous versions for any reason, use the following Fluent API code:

// The following applies the "utf8mb4" charset recursively only to columns, without changing the default charset of
// the database or its tables.
modelBuilder.HasCharSet("utf8mb4", DelegationModes.ApplyToColumns);

// The following disables the newly introduced default collation 'ascii_general_ci' for character based Guid columns.
modelBuilder.UseGuidCollation(string.Empty);

Otherwise, just use the default (utf8mb4 all the way through, except for character based Guid properties, which now use the ascii_general_ci collation and thus the ascii charset) by not calling anything explicitly, or setup precisely what you need.

It should be mentioned that if you set a collation, it is redundant to also set the charset.

Special thanks to @mc0re for his feedback, which significantly helped to improve this release and iron out the remaining issues.

[mstaal]

I am currently using ASP.NET Core Identity based on usage of your software. I have created my MySQL tables using statements like below. Is it correct that I should alter the type of Id, and is there an obvious way of changing the CREATE statement? Can you refer to a reason why the changes were made?

CREATE TABLE IF NOT EXISTS AspNetRoles (
  `Id` varchar(128) NOT NULL,
  `Name` varchar(256) NOT NULL,
  `NormalizedName` varchar(256) NOT NULL,
  `ConcurrencyStamp` varchar(256) DEFAULT NULL,
  PRIMARY KEY (`Id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

[khteh]

I don't think you need to change anything there. ascii_general_ci means that it will use exactly 128 bytes to store the GUID instead of variable 1 to 4 bytes if using UTF8, for instance.

[mstaal]

Thanks, @khteh! Lovely to know. I look much forward to support for (EF Core) v5!

[khteh]

But @mstaal why varchar(128) for GUID? I would use either varchar(36) or BINARY(16) for GUID.

[mstaal]

@khteh : To be honest I must admit that I once found inspiration in this gist, which is not a very mature explanation. It is a good point that I could perhaps benefit from changing that. I always welcome feedback.

[lauxjpn]

I don't think you need to change anything there. ascii_general_ci means that it will use exactly 128 bytes to store the GUID instead of variable 1 to 4 bytes if using UTF8, for instance.

In the case of @mstaal, it would indeed use up to 128 bytes for storage. For a usual char(36) column, it would use exactly 36 bytes. Storage space is indeed one of the three main reasons for the change.

The second one is performance: the ascii charset has the best performance (due to less storage space and minimal collation logic).

The third one is making sure, that users don't accidentally use a case sensitive collation. This might not be strictly necessary, if all GUIDs are always newly created, as they don't collide. But if for some reason someone tries to insert an existing entity, but with a differently cased GUID into to table, we want to make sure that this leads to an error by default. If this cannot happen to a user's app, performance could be improved slightly by manually using a CS collation instead (though if it is max. performance you are after, then using binary(16) is the way to go).

[...] I look much forward to support for (EF Core) v5!

@mstaal Pomelo 5.0.0 GA is going to be released later today. We are just waiting a bit to make sure, that nobody encounters any serious last minute issues.

[lauxjpn]

Release 5.0.0 is now available on nuget.org.

Thanks to everybody for their patience and special thanks to anybody who contributed!

[passuied]

thanks for making it happen!

…

On Wed, Apr 28, 2021 at 1:56 PM Laurents Meyer ***@***.***> wrote: Release <https://github.com/PomeloFoundation/Pomelo.EntityFrameworkCore.MySql/releases/tag/5.0.0> 5.0.0 is now available on nuget.org <https://www.nuget.org/packages/Pomelo.EntityFrameworkCore.MySql/5.0.0>. Thanks to everybody for their patience and special thanks to anybody who contributed! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1088 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHV5BRAU4HO5QNDSVR2QZ3TLBY7TANCNFSM4NJMEBPQ> .

[mstaal]

Thanks for the update! I just deployed my .NET 5 version of my small user management system, and everything seems to work just fine!

[LeaFrock]

So ... EF Core 6 Support is on the way, haha 🍻

[lauxjpn]

So ... EF Core 6 Support is on the way, haha 🍻

Absolutely, I started working on it immediately after the 5.0 GA release, to target 6.0.0-preview4. See #1413 for upcoming information.