DataFusion 12.0.0 Release

[andygrove]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Planning for DataFusion 12.0.0

• sqlparser 0.21 upgrade #3192
• Upgrade to arrow 21 #3225
• Fix some bugs in TypeCoercion rule #3407
• Fix the test failure of size_of_scalar failure on M1/M2 Mac #3398

Describe the solution you'd like
The usual schedule would have the RC cut on September 9th, but I plan on releasing it one week earlier, on September 2nd. This is partially due to the 9th being before the Labor Day weekend in the US, but also to line up with Dask SQL's planned GA release in September.

Describe alternatives you've considered
None

Additional context
None

[waitingkuo]

should be arrow-datafusion 12.0.0?

[andygrove]

should be arrow-datafusion 12.0.0?

Thanks, yes!

[andygrove]

I am no longer planning on releasing this early and will stick to the regular schedule, with the RC being created on September 9th.

[andygrove]

@alamb @liukun4515 I updated the release checklist to include a few issues that are still open. I think it may be better to delay the release by a few days and get these bug fixes in, but I am also open to releasing without them and creating a new release in two weeks. What do you think?

[alamb]

I think the list of bugs looks good @andygrove -- I will apply extra effort later today and over the weekend to reviewing / helping them move along.

I don't have any particular driver to get the release out any earlier, but I think the community should probably weigh in (as we don't currently use the released versions of datafusion in my project, IOx)

[liukun4515]

@alamb @liukun4515 I updated the release checklist to include a few issues that are still open. I think it may be better to delay the release by a few days and get these bug fixes in, but I am also open to releasing without them and creating a new release in two weeks. What do you think?

agree with that.

I think the datafusion is not stable, and there are some bugs from type coercion migration.
We should find them and resolves them first.

It's better to add this pr to this plan for release #3396

[iajoiner]

@andygrove @alamb #3383
Shall this one also be resolved before 12?

[alamb]

@iajoiner I think it would be great to get #3383 resolved. However, since it is a test issue (rather than some issue with the underlying code) it is my opinion that it wouldn't hold up the release and we could release 12.0.0 even if #3383 were still open

[andygrove]

@alamb @liukun4515 I reviewed #3421, and it is not a new regression - there is no change from the behavior in 11.0.0, so I don't think that this is something that should be holding up the release. I added notes to that issue.

The other issues that we were working on related to this (#3289 and #3422) are new features and not fixing any regression so also should not be holding up the release IMO.

[alamb]

🎉