DOC: prepare 2.3.1 whatsnew notes for release

[jorisvandenbossche]

Prepping for doing a 2.3.1 release today, xref #61590

[simonjayhawkins]

@jorisvandenbossche for the "Comparisons between different string dtypes" is there an issue ref?

I don't get why:

When pd.StringDtype("pyarrow", na_value=pd.NA) is compared against any other string dtype, the result will always be boolean[pyarrow].

since when did we start mixing the pandas nullable dtypes with the ArrowDtypes?

If this is now policy, when do the ArrowDtypes return the ArrowDtype version of the string array and not the new default string array (pd.NA variant)?

[simonjayhawkins]

in "Index set operations ignore empty RangeIndex and object dtype Index"

the code block uses

pd.options.mode.infer_string = True

this should be

pd.options.future.infer_string = True

[jorisvandenbossche]

for the "Comparisons between different string dtypes" is there an issue ref?

#61138 is the PR, #60639 the issue. Will add a link

since when did we start mixing the pandas nullable dtypes with the ArrowDtypes?

Unfortunately for some time .. (and it is also a change that I don't really agree with). I was also again confused about it when finalizing that PR (see #61138 (comment)). But, this has been like this now for some releases, so not something to change here in pandas 2.3 (if we want to change it, it's something for 3.0 I think).
I know there was some discussion about this in the past, looking it up.

[simonjayhawkins]

Well I need to do some more research to be sure, but I'm not happy on two fronts: the change itself and the fact that you were "required" to do PDEP-14 and maintain backwards compat with the "experimental" StringDtype because it had been available for so long. So it appears others seem to have changed the API without any deprecation or warning. Hopefully this will be clarified in the roadmap discussion.

[simonjayhawkins]

for the "Comparisons between different string dtypes" is there an issue ref?

#61138 is the PR, #60639 the issue. Will add a link

great.

Let's just do that for now. No need to block on the rest of my comment.

[jorisvandenbossche]

and the fact that you were "required" to do PDEP-14

Just to clarify here: this behaviour stems from before PDEP-14, and it is only for the NA-variant of the dtype, not for the future-default NaN-variant (so that's another reason that resolving this specific item is not a priority for 2.3)

[jorisvandenbossche]

Putting it here now just because I looked it up (but further not related to the content of this PR): the change for returning bool[pyarrow] instead of BooleanDtype was done in 2.0 in #51643, triggered by doing something similar for value_counts returning int64[pyarrow] instead of Int64 (#51542). This came up again at #59330 (comment) and then Will created #59346 to discuss (but we haven't actually further discussed it)

[jorisvandenbossche]

/preview

[github-actions]

Website preview of this PR available at: https://pandas.pydata.org/preview/pandas-dev/pandas/61794/