Thanks for the report. I'd hazard a guess that we are determining the dtype of the result prior to column selection. Further investigations are welcome!

take

take

Hi @parthi-siva and @DarthKitten2130 ,
Are you still working on this issue? I would like to work on this one if you don't mind.

Hi @sanggon6107 I'm still working on this..

Hi @sanggon6107 I'm still working on this..

Well noted. Thanks for the quick reply.

for this input

df = pd.DataFrame([['a',1.,2.],['b',3.,4.]])
df.loc[0,[1,2]].dtypes

pandas.core.array_algos.take._take_preprocess_indexer_and_fill_value:131 (Here we get the data type for the resulting series. )

 dtype, fill_value, mask_info = _take_preprocess_indexer_and_fill_value(
        arr, indexer, fill_value, allow_fill
    )

arr = [a, 1, 2]
indexer = [1,2]

as we can see that arr contains string so the datatype returned will be object only.

Then we are creating empty numpy array using the dtype which will be of type object

pandas.core.array_algos.take._take_preprocess_indexer_and_fill_value:155

out = np.empty(out_shape, dtype=dtype) 

Then we do slice using cpython function

pandas.core.array_algos.take._take_preprocess_indexer_and_fill_value:160

func(arr, indexer, out, fill_value)

After the func(arr, indexer, out, fill_value) call, the out array is populated with the selected elements. However, the dtype of out will not match the dtype of the elements in arr.

I tried to add a step to check and adjust the dtype of out after the func call.

  # Check if the dtype of out matches the dtype of the elements in arr
    if out.size > 0:  # Only check if out is not empty
        first_element = out.flat[0]  # Get the first element

        # Check if the first element's type is different from out.dtype
        if isinstance(first_element, (int, float, np.number)) and out.dtype == object:
            # If the first element is numeric but out.dtype is object, update the dtype
            new_dtype = np.result_type(first_element)
            out = out.astype(new_dtype)

This fixed the op's issue but test cases are failing. Also I feel this is not a right way to address the issue

So I'm not sure how to infer the dtype pragmatically before this for df.loc[0,[1,2]]

@rhshadrach @sanggon6107

Hi @parthi-siva,
thanks for the comment.

I had also tried similar thing, but it seems there could be side effects including test failures, since there could be many other pandas functions that call take_nd().
I would rather change some codes at the relatively outer level of the call stack so that we can minimize the impact.
Since it seems this issue only appears where the first axis is integer and the second one is list or slice - loc[int,list/slice], I think we could re-interpret the dtype of the output at the end of _LocationIndexer._getitem_lowerdim().

Proposed solution

    @final
    def _getitem_lowerdim(self, tup: tuple):

...

                # This is an elided recursive call to iloc/loc
                out = getattr(section, self.name)[new_key]
                # Re-interpret dtype of out.values for loc/iloc[int, list/slice]. # GH60600
                if i == 0 and isinstance(key, int) and isinstance(new_key, (list, slice)):
                    inferred_dtype = np.array(out.values.tolist()).dtype
                    if inferred_dtype != out.dtype:
                        out = out.astype(inferred_dtype)
                return out

There was only one failing test when I locally ran pytest, but the failing case should be revised according to this code change since the test is currently expecting loc[int, list] to be an object dataframe.
My concern is that, we have to create a new np.array only to re-interpret the dtype. I'm not sure if there's more elegant way to infer the output's dtype.

Please let me know what you think about the proposal. I'd be glad to co-author a commit and make a PR if you don't mind.

cc @rhshadrach

Thanks!

Hi @sanggon6107 ,

Thanks for the reply.

Pls proceed with you proposal. I'm good!

I spent some time regarding your concern about creating a np.array just to find dtype.

can we try using np.result_type

either like this

inferred_dtype = reduce(np.result_type, out)

or like this

inferred_dtype = np.result_type(out.values.tolist())

Please let me know if it helps.

@sanggon6107 - it's not clear to me what the proposal is. Best to open a PR I think.

Hi @parthi-siva , your suggestion helped a lot!

I've also found that we could simplify the code by using infer_objects().
I'll make a PR based on this discussion.

take

Hi @parthi-siva , your suggestion helped a lot!

I've also found that we could simplify the code by using infer_objects().
I'll make a PR based on this discussion.

Sure @sanggon6107 :)