BUG: pd.unique(index) does not always return an index

[mvashishtha]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

# unique on this datetime index returns another index:
# DatetimeIndex(['2016-01-01 00:00:00-05:00'], dtype='datetime64[ns, US/Eastern]', freq=None)
pd.unique(pd.Index([pd.Timestamp("20160101", tz="US/Eastern")]))

# unique on this datetime index returns a numpy array:
# array(['2016-01-01T00:00:00.000000000'], dtype='datetime64[ns]')
pd.unique(pd.Index([pd.Timestamp("20160101")]))

# unique on this int index returns a numpy array:
# array([1])
pd.unique(pd.Index([1]))

Issue Description

pd.unique documentation says the return value is Index : when the input is an Index, but the return value is sometimes a numpy array.

Expected Behavior

pd.unique should return an index of the same type as the one provided to it.

Installed Versions

INSTALLED VERSIONS

commit : fd3f571
python : 3.9.18.final.0
python-bits : 64
OS : Darwin
OS-release : 23.2.0
Version : Darwin Kernel Version 23.2.0: Wed Nov 15 21:55:06 PST 2023; root:xnu-10002.61.3~2/RELEASE_ARM64_T6020
machine : arm64
processor : arm
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 2.2.0
numpy : 1.26.3
pytz : 2023.3.post1
dateutil : 2.8.2
setuptools : 68.2.2
pip : 23.3.1
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : 8.18.1
pandas_datareader : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
bottleneck : None
dataframe-api-compat : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2023.4
qtpy : None
pyqt5 : None

[rhshadrach]

Thanks for the report, confirmed on main. Further investigations and PRs to fix are welcome!

[aidoskanapyanov]

Hi, I found that pd.Index infers its dtype as ExtensionDtype (i.e. DatetimeTZDtype) only for one of them, and for the rest it infers as numpy.dtypes:

>>> print(pd.Index([pd.Timestamp("20160101", tz="US/Eastern")]).dtype.__class__)
<class 'pandas.core.dtypes.dtypes.DatetimeTZDtype'>
>>> print(pd.Index([pd.Timestamp("20160101")]).dtype.__class__)
<class 'numpy.dtypes.DateTime64DType'>
>>> print(pd.Index([1]).dtype.__class__)
<class 'numpy.dtypes.Int64DType'>

Therefore, in pd.unique it returns as ndarray.

Here are the lines in pd.unique where it dispatches to extension dtype's unique implementation, which only works for DatetimeTZDtype example:

pandas/pandas/core/algorithms.py

Lines 433 to 442 in d928a5c

	def unique_with_mask(values, mask: npt.NDArray[np.bool_] | None = None):
	"""See algorithms.unique for docs. Takes a mask for masked arrays."""
	values = _ensure_arraylike(values, func_name="unique")
	if isinstance(values.dtype, ExtensionDtype):
	# Dispatch to extension dtype's unique.
	return values.unique()
	original = values
	hashtable, values = _get_hashtable_algo(values)

[aidoskanapyanov]

It is possible to resolve the issue if we provide the dtype or dtype string alias manually to pd.Index like so:

>>> pd.unique(pd.Index([pd.Timestamp("20160101")], dtype='datetime64[ns, US/Eastern]'))
DatetimeIndex(['2016-01-01 00:00:00-05:00'], dtype='datetime64[ns, US/Eastern]', freq=None)
>>> pd.unique(pd.Index([1], dtype='Int32'))
Index([1], dtype='Int32')