BUG: (v0.16.0) Series object being treated as a data frame object when using dt accessor

[susmitpy]

Modin version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest released version of Modin.

• I have confirmed this bug exists on the main branch of Modin. (In order to do this you can follow this guide.)

Reproducible Example

import modin.pandas as pd
df = pd.DataFrame(
            {
                "A": ["26/10/2020 11:00"],
                "B": ["30/10/2020 12:00"]
            }
)
df["A"] = pd.to_datetime(df["A"])
df["B"] = pd.to_datetime(df["B"])
s = df.eval("B-A", engine="python")
print(s.dt.days)

Issue Description

Upon performing an operation on two date time columns, when the dt accessor is used on the resulting series, even though the type of the variable "s" is "<class 'modin.pandas.series.Series'>". The error thrown is,

AttributeError: 'DataFrame' object has no attribute 'dt'

Expected Behavior

The output should be same as the output on version '0.15.3' and that is

0   4 days 01:00:00

Error Logs

2022-10-10 16:24:43,088	ERROR worker.py:84 -- Unhandled error (suppress with RAY_IGNORE_UNHANDLED_ERRORS=1): ray::_apply_func() (pid=12608, ip=127.0.0.1)
  File "/Users/susmit/.virtualenvs/venv/lib/python3.9/site-packages/modin/core/execution/ray/implementations/pandas_on_ray/partitioning/partition.py", line 411, in _apply_func
    result = func(partition, *args, **kwargs)
  File "/Users/susmit/.virtualenvs/venv/lib/python3.9/site-packages/modin/core/dataframe/algebra/map.py", line 48, in <lambda>
    lambda x: function(x, *args, **kwargs), *call_args, **call_kwds
  File "/Users/susmit/.virtualenvs/venv/lib/python3.9/site-packages/modin/core/storage_formats/pandas/query_compiler.py", line 155, in dt_op_builder
    prop_val = getattr(df.squeeze(axis=1).dt, property_name)
  File "/Users/susmit/.virtualenvs/venv/lib/python3.9/site-packages/pandas/core/generic.py", line 5907, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'dt'

Installed Versions

INSTALLED VERSIONS

commit : 621bc10
python : 3.9.13.final.0
python-bits : 64
OS : Darwin
OS-release : 21.6.0
Version : Darwin Kernel Version 21.6.0: Sat Jun 18 17:07:25 PDT 2022; root:xnu-8020.140.41~1/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : None
LOCALE : None.UTF-8

Modin dependencies

modin : 0.16.0
ray : 1.9.0
dask : None
distributed : None
hdk : None

pandas dependencies

pandas : 1.5.0
numpy : 1.21.2
pytz : 2022.1
dateutil : 2.8.2
setuptools : 65.3.0
pip : 22.2.2
Cython : None
pytest : 7.1.3
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : 2.9.4
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
brotli : None
fastparquet : None
fsspec : 2022.01.0
gcsfs : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : 3.0.10
pandas_gbq : None
pyarrow : 9.0.0
pyreadstat : None
pyxlsb : None
s3fs : 2022.01.0
scipy : 1.9.1
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None
tzdata : None

[pyrito]

@susmitpy thanks for opening the issue! I was able to reproduce this issue. It looks like it was working properly on 0.15.3, but not anymore on the latest version of Modin as you mentioned.

[pyrito]

This seems like an issue at the partitioning layer. It looks like the error is happening in filter_empties where we are trying to prune out empty partitions. However, since partitions are lazy, the metadata is materialized in the filter_empties method so it tries to apply the df accessor to the empty DataFrame which throws the error that we are seeing here.

@anmyachev was wondering if you had any thoughts here?

[mvashishtha]

In the reproducer in the first post, s is a 1-by-2 partitioned dataframe. The partition at [0,1] is empty. We implement dt.days as a Map across all partitions. We try to squeeze the empty pandas dataframe, but squeezing the empty dataframe pandas.DataFrame([], index=[0]) on axis=1 doesn't do anything. As the squeeze documentation says, squeezing only changes the object if the squeeze dimension(s) has (have) a single row.

We could probably fix this and str series properties by adding a special case for when the pandas dataframe in a partition has no columns.

cc @vnlitvinov as well since you helped get reduce usage of _filter_empties :)