You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When running the quality report, the "Column Pair Trends" property is now producing a warning message that goes on for hundreds of lines. It has to do with converting datetime columns to Unix timestamps.
Steps to reproduce
The code below uses the SDV to evaluate quality, which calls the Quality Report.
Generating report ...
(1/3) Evaluating Column Shapes: : 100%|██████████| 15/15 [00:00<00:00, 345.64it/s]
(2/3) Evaluating Column Pair Trends: : 20%|██ | 11/55 [00:00<00:00, 95.07it/s]/usr/local/lib/python3.10/dist-packages/sdmetrics/reports/single_table/_properties/column_pair_trends.py:58: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '[1609027200000000000 1609286400000000000 1600300800000000000
1609113600000000000 1586044800000000000 1602979200000000000
1606003200000000000 1583280000000000000 1578268800000000000
1579651200000000000 1592179200000000000 1603411200000000000
1583539200000000000 1597190400000000000 1601510400000000000
1595116800000000000 1594684800000000000 1583107200000000000
1594857600000000000 1590883200000000000 1590796800000000000
1602547200000000000 1590105600000000000 1599955200000000000
1595808000000000000 1606694400000000000 1598400000000000000
1593907200000000000 1607385600000000000 1604534400000000000
...
'] has dtype incompatible with datetime64[ns], please explicitly cast to a compatible dtype first.
data.loc[~pd.isna(data[column_name]), column_name] = pd.to_numeric(
/usr/local/lib/python3.10/dist-packages/sdmetrics/reports/single_table/_properties/column_pair_trends.py:58: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '[1609113600000000000 1609372800000000000 1600473600000000000
Additional Context
We think this is because we are only overriding the non-null items to numerical values. Meanwhile, the null items are still left as pd.NaT which is a datetime type. This makes the column mixed type (datetime and numeric)
The text was updated successfully, but these errors were encountered:
Environment Details
Error Description
When running the quality report, the
"Column Pair Trends"
property is now producing a warning message that goes on for hundreds of lines. It has to do with converting datetime columns to Unix timestamps.Steps to reproduce
The code below uses the SDV to evaluate quality, which calls the Quality Report.
Output:
Additional Context
We think this is because we are only overriding the non-null items to numerical values. Meanwhile, the null items are still left as
pd.NaT
which is a datetime type. This makes the column mixed type (datetime and numeric)The text was updated successfully, but these errors were encountered: