az.compare() fails with the documentation example

[tomicapretto]

Describe the bug

az.compare raises a InvalidIndexError using an example from the documentation. I think it's because of the usage of the .at method in pandas DataFrames.

To Reproduce

import arviz as az

data1 = az.load_arviz_data("non_centered_eight")
data2 = az.load_arviz_data("centered_eight")
compare_dict = {"non centered": data1, "centered": data2}
az.compare(compare_dict)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File ~/anaconda3/envs/kulprit/lib/python3.9/site-packages/pandas/core/indexes/base.py:3800, in Index.get_loc(self, key, method, tolerance)
   3799 try:
-> 3800     return self._engine.get_loc(casted_key)
   3801 except KeyError as err:

File ~/anaconda3/envs/kulprit/lib/python3.9/site-packages/pandas/_libs/index.pyx:138, in pandas._libs.index.IndexEngine.get_loc()

File ~/anaconda3/envs/kulprit/lib/python3.9/site-packages/pandas/_libs/index.pyx:144, in pandas._libs.index.IndexEngine.get_loc()

TypeError: 'slice(None, None, None)' is an invalid key

During handling of the above exception, another exception occurred:

InvalidIndexError                         Traceback (most recent call last)
Input In [1], in <cell line: 5>()
      3 data2 = az.load_arviz_data("centered_eight")
      4 compare_dict = {"non centered": data1, "centered": data2}
----> 5 az.compare(compare_dict)

File ~/anaconda3/envs/kulprit/lib/python3.9/site-packages/arviz/stats/stats.py:306, in compare(compare_dict, ic, method, b_samples, alpha, seed, scale, var_name)
    304         std_err = ses.loc[val]
    305         weight = weights[idx]
--> 306         df_comp.at[val] = (
    307             idx,
    308             res[ic],
    309             res[p_ic],
    310             d_ic,
    311             weight,
    312             std_err,
    313             d_std_err,
    314             res["warning"],
    315             res[scale_col],
    316         )
    318 df_comp["rank"] = df_comp["rank"].astype(int)
    319 df_comp["warning"] = df_comp["warning"].astype(bool)

File ~/anaconda3/envs/kulprit/lib/python3.9/site-packages/pandas/core/indexing.py:2438, in _AtIndexer.__setitem__(self, key, value)
   2435     self.obj.loc[key] = value
   2436     return
-> 2438 return super().__setitem__(key, value)

File ~/anaconda3/envs/kulprit/lib/python3.9/site-packages/pandas/core/indexing.py:2393, in _ScalarAccessIndexer.__setitem__(self, key, value)
   2390 if len(key) != self.ndim:
   2391     raise ValueError("Not enough indexers for scalar access (setting)!")
-> 2393 self.obj._set_value(*key, value=value, takeable=self._takeable)

File ~/anaconda3/envs/kulprit/lib/python3.9/site-packages/pandas/core/frame.py:4208, in DataFrame._set_value(self, index, col, value, takeable)
   4206     iindex = cast(int, index)
   4207 else:
-> 4208     icol = self.columns.get_loc(col)
   4209     iindex = self.index.get_loc(index)
   4210 self._mgr.column_setitem(icol, iindex, value)

File ~/anaconda3/envs/kulprit/lib/python3.9/site-packages/pandas/core/indexes/base.py:3807, in Index.get_loc(self, key, method, tolerance)
   3802         raise KeyError(key) from err
   3803     except TypeError:
   3804         # If we have a listlike key, _check_indexing_error will raise
   3805         #  InvalidIndexError. Otherwise we fall through and re-raise
   3806         #  the TypeError.
-> 3807         self._check_indexing_error(key)
   3808         raise
   3810 # GH#42269

File ~/anaconda3/envs/kulprit/lib/python3.9/site-packages/pandas/core/indexes/base.py:5963, in Index._check_indexing_error(self, key)
   5959 def _check_indexing_error(self, key):
   5960     if not is_scalar(key):
   5961         # if key is not a scalar, directly raise an error (the code below
   5962         # would convert to numpy arrays and raise later any way) - GH29926
-> 5963         raise InvalidIndexError(key)

InvalidIndexError: slice(None, None, None)

Expected behavior

This should work. The iteration that is failing should fill rows of the empty df_comp DataFrame.

Additional context

import arviz as az
import pandas as pd
print(az.__version__)
print(pd.__version__)

0.12.1
1.5.0

See this issue in pandas pandas-dev/pandas#48296. Looks like it's a behavior that changed in pandas 1.5.0, which is the version I'm using. Maybe there's something better than using .at.

[ahartikainen]

Thanks for the report.

Issue was fixed in #2104

Edit.

Please update to the latest version

python -m pip install arviz

[tomicapretto]

Sorry! I must have checked an older version of the source code because I thought it was still using the .at. Thanks for the fix!