BUG: error message appearing in debugging mode

[PabloRuizCuevas]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of staircase.

---

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

# This in the code
import staircase as sc
import pandas as pd

# this run in debug interactive mode
pd.DataFrame()

Problem description

I'm still trying to make a minimum reproducible example, but for me it seems that somehow the only inclusion of the staircase dependence even without using it, is changing some pandas behaviour and raising an error. The error is non breaking and it appears when debugging code (interactively) that contains pandas dataframes even if staircase is not actually used in any operation and only imported at some point.

The problem appears with delay and is not breaking, I'm not sure how to find the issue.

Expected Output

Nothing

Dependency Versions``

staircase 2.4.2
pandas 1.4.1
numpy 1.22.2

Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm Community Edition 2021.1.1\plugins\python-ce\helpers\pydev_pydevd_bundle\pydevd_comm.py", line 1590, in do_it
msg = _pydev_completer.generate_completions_as_xml(frame, self.act_tok)
File "C:\Program Files\JetBrains\PyCharm Community Edition 2021.1.1\plugins\python-ce\helpers\pydev_pydev_bundle_pydev_completer.py", line 172, in generate_completions_as_xml
completions = completer.complete(act_tok)
File "C:\Program Files\JetBrains\PyCharm Community Edition 2021.1.1\plugins\python-ce\helpers\pydev_pydev_bundle_pydev_completer.py", line 89, in complete
return self.attr_matches(text)
File "C:\Program Files\JetBrains\PyCharm Community Edition 2021.1.1\plugins\python-ce\helpers\pydev_pydev_bundle_pydev_completer.py", line 151, in attr_matches
words = dir2(obj, filter=filter)
File "C:\Program Files\JetBrains\PyCharm Community Edition 2021.1.1\plugins\python-ce\helpers\pydev_pydev_bundle_pydev_imports_tipper.py", line 178, in generate_imports_tip_for_module
dir_comps = dir(obj_to_complete)
File "C:\Users\PabloRuiz\AppData\Local\pypoetry\Cache\virtualenvs\transformer-_p2xsS-z-py3.8\lib\site-packages\pandas\core\accessor.py", line 40, in dir
rv = (rv - self._dir_deletions()) | self._dir_additions()
File "C:\Users\PabloRuiz\AppData\Local\pypoetry\Cache\virtualenvs\transformer-_p2xsS-z-py3.8\lib\site-packages\pandas\core\generic.py", line 5632, in _dir_additions
additions = super()._dir_additions()
File "C:\Users\PabloRuiz\AppData\Local\pypoetry\Cache\virtualenvs\transformer-_p2xsS-z-py3.8\lib\site-packages\pandas\core\accessor.py", line 29, in _dir_additions
return {accessor for accessor in self._accessors if hasattr(self, accessor)}
File "C:\Users\PabloRuiz\AppData\Local\pypoetry\Cache\virtualenvs\transformer-_p2xsS-z-py3.8\lib\site-packages\pandas\core\accessor.py", line 29, in
return {accessor for accessor in self._accessors if hasattr(self, accessor)}
File "C:\Users\PabloRuiz\AppData\Local\pypoetry\Cache\virtualenvs\transformer-_p2xsS-z-py3.8\lib\site-packages\pandas\core\accessor.py", line 182, in get
accessor_obj = self._accessor(obj)
File "C:\Users\PabloRuiz\AppData\Local\pypoetry\Cache\virtualenvs\transformer-_p2xsS-z-py3.8\lib\site-packages\staircase\core\arrays\accessor.py", line 14, in init
raise TypeError(
TypeError: sc accessor only valid for Series with Stairs dtype. Convert using .astype("Stairs").

I think the problem is related to the word "sc" in the import and:

@register_series_accessor("sc")
class StairsAccessor:
...

in accessor.

[venaturum]

Hi @PabloRuizCuevas , can you check the line from staircase import as sc, I presume it has a typo

[PabloRuizCuevas]

Hi @venaturum , yes, sorry but the typo is only here in the issue,

I have been looking for this issue a lot, and I'm still extremely lost with it. It after importing staircase but also will appear with just "from staircase import Stairs" and not doing absolutely anything with staircase, but only seems to appear in debugging mode, it does not break anything and it also happens at a random timing.

Is there any thread in staircase working in the background?

I will try to make a couple of test when I have some time.

[venaturum]

Staircase is just vanilla python and pandas code, nothing async happening. I'm suspecting this may be limited to PyCharm - I'll download it and see if it also occurs for me. It could be that the python interpreter for debugging in PyCharm is doing some non standard stuff and we're getting clashes in the namespace that wouldn't ordinarily happen.

[PabloRuizCuevas]

Yes it may be related to PyCharm, so no worries, let's this issue open for now and I will try to send you a complete example if I can do it in a reproducible way.

[venaturum]

Is this what you are referring to when you say interactive debug mode?

[venaturum]

Made some progress on this and can trigger the bug without the need for interactive or debugging.

> import pandas as pd
> import staircase as sc
> import cyberpandas

> x = pd.Series([1])
> dir(x)
TypeError ... # this reproduces the error

> x._accessors
{'cat', 'dt', 'ip', 'sc', 'sparse', 'str'}

> x.sc
TypeError  # thrown by staircase code - complaining because x is not StairsDType (as it should)

> x.ip
AttributeError... # thrown by cyberpandas code - complaining because x is not IPType (as it should)

> hasattr(x, "ip")
False

> hasattr(x, "sc")   # this is the root cause problem
TypeError...

The error is caused by Series inspecting it's accessors, which is a list of strings, and then using hasattr to check if there is an attribute on Series with the same name as the accessor. The path to understanding the fix probably involves understanding the difference between ip and sc accessors and why hasattr(x, "ip") results in a boolean but hasattr(x, "sc") results in an error.

[venaturum]

Found a solution, and not 100% sure why it works.... but probably don't need to. When staircase code finds a Series which is not of StairsDType it throws a TypeError. If it throws an AttributeError (like cyberpandas does) then we no longer see the above bug. Somewhere in the depths of pandas code it is handling TypeErrors thrown by accessors but not AttributeErrors.

I'll do a bugfix. Thanks for bringing it to my attention!

[PabloRuizCuevas]

Great, incredible that you found it! specially with the vague description that I could provide. For me it was extremely intriguing, but I could not figure it out where did it come from. I will wait to the bug fix, let me know if I can help, from my side I will test that it doesn't appear any more.

[venaturum]

I think I got lucky - wasn't able to reproduce it in PyCharm like I'd hoped but then happened to get the same error when doing the day job in VSCode so quite the coincidence. Perhaps something has changed in more recent versions of pandas which leads to the bug. Thanks for taking the time to document it! The fix has been merged into the master branch along with a few other changes mostly concerned with docs generation. I'm going to make a couple of more changes in the master in prep for a patch release, and then its probably a good idea for you to pull the changes into your fork (I'll ping you once I'm done).

[venaturum]

@PabloRuizCuevas All good to sync fork. No changes to environment.

[PabloRuizCuevas]

@venaturum Great! thanks for the quick solution :) I will sync it as soon as I have some time.