[tradingcal] [bugfix] Index.searchsorted() method accepts only pandas timestamps in v1.0 of the library.

[vladisld]

Description

PandasMarketCalendar class uses the pandas dataframe to cache the trading dates schedule. Unfortunately in schedule method a wrong data type is used to search in this cache in case the pandas library v1.0 is used.

def schedule(self, day, tz=None):
        '''
        Returns the opening and closing times for the given ``day``. If the
        method is called, the assumption is that ``day`` is an actual trading
        day

        The return value is a tuple with 2 components: opentime, closetime
        '''
        while True:
            #### The problem is here #####
            i = self.idcache.index.searchsorted(ay.date())

            if i == len(self.idcache):
                # keep a cache of 1 year to speed up searching
                self.idcache = self._calendar.schedule(day, day + self.csize)
                continue

            st = (x.tz_localize(None) for x in self.idcache.iloc[i, 0:2])
            opening, closing = st  # Get utc naive times
            if day > closing:  # passed time is over the sessionend
                day += ONEDAY  # wrap over to next day
                continue

            return opening.to_pydatetime(), closing.to_pydatetime()

Unexpected behavior

I don't remember exactly whether or not an exception is raised or the search just return empty result

[neilsmurphy]

I think you're right. day.date() needs to go to pandas datetime. The pandas module library gets imported in init of PandasMarketCalendar. I set an attribute to the library like this at line 236:

import pandas as pd  # guaranteed because of pandas_market_calendars
self.pandas = pd

And then use this to convert the day.date() value:

i = self.idcache.index.searchsorted(self.pandas.to_datetime(day.date()))

Seems to fix the error but I haven't checked if it's working yet. Does this method make sense or is there a better way to convert the date?

[Mkima]

I think you're right. day.date() needs to go to pandas datetime. The pandas module library gets imported in init of PandasMarketCalendar. I set an attribute to the library like this at line 236:

import pandas as pd  # guaranteed because of pandas_market_calendars
self.pandas = pd

And then use this to convert the day.date() value:

i = self.idcache.index.searchsorted(self.pandas.to_datetime(day.date()))

Seems to fix the error but I haven't checked if it's working yet. Does this method make sense or is there a better way to convert the date?

Hi, I've verified this one works for me as well. who can help committing this fix?

[vladisld]

I'm using the following patch in my fork - seems to be working for my system.

i = self.idcache.index.searchsorted(pd.Timestamp(day.date()))

Anyway, an appropriate test should be provided with any PR.

[zhemaituk]

It seems that timezones are not handled in the schedule method: it ignores tz parameter, expects tz-naive date, and returns times in UTC.

Internal _calendar though does not seem to have any issues with dates or timezones. Below code works, but public API would be preferred of course:

schedule = nyse_calendar._calendar.schedule(market_day, market_day, eastern_tz)
market_open = schedule['market_open'][0].time()
market_close = schedule['market_close'][0].time()

[ultra1971]

I have started to use trading calendar with pandas_trading_calendar, the solution proposed fixed the issue for me.
Thanks

[sohamsjain]

Line no. 251 in the _nextday method in tradingcal.py throws this error.

TypeError: value should be a 'Timestamp', 'NaT', or array of those. Got 'date' instead.

A quickfix that worked is converting the day to a pandas Timestamp:

# original
i = self.dcache.searchsorted(day)

# fix
i = self.dcache.searchsorted(pd.Timestamp(day))

In case your calendar is running on a different timezone, you're likely to encounter something like this as well:

TypeError: Cannot compare tz-naive and tz-aware datetime-like objects

Here's a workaround for it.

i = self.dcache.searchsorted(pd.Timestamp(day).tz_localize(self.dcache.tz))

Note: I've imported pandas within the module level imports.

The above fixes work well for me.
Do give it a check