pvlib.forecast compatibility with cftime 1.1

[dankestship]

Hi, I have been using PVLIB to model a solar array in Hawaii. I just updated my computer to the new catalina software. I reinstalled pvlib on my system and now when I run my code to get the weather data the following error occurs:

TypeError: <class 'cftime._cftime.DatetimeGregorian'> is not convertible to datetime

I have been trying to figure out why this is happening. Below is my code. Thank you for any help!!

```python # built-in python modules import datetime import inspect import os import pytz

scientific python add-ons

import numpy as np
import pandas as pd

plotting

first line makes the plots appear in the notebook

%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib as mpl

#import the pvlib library
from pvlib import solarposition,irradiance,atmosphere,pvsystem
from pvlib.forecast import GFS
from pvlib.modelchain import ModelChain

pd.set_option('display.max_rows', 500)
latitude, longitude, tz = 21.300268, -157.818044, 'Pacific/Honolulu'

# specify time range.

start2 = pd.Timestamp(datetime.date.today(), tz=tz)

pacific = pytz.timezone('Etc/GMT+10')

# print(pacific)

# datetime.datetime(year, month, day, hour, minute, second, microsecond, tzinfo)

start2 = pd.Timestamp(datetime.datetime(2020, 2, 14, 12, 0, 0, 0, pacific))

# print(start)

# print(start2)

# print(datetime.date.today())

end = start2 + pd.Timedelta(days=1.5)

pacific = pytz.timezone('Etc/GMT+10')

# use python datetime and pytz correctly

start2 = pd.Timestamp(pacific.localize(datetime.datetime(2020, 2, 14, 12, 0, 0, 0)))

naive python datetime, let pandas/pytz handle tz

start2 = pd.Timestamp(datetime.datetime(2020, 2, 17, 12, 0, 0, 0), tz=pacific)

# skip pytz and just use pandas

pacificstr = 'Etc/GMT+10'

start2 = pd.Timestamp(2020, 2, 14, 12, 0, 0, 0, tz=pacificstr)

end = start2 + pd.Timedelta(days=1.5)

from datetime import datetime

start2 = datetime.now()
end = start2 + pd.Timedelta(days=1.5)

print(type(start2))
print(type(end))

Define forecast model

fm = GFS()

get data from location specified above

forecast_data = fm.get_processed_data(latitude, longitude, start2, end)

print(forecast_data)

Full stack trace of error

TypeError Traceback (most recent call last)
in
3
4 # get data from location specified above
----> 5 forecast_data = fm.get_processed_data(latitude, longitude, start2, end)
6 # print(forecast_data)

~/opt/anaconda3/lib/python3.7/site-packages/pvlib/forecast.py in get_processed_data(self, *args, **kwargs)
306 Processed forecast data
307 """
--> 308 return self.process_data(self.get_data(*args, **kwargs), **kwargs)
309
310 def rename(self, data, variables=None):

~/opt/anaconda3/lib/python3.7/site-packages/pvlib/forecast.py in get_data(self, latitude, longitude, start, end, vert_level, query_variables, close_netcdf_data, **kwargs)
264 # higher dimensional data for more advanced applications
265 self.data = self._netcdf2pandas(self.netcdf_data, self.query_variables,
--> 266 self.start, self.end)
267
268 if close_netcdf_data:

~/opt/anaconda3/lib/python3.7/site-packages/pvlib/forecast.py in _netcdf2pandas(self, netcdf_data, query_variables, start, end)
349 try:
350 time_var = 'time'
--> 351 self.set_time(netcdf_data.variables[time_var])
352 except KeyError:
353 # which model does this dumb thing?

~/opt/anaconda3/lib/python3.7/site-packages/pvlib/forecast.py in set_time(self, time)
391 '''
392 times = num2date(time[:].squeeze(), time.units)
--> 393 self.time = pd.DatetimeIndex(pd.Series(times), tz=self.location.tz)
394
395 def cloud_cover_to_ghi_linear(self, cloud_cover, ghi_clear, offset=35,

~/opt/anaconda3/lib/python3.7/site-packages/pandas/core/indexes/datetimes.py in new(cls, data, freq, start, end, periods, tz, normalize, closed, ambiguous, dayfirst, yearfirst, dtype, copy, name, verify_integrity)
332 yearfirst=yearfirst,
333 ambiguous=ambiguous,
--> 334 int_as_wall_time=True,
335 )
336

~/opt/anaconda3/lib/python3.7/site-packages/pandas/core/arrays/datetimes.py in _from_sequence(cls, data, dtype, copy, tz, freq, dayfirst, yearfirst, ambiguous, int_as_wall_time)
444 yearfirst=yearfirst,
445 ambiguous=ambiguous,
--> 446 int_as_wall_time=int_as_wall_time,
447 )
448

~/opt/anaconda3/lib/python3.7/site-packages/pandas/core/arrays/datetimes.py in sequence_to_dt64ns(data, dtype, copy, tz, dayfirst, yearfirst, ambiguous, int_as_wall_time)
1864 # or M8[ns] to denote wall times
1865 data, inferred_tz = objects_to_datetime64ns(
-> 1866 data, dayfirst=dayfirst, yearfirst=yearfirst
1867 )
1868 tz = maybe_infer_tz(tz, inferred_tz)

~/opt/anaconda3/lib/python3.7/site-packages/pandas/core/arrays/datetimes.py in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, require_iso8601, allow_object)
1973 dayfirst=dayfirst,
1974 yearfirst=yearfirst,
-> 1975 require_iso8601=require_iso8601,
1976 )
1977 except ValueError as e:

pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()

pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()

pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime_object()

pandas/_libs/tslib.pyx in pandas._libs.tslib.array_to_datetime()

TypeError: <class 'cftime._cftime.DatetimeGregorian'> is not convertible to datetime

</details>

[cwhanse]

Very hard to tell with all the repeated lines. But it's clear that it's related to the datetimes and timezone localization.

start2 = pd.Timestamp(2020, 2, 14, 12, 0, 0, 0, tz=pacificstr)

This line will fail with a different error, which tells me that I can't reproduce the error you see. Use named argument names, you are passing positional values to the wrong arguments.

Once you can narrow the code down to isolate the problem, please post to SO or the Google group for help.

[dankestship]

Hi Cliff, Thanks for the help and suggestions to fix my problem. I tried to run the example code from https://pvlib-python.readthedocs.io/en/stable/forecasts.html. I still get the error "TypeError: <class 'cftime._cftime.DatetimeGregorian'> is not convertible to datetime", the code I used is below and I attached a jupyter notebook of the example. Any help in solving this issue would be greatly appreciated!! import pandas as pd import matplotlib.pyplot as plt import datetime # import pvlib forecast models from pvlib.forecast import GFS, NAM, NDFD, HRRR, RAP # specify location (Tucson, AZ) latitude, longitude, tz = 32.2, -110.9, 'US/Arizona' # specify time range. start = pd.Timestamp(datetime.date.today(), tz=tz) end = start + pd.Timedelta(days=7) model = GFS() # retrieve data. returns pandas.DataFrame object raw_data = model.get_data(latitude, longitude, start, end) print(raw_data.head())

[cwhanse]

I can't reproduce that error, having firewall issues. I'm guessing that there's a clash among python packages. Which versions of pandas, numpy and datetime are you using?

[kandersolar]

@dankestship I only get this error when using cftime==1.1.0 (released Feb 13). Downgrading to the prior release with pip install cftime==1.0.4.2 fixes the issue on my computer.

@wholmgren Looks like this was the relevant change: Unidata/cftime#136 -- possible fix is to specify only_use_cftime_datetimes=False here: https://github.com/pvlib/pvlib-python/blob/master/pvlib/forecast.py#L409

[dankestship]

Thank you both for all your help!! Downgrading to a prior release fixed the issue and the example code now runs without error. Thank you again!

…

On Tue, Feb 18, 2020 at 6:23 AM Kevin Anderson ***@***.***> wrote: @dankestship <https://github.com/dankestship> I only get this error when using cftime==1.1.0 (released Feb 13). Downgrading to the prior release with pip install cftime==1.0.4.2 fixes the issue on my computer. @wholmgren <https://github.com/wholmgren> Looks like this was the relevant change: Unidata/cftime#136 <Unidata/cftime#136> -- possible fix is to specify only_use_cftime_datetimes=False here: https://github.com/pvlib/pvlib-python/blob/master/pvlib/forecast.py#L409 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#895?email_source=notifications&email_token=AOSUNZTV63LGGOSHT2XQJQLRDQDR3A5CNFSM4KWXMVGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMCT7SY#issuecomment-587546571>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOSUNZVTVOI5I2JLVP4C4STRDQDR3ANCNFSM4KWXMVGA> .