Dealing with time and time zones can be a frustrating experience in any programming language and for any application. pvlib-python relies on :pypandas
and pytz to handle time and time zones. Therefore, the vast majority of the information in this document applies to any time series analysis using pandas and is not specific to pvlib-python.
pvlib makes extensive use of pandas due to its excellent time series functionality. Take the time to become familiar with pandas' Time Series / Date functionality page. It is also worthwhile to become familiar with pure Python's :pypython:datetime
module, although we usually recommend using the corresponding pandas functionality where possible.
First, we'll import the libraries that we'll use to explore the basic time and time zone functionality in python and pvlib.
python
import datetime import pandas as pd import pytz
pytz is based on the Olson time zone database. You can obtain a list of all valid time zone strings with pytz.all_timezones
. It's a long list, so we only print every 20th time zone.
python
len(pytz.all_timezones) pytz.all_timezones[::20]
Wikipedia's List of tz database time zones is also good reference.
The pytz.country_timezones
function is useful, too.
python
pytz.country_timezones('US')
And don't forget about Python's :pypython:filter
function.
python
list(filter(lambda x: 'GMT' in x, pytz.all_timezones))
Note that while pytz has 'EST'
and 'MST'
, it does not have 'PST'
. Use 'Etc/GMT+8'
instead, or see fixedoffsets
.
:pypandas.Timestamp
and :pypandas.DatetimeIndex
can be created in many ways. Here we focus on the time zone issues surrounding them; see the pandas documentation for more information.
First, create a time zone naive pandas.Timestamp.
python
pd.Timestamp('2015-1-1 00:00')
You can specify the time zone using the tz
keyword argument or the tz_localize
method of Timestamp and DatetimeIndex objects.
python
pd.Timestamp('2015-1-1 00:00', tz='America/Denver') pd.Timestamp('2015-1-1 00:00').tz_localize('America/Denver')
Localized Timestamps can be converted from one time zone to another.
python
midnight_mst = pd.Timestamp('2015-1-1 00:00', tz='America/Denver') corresponding_utc = midnight_mst.tz_convert('UTC') # returns a new Timestamp corresponding_utc
It does not make sense to convert a time stamp that has not been localized, and pandas will raise an exception if you try to do so.
python
midnight = pd.Timestamp('2015-1-1 00:00') midnight.tz_convert('UTC')
The difference between tz_localize
and tz_convert
is a common source of confusion for new users. Just remember: localize first, convert later.
Some time zones are aware of daylight savings time and some are not. For example the winter time results are the same for US/Mountain and MST, but the summer time results are not.
Note the UTC offset in winter...
python
pd.Timestamp('2015-1-1 00:00').tz_localize('US/Mountain') pd.Timestamp('2015-1-1 00:00').tz_localize('Etc/GMT+7')
vs. the UTC offset in summer...
python
pd.Timestamp('2015-6-1 00:00').tz_localize('US/Mountain') pd.Timestamp('2015-6-1 00:00').tz_localize('Etc/GMT+7')
pandas and pytz make this time zone handling possible because pandas stores all times as integer nanoseconds since January 1, 1970. Here is the pandas time representation of the integers 1 and 1e9.
python
pd.Timestamp(1) pd.Timestamp(1e9)
So if we specify times consistent with the specified time zone, pandas will use the same integer to represent them.
python
# US/Mountain pd.Timestamp('2015-6-1 01:00', tz='US/Mountain').value
# MST pd.Timestamp('2015-6-1 00:00', tz='Etc/GMT+7').value
# Europe/Berlin pd.Timestamp('2015-6-1 09:00', tz='Europe/Berlin').value
# UTC pd.Timestamp('2015-6-1 07:00', tz='UTC').value
# UTC pd.Timestamp('2015-6-1 07:00').value
It's ultimately these integers that are used when calculating quantities in pvlib such as solar position.
As stated above, pandas will assume UTC if you do not specify a time zone. This is dangerous, and we recommend using localized timeseries, even if it is UTC.
The 'Etc/GMT*'
time zones mentioned above provide fixed offset specifications, but watch out for the counter-intuitive sign convention.
python
pd.Timestamp('2015-1-1 00:00', tz='Etc/GMT-2')
Fixed offset time zones can also be specified as offset minutes from UTC using pytz.FixedOffset
.
python
pd.Timestamp('2015-1-1 00:00', tz=pytz.FixedOffset(120))
You can also specify the fixed offset directly in the tz_localize
method, however, be aware that this is not documented and that the offset must be in seconds, not minutes.
python
pd.Timestamp('2015-1-1 00:00', tz=7200)
Yet another way to specify a time zone with a fixed offset is by using the string formulation.
python
pd.Timestamp('2015-1-1 00:00+0200')
Sometimes it's convenient to use native Python :pypython:datetime.date
and :pypython:datetime.datetime
objects, so we demonstrate their use next. pandas Timestamp objects can also be created from time zone aware or naive :pypython:datetime.datetime
objects. The behavior is as expected.
python
# tz naive python datetime.datetime object naive_python_dt = datetime.datetime(2015, 6, 1, 0)
# tz naive pandas Timestamp object pd.Timestamp(naive_python_dt)
# tz aware python datetime.datetime object aware_python_dt = pytz.timezone('US/Mountain').localize(naive_python_dt)
# tz aware pandas Timestamp object pd.Timestamp(aware_python_dt)
One thing to watch out for is that python :pypython:datetime.date
objects gain time information when passed to Timestamp
.
python
# tz naive python datetime.date object (no time info) naive_python_date = datetime.date(2015, 6, 1)
# tz naive pandas Timestamp object (time=midnight) pd.Timestamp(naive_python_date)
You cannot localize a native Python date object.
python
# fail pytz.timezone('US/Mountain').localize(naive_python_date)
How does this general functionality interact with pvlib? Perhaps the two most common places to get tripped up with time and time zone issues in solar power analysis occur during data import and solar position calculations.
Let's first examine how pvlib handles time when it imports a TMY3 file.
python
import os import inspect import pvlib
# some gymnastics to find the example file pvlib_abspath = os.path.dirname(os.path.abspath(inspect.getfile(pvlib))) file_abspath = os.path.join(pvlib_abspath, 'data', '703165TY.csv') tmy3_data, tmy3_metadata = pvlib.iotools.read_tmy3(file_abspath, map_variables=True)
tmy3_metadata
The metadata has a 'TZ'
key with a value of -9.0
. This is the UTC offset in hours in which the data has been recorded. The :py~pvlib.iotools.read_tmy3
function read the data in the file, created a :py~pandas.DataFrame
with that data, and then localized the DataFrame's index to have this fixed offset. Here, we print just a few of the rows and columns of the large dataframe.
python
tmy3_data.index.tz
tmy3_data.loc[tmy3_data.index[0:3], ['ghi', 'dni', 'AOD (unitless)']]
The :py~pvlib.iotools.read_tmy2
function also returns a DataFrame with a localized DatetimeIndex.
The correct solar position can be immediately calculated from the DataFrame's index since the index has been localized.
python
- solar_position = pvlib.solarposition.get_solarposition(tmy3_data.index,
tmy3_metadata['latitude'], tmy3_metadata['longitude'])
ax = solar_position.loc[solar_position.index[0:24], ['apparent_zenith', 'apparent_elevation', 'azimuth']].plot()
ax.legend(loc=1); ax.axhline(0, color='darkgray'); # add 0 deg line for sunrise/sunset ax.axhline(180, color='darkgray'); # add 180 deg line for azimuth at solar noon ax.set_ylim(-60, 200); # zoom in, but cuts off full azimuth range ax.set_xlabel('Local time ({})'.format(solar_position.index.tz)); @savefig solar-position.png width=6in ax.set_ylabel('(degrees)');
According to the US Navy, on January 1, 2024 at Sand Point, Alaska (55.34N, -160.5W), sunrise was at 10:09 am, solar noon was at 1:46 pm, and sunset was at 5:22 pm. This is consistent with the data plotted above (and depressing).
What if we had a DatetimeIndex that was not localized, such as the one below? The solar position calculator will assume UTC time.
python
index = pd.date_range(start='1997-01-01 01:00', freq='1h', periods=24) index
- solar_position_notz = pvlib.solarposition.get_solarposition(index,
tmy3_metadata['latitude'], tmy3_metadata['longitude'])
ax = solar_position_notz.loc[solar_position_notz.index[0:24], ['apparent_zenith', 'apparent_elevation', 'azimuth']].plot()
ax.legend(loc=1); ax.axhline(0, color='darkgray'); # add 0 deg line for sunrise/sunset ax.axhline(180, color='darkgray'); # add 180 deg line for azimuth at solar noon ax.set_ylim(-60, 200); # zoom in, but cuts off full azimuth range ax.set_xlabel('Time (UTC)'); @savefig solar-position-nolocal.png width=6in ax.set_ylabel('(degrees)');
This looks like the plot above, but shifted by 9 hours.
In principle, one could localize the tz-naive solar position data to UTC, and then convert it to the desired time zone.
python
fixed_tz = pytz.FixedOffset(tmy3_metadata['TZ'] * 60) solar_position_hack = solar_position_notz.tz_localize('UTC').tz_convert(fixed_tz)
solar_position_hack.index
ax = solar_position_hack.loc[solar_position_hack.index[0:24], ['apparent_zenith', 'apparent_elevation', 'azimuth']].plot()
ax.legend(loc=1); ax.axhline(0, color='darkgray'); # add 0 deg line for sunrise/sunset ax.axhline(180, color='darkgray'); # add 180 deg line for azimuth at solar noon ax.set_ylim(-60, 200); # zoom in, but cuts off full azimuth range ax.set_xlabel('Local time ({})'.format(solar_position_hack.index.tz)); @savefig solar-position-hack.png width=6in ax.set_ylabel('(degrees)');
Note that the time has been correctly localized and converted, however, the calculation bounds still correspond to the original assumed-UTC range.
For this and other reasons, we recommend that users supply time zone information at the beginning of a calculation rather than localizing and converting the results at the end of a calculation.