Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

datetime to astropy Time: examples #2679

Merged
merged 7 commits into from Jul 14, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Expand Up @@ -209,3 +209,5 @@ docs/generated
docs/api/
# This is incase you run the figure tests
figure_test_images*

tags
2 changes: 1 addition & 1 deletion docs/code_ref/time.rst
Expand Up @@ -2,7 +2,7 @@ SunPy time
==========

The time submodule contains helpers for converting strings to Python
`datetime.datetime` objects and handling common operations on these objects. As
`astropy.time.Time` objects and handling common operations on these objects. As
well as this a `~sunpy.time.TimeRange` object is provided for representing a
period of time and performing operations on that range.

Expand Down
2 changes: 1 addition & 1 deletion docs/dev_guide/documentation.rst
Expand Up @@ -224,7 +224,7 @@ Example (:class:`sunpy.map.Map`) ::
----------
header : dict
A dictionary representation of the image header
date : datetime
date : `astropy.time.Time`
Image observation time
det : str
Detector name
Expand Down
2 changes: 1 addition & 1 deletion docs/guide/acquiring_data/hek.rst
Expand Up @@ -336,7 +336,7 @@ HEK results and create the corresponding VSO query attributes.

>>> vso_query = hek2vso.translate_results_to_query(result[10:11]) # doctest: +REMOTE_DATA
>>> vso_query[0] # doctest: +REMOTE_DATA
[<Time(datetime.datetime(2011, 8, 9, 7, 22, 44), datetime.datetime(2011, 8, 9, 7, 28, 56), None)>, <Source('SDO')>, <Instrument('AIA')>, <Wavelength(193.0, 193.0, 'Angstrom')>]
[<Time(<Time object: scale='utc' format='isot' value=2011-08-09T07:22:44.000>, <Time object: scale='utc' format='isot' value=2011-08-09T07:28:56.000>, None)>, <Source('SDO')>, <Instrument('AIA')>, <Wavelength(193.0, 193.0, 'Angstrom')>]

This function allows users finer-grained control of VSO queries
generated from HEK results.
Expand Down
34 changes: 19 additions & 15 deletions docs/guide/time.rst
Expand Up @@ -16,29 +16,27 @@ SunPy builds upon its functionality.

Solar data is associated with a number of different time formats. SunPy provides a simple
parsing function which can deal with most every format that a user may encounter. Called
`sunpy.time.parse_time()`, this function takes a string as input and returns a datetime object.
`sunpy.time.parse_time()`, this function takes a string as input and returns a `~astropy.time.Time` object.
Here are few examples of formats which `sunpy.time.parse_time()` accepts::

>>> from sunpy.time import parse_time
>>> parse_time('2007-05-04T21:08:12')
datetime.datetime(2007, 5, 4, 21, 8, 12)
<Time object: scale='utc' format='isot' value=2007-05-04T21:08:12.000>
>>> parse_time('2007/05/04T21:08:12')
datetime.datetime(2007, 5, 4, 21, 8, 12)
<Time object: scale='utc' format='isot' value=2007-05-04T21:08:12.000>
>>> parse_time('20070504T210812')
datetime.datetime(2007, 5, 4, 21, 8, 12)
<Time object: scale='utc' format='isot' value=2007-05-04T21:08:12.000>
>>> parse_time('2007-May-04 21:08:12')
datetime.datetime(2007, 5, 4, 21, 8, 12)
<Time object: scale='utc' format='isot' value=2007-05-04T21:08:12.000>
>>> parse_time('20070504_210812')
datetime.datetime(2007, 5, 4, 21, 8, 12)
<Time object: scale='utc' format='isot' value=2007-05-04T21:08:12.000>

Each of the above returns the same datetime object ``datetime.datetime(2007,
5, 4, 21, 8, 12)``. One of the most standard time formats used in solar
physics is the number of seconds since 1979 January 01. The parse_time
Each of the above returns the same `~astropy.time.Time` object ``<Time object: scale='utc' format='isot' value=2007-05-04T21:08:12.000>``. One of the most standard time formats used in solar
physics is the number of seconds since 1979 January 01 called ``utime``. The parse_time
function also accepts this as input, e.g.::

>>> parse_time(894316092.00000000)
datetime.datetime(2007, 5, 4, 21, 8, 12)

>>> parse_time(894316092.00000000, format='utime')
<Time object: scale='utc' format='utime' value=894316092.0>

All SunPy functions which require
time as an input sanitize the input using parse_time.
Expand All @@ -60,13 +58,19 @@ You can also pass the start and end times as a tuple: ::

This object makes use of parse_time() so it can accept a wide variety of time formats.
A time range object can also be created by providing a start time and a duration.
The duration must be provided as a `datetime.timedelta` object or
time-equivalent `astropy.units.Quantity`
The duration must be provided as a `~astropy.time.TimeDelta` or
time-equivalent `astropy.units.Quantity` or `datetime.timedelta` object
example: ::

>>> import astropy.units as u
>>> time_range = TimeRange('2010/03/04 00:10', 400 * u.second)

or: ::

>>> import astropy.units as u
>>> from astropy.time import TimeDelta
>>> time_range = TimeRange('2010/03/04 00:10', TimeDelta(400 * u.second))

or: ::

>>> from datetime import timedelta
Expand All @@ -77,7 +81,7 @@ get the time at the center of your interval or the length of your interval in mi
or days or seconds: ::

>>> time_range.center
datetime.datetime(2010, 3, 4, 0, 13, 20)
<Time object: scale='utc' format='isot' value=2010-03-04T00:13:20.000>
>>> time_range.minutes
<Quantity 6.66666667 min>
>>> time_range.days
Expand Down
6 changes: 3 additions & 3 deletions docs/guide/tour.rst
Expand Up @@ -264,9 +264,9 @@ and times. Here is a short example: ::

# parsing a standard time strings
>>> sunpy.time.parse_time('2004/02/05 12:00')
datetime.datetime(2004, 2, 5, 12, 0)
<Time object: scale='utc' format='isot' value=2004-02-05T12:00:00.000>

# This returns a datetime object. All SunPy functions which require
# This returns a astropy.time.Time object. All SunPy functions which require
# time as an input sanitize the input using parse_time.
>>> sunpy.time.day_of_year('2004-Jul-05 12:00:02')
187.50002314814816
Expand All @@ -278,7 +278,7 @@ and times. Here is a short example: ::
# TimeRange objects are useful for representing ranges of time
>>> time_range = sunpy.time.TimeRange('2010/03/04 00:10', '2010/03/04 00:20')
>>> time_range.center
datetime.datetime(2010, 3, 4, 0, 15)
<Time object: scale='utc' format='isot' value=2010-03-04T00:15:00.000>

For more information about working with time in SunPy checkout the :doc:`time guide <time>`.

Expand Down
2 changes: 1 addition & 1 deletion examples/plotting/finegrained_plot.py
Expand Up @@ -27,7 +27,7 @@
top_right = SkyCoord(800*u.arcsec, 700*u.arcsec, frame=aiamap.coordinate_frame)
aiamap_sub = aiamap.submap(bottom_left, top_right)

title_obsdate = '{:%Y-%b-%d %H:%M:%S}'.format(aiamap_sub.date)
title_obsdate = aiamap_sub.date.strftime('%Y-%b-%d %H:%M:%S')

###############################################################################
# The SunPy map peek method shows a helioprojective grid by default.
Expand Down
6 changes: 3 additions & 3 deletions examples/plotting/overplot_hek_polygon.py
Expand Up @@ -11,13 +11,13 @@

from __future__ import print_function, division

from datetime import timedelta
import numpy as np

import matplotlib.pyplot as plt

import astropy.units as u
from astropy.coordinates import SkyCoord
from astropy.time import TimeDelta

import sunpy.map
import sunpy.data.sample
Expand All @@ -40,8 +40,8 @@
##############################################################################
# Look for coronal holes detected using the SPoCA feature recognition method:

start_time = aia_map.date - timedelta(hours=2)
end_time = aia_map.date + timedelta(hours=2)
start_time = aia_map.date - TimeDelta(2*u.hour)
end_time = aia_map.date + TimeDelta(2*u.hour)
responses = hek_client.search(hek.attrs.Time(start_time, end_time),
hek.attrs.CH, hek.attrs.FRM.Name == 'SPoCA')

Expand Down
7 changes: 3 additions & 4 deletions examples/plotting/simple_differential_rotation.py
Expand Up @@ -18,13 +18,12 @@

from __future__ import print_function, division

from datetime import timedelta

import numpy as np
import matplotlib.pyplot as plt

import astropy.units as u
from astropy.coordinates import SkyCoord
from astropy.time import TimeDelta

import sunpy.map
import sunpy.data.sample
Expand Down Expand Up @@ -61,7 +60,7 @@
##############################################################################
# Let's define how many days in the future we want to rotate to

dt = timedelta(days=4)
dt = TimeDelta(4*u.day)
future_date = aia_map.date + dt

##############################################################################
Expand All @@ -70,7 +69,7 @@
fig = plt.figure()
ax = plt.subplot(projection=aia_map)
aia_map.plot()
ax.set_title('The effect of {0} days of differential rotation'.format(dt.days))
ax.set_title('The effect of {0} days of differential rotation'.format(dt.to(u.day).value))
aia_map.draw_grid()

for this_hpc_x, this_hpc_y in zip(hpc_x, hpc_y):
Expand Down
5 changes: 2 additions & 3 deletions examples/plotting/skip_magnetogram_active_regions.py
Expand Up @@ -10,13 +10,12 @@
# Start by importing the necessary modules.
from __future__ import print_function, division

import datetime

import numpy as np
import matplotlib.pyplot as plt

from astropy import units as u
from astropy.coordinates import SkyCoord
from astropy.time import TimeDelta

import sunpy.map
import sunpy.coordinates
Expand All @@ -33,7 +32,7 @@
# We will select the entire day as our timerange.

start_time = day
end_time = day + datetime.timedelta(hours=23, minutes=59, seconds=59)
end_time = day + TimeDelta(23*u.hour + 59*u.minute + 59*u.second)

##############################################################################
# Send the search query.
Expand Down
6 changes: 3 additions & 3 deletions examples/time_series/goes_hek_m25.py
Expand Up @@ -38,9 +38,9 @@
# Finally lets plot everything together

goes.peek()
plt.axvline(parse_time(flares_hek[0].get('event_peaktime')))
plt.axvspan(parse_time(flares_hek[0].get('event_starttime')),
parse_time(flares_hek[0].get('event_endtime')),
plt.axvline(parse_time(flares_hek[0].get('event_peaktime')).plot_date)
plt.axvspan(parse_time(flares_hek[0].get('event_starttime')).plot_date,
parse_time(flares_hek[0].get('event_endtime')).plot_date,
alpha=0.2, label=flares_hek[0].get('fl_goescls'))
plt.legend(loc=2)
plt.show()
2 changes: 1 addition & 1 deletion examples/time_series/skip_timeseries_example.py
Expand Up @@ -99,7 +99,7 @@
# You can access a specific value within the TimeSeries data DataFrame using
# all the normal Pandas methods.
# For example, the row with the index of 2015-01-01 00:02:00.008000:
ts_lyra.data.loc[parse_time('2011-06-07 00:02:00.010')]
ts_lyra.data.loc[parse_time('2011-06-07 00:02:00.010').datetime]
# Pandas will actually parse a string to a datetime automatically if it can:
ts_lyra.data.loc['2011-06-07 00:02:00.010']
# Pandas includes methods to find the indexes of the max/min values in a dataframe:
Expand Down
8 changes: 8 additions & 0 deletions sunpy/instr/tests/test_fermi.py
Expand Up @@ -2,8 +2,11 @@
from numpy.testing import assert_almost_equal
from sunpy.instr import fermi
from sunpy.time import parse_time
from sunpy.time.astropy_time import _is_time_equal
from sunpy.extern import six

import astropy.units as u


@pytest.mark.remote_data
def test_download_weekly_pointing_file():
Expand Down Expand Up @@ -50,3 +53,8 @@ def test_detector_angles():
assert_almost_equal(det2['n7'].value, 127.35783, decimal=1)
assert_almost_equal(det2['n8'].value, 122.98894, decimal=1)
assert_almost_equal(det2['n9'].value, 126.95987, decimal=1)


def test_met_to_utc():
time = fermi.met_to_utc(500000000)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aringlis @wafels We discovered by doing this that the result when converting Fermi MET to UTC is 2s different when using astropy Time than the previous datetime implementation. We talked through this and we think that it's because the previous implementation was not taking into account the two UTC leap seconds since the start of Fermi MET. Do either of you have any ideas if we could validate this somehow?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Cadair hmm, interesting. A quick test is trying some different input time values, to see if the same 2s discrepancy is always there, e.g. try timestamps before and after the leap second.

I can also look into what the Fermi team itself does with their MET values.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See Issue #2710

assert (time - parse_time('2016-11-05T00:53:16.000')) < 1e-7 * u.s
2 changes: 1 addition & 1 deletion sunpy/net/jsoc/attrs.py
Expand Up @@ -8,7 +8,7 @@
from sunpy.net.vso.attrs import Wavelength


__all__ = ['Series', 'Time', 'Protocol', 'Notify', 'Segment', 'Keys', 'PrimeKey']
__all__ = ['Series', 'Protocol', 'Notify', 'Segment', 'Keys', 'PrimeKey']

from sunpy.net.vso.attrs import Time

Expand Down
4 changes: 2 additions & 2 deletions sunpy/tests/figure_hashes_py36.json
Expand Up @@ -19,11 +19,11 @@
"sunpy.map.tests.test_plotting.test_plot_masked_aia171_superpixel_nowcsaxes": "a34f0342097bd3967aefdaaea335ff084df2fbe6466330312abc8398538f7b1a",
"sunpy.map.tests.test_plotting.test_rectangle_aia171": "3c2e977ccf7d07ca1b56e216ea06fc634259fa425526f643eaaab685d79a0afe",
"sunpy.timeseries.tests.test_timeseriesbase.test_eve_peek": "c483d82bacee7217e4d18f6a8433457fcc698746fae71065208223c2d5e5eb6a",
"sunpy.timeseries.tests.test_timeseriesbase.test_fermi_gbm_peek": "213ea4e7e86de1951c357fa3e149c4c76f95ac7fe5e1ba035d78cbd5122b7947",
"sunpy.timeseries.tests.test_timeseriesbase.test_fermi_gbm_peek": "56da147444455d1466e0073656aff59b901c670b931e94dd878613e7b2bb55fb",
"sunpy.timeseries.tests.test_timeseriesbase.test_generic_ts_peek": "ca11e496d4dcb157841a76513543694af61a7562181f61250619c1bfa9e5b70c",
"sunpy.timeseries.tests.test_timeseriesbase.test_lyra_peek": "7cc6b761b6f015e5c2470f561de2ef029592dbd77ed97203a2f9895c6b140bc4",
"sunpy.timeseries.tests.test_timeseriesbase.test_noaa_ind_peek": "10f1fc9e29f41f07ba63b119170e2ba649fa6440b0e23ffd32baee59b5df9aed",
"sunpy.timeseries.tests.test_timeseriesbase.test_noaa_pre_peek": "09c2fb9890be936cdd414309c456e624c39183f594c663d4956fa9c20cf7a289",
"sunpy.timeseries.tests.test_timeseriesbase.test_norh_peek": "1e03ed25af3b6534a50c849e3b37c7307f8903009966744440d77cfc231c3855",
"sunpy.timeseries.tests.test_timeseriesbase.test_rhessi_peek": "0efa07c81fcf5ef99f9b09be131dac2bf2ae5509e4c123b72930f09562ed4b7c"
}
}
4 changes: 2 additions & 2 deletions sunpy/timeseries/tests/test_timeseriesbase.py
Expand Up @@ -109,11 +109,11 @@ def noaa_pre_test_ts():
def generic_ts():
# Generate the data and the corrisponding dates
base = parse_time("2016/10/01T05:00:00")
dates = [base - TimeDelta(x*u.minute) for x in range(0, 24 * 60)]
dates = parse_time([base - TimeDelta(x*u.minute) for x in range(0, 24 * 60)])
intensity = np.sin(np.arange(0, 12 * np.pi, ((12 * np.pi) / (24 * 60))))

# Create the data DataFrame, header MetaDict and units OrderedDict
data = DataFrame(intensity, index=dates, columns=['intensity'])
data = DataFrame(intensity, index=dates.isot.astype('datetime64'), columns=['intensity'])
units = OrderedDict([('intensity', u.W / u.m**2)])
meta = MetaDict({'key': 'value'})

Expand Down