Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving HEK Representation #7619

Draft
wants to merge 99 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
0abd5d0
Integrate Astropy units into the HEK Results table
aritrasinha108 Jun 10, 2023
d29163c
Resolve comments and sugesstions to separate JSON files
aritrasinha108 Jun 18, 2023
9100337
Add support for chaincodes
aritrasinha108 Jun 30, 2023
ff82a15
Add function to parse chaincodes into SkyCoord objects
aritrasinha108 Jul 7, 2023
b8e8d4b
Resolve suggestions
aritrasinha108 Jul 7, 2023
9a8d56c
Add regions library to setup.cfg
aritrasinha108 Jul 7, 2023
78191aa
Resolve suggestion
aritrasinha108 Jul 8, 2023
b50862d
Add tests and fix bugs
aritrasinha108 Jul 8, 2023
4a634a9
fix conflicts
aritrasinha108 Jul 9, 2023
61b81d0
pre-commit fix
ahmedhosssam May 11, 2024
fd3676c
create util.py
ahmedhosssam May 13, 2024
fc18875
fix hek util errors
ahmedhosssam May 13, 2024
854ad7c
add notes
ahmedhosssam May 14, 2024
d69f7f2
removed superseded test
ahmedhosssam May 14, 2024
e230d5a
create util.py
ahmedhosssam May 17, 2024
98a9390
pre-commit
ahmedhosssam May 17, 2024
ac0dede
add regions to pyproject.toml
ahmedhosssam May 17, 2024
6e4a5e6
remove unit_mapping
ahmedhosssam May 18, 2024
45c4cc6
remove parse_astropy_unit
ahmedhosssam May 18, 2024
cbbf1a7
import util functions
ahmedhosssam May 20, 2024
818f0df
use alias inside context manager
ahmedhosssam May 21, 2024
ab783c5
change name of util.py to utils.py
ahmedhosssam May 21, 2024
fe047af
reorder utils.py functions
ahmedhosssam May 21, 2024
3e00040
fix import error
ahmedhosssam May 21, 2024
6a26719
get_unit documentation string
ahmedhosssam May 21, 2024
2238d82
fix merge conflict
ahmedhosssam May 22, 2024
2815f68
remove is_coord_prop from get_unit
ahmedhosssam May 22, 2024
d0f5d98
remove is_coord_prop from get_unit calls
ahmedhosssam May 22, 2024
672da3d
get_unit docstring
ahmedhosssam May 22, 2024
bca74e7
remove debugging statements
ahmedhosssam May 22, 2024
38aed04
modify doc string
ahmedhosssam May 23, 2024
011be90
modify get_unit interface
ahmedhosssam May 23, 2024
5d6b433
remove unused parameter & add new variable
ahmedhosssam May 23, 2024
2c90ccc
remove unit_prop from get_unit
ahmedhosssam May 23, 2024
37fdb88
add note
ahmedhosssam May 23, 2024
93e7872
modify unit aliases
ahmedhosssam May 23, 2024
8a34afe
remove debugging statements
ahmedhosssam May 23, 2024
f400b14
parse_chaincode doc string
ahmedhosssam May 23, 2024
4da3fee
remove parse_unit function
ahmedhosssam May 30, 2024
574411f
refactor parse_columns_to_table
ahmedhosssam May 30, 2024
ef04cdd
parse_columns_to_table doc string
ahmedhosssam May 30, 2024
2f35ca3
modify get_unit comments
ahmedhosssam May 31, 2024
c58cf21
add __all__ to utils.py
ahmedhosssam May 31, 2024
7b1729b
add paths to __all__
ahmedhosssam Jun 14, 2024
082d7a0
modify freeze() doc string
ahmedhosssam Jun 14, 2024
9cf7ca5
parse_values_to_quantities doc string
ahmedhosssam Jun 14, 2024
cc60811
parse_times doc string
ahmedhosssam Jun 14, 2024
b2f5fa2
ignore UnitsWarning in get_unit
ahmedhosssam Jun 15, 2024
2bbd530
modify test_missing_times
ahmedhosssam Jun 15, 2024
ea21c0d
changed hek results
ahmedhosssam Jun 15, 2024
6c3712e
Merge branch 'main' into gsoc24-hek
ahmedhosssam Jun 15, 2024
155af43
ruff fixes
ahmedhosssam Jun 15, 2024
732d5a8
add erg_per_cm3 to aliases
ahmedhosssam Jun 16, 2024
2f78964
test_utils init && get_unit test
ahmedhosssam Jun 16, 2024
6f4f715
remove NOTE comment
ahmedhosssam Jun 17, 2024
b654b26
test parse_chaincode
ahmedhosssam Jun 17, 2024
9bb9caa
remote NOTE comment
ahmedhosssam Jun 17, 2024
db79d94
fix style
ahmedhosssam Jun 20, 2024
2553c95
use interlink in doc string
ahmedhosssam Jun 24, 2024
198c484
move tests to test_utils
ahmedhosssam Jun 24, 2024
5406eae
move tests to test_hek
ahmedhosssam Jun 25, 2024
dda55c0
merge event coords columns
ahmedhosssam Jun 30, 2024
0a03902
fix gibberish column values
ahmedhosssam Jun 30, 2024
ce091b2
ignore coord3 if None
ahmedhosssam Jun 30, 2024
5e3c7ae
convert obs_meanwavel to angstrom
ahmedhosssam Jul 1, 2024
8cd5875
support remaining coords
ahmedhosssam Jul 1, 2024
2fdb058
add PointSkyRegion to tests
ahmedhosssam Jul 2, 2024
cbc66aa
modify tests
ahmedhosssam Jul 2, 2024
d51e3a3
results from hek changed, again
ahmedhosssam Jul 2, 2024
d2d12a9
support remaining coords
ahmedhosssam Jul 2, 2024
dff59e4
test parse chaincode points
ahmedhosssam Jul 2, 2024
f954334
test merging event_coords
ahmedhosssam Jul 2, 2024
81d4879
move test to test_hek
ahmedhosssam Jul 4, 2024
44f9336
add obstime and observer to parse_chaincode
ahmedhosssam Jul 5, 2024
429c837
update the overplot_hek_polygon example
ahmedhosssam Jul 5, 2024
bf0eacc
pre-commit
ahmedhosssam Jul 5, 2024
b28fe2f
add comment
ahmedhosssam Jul 6, 2024
de2c666
make the unit appear in the output
ahmedhosssam Jul 6, 2024
ff16567
test_obs_meanwavel
ahmedhosssam Jul 6, 2024
8838694
remove note comments
ahmedhosssam Jul 6, 2024
81c94ef
fix times datatype
ahmedhosssam Jul 8, 2024
8907336
fix hek2vso tests
ahmedhosssam Jul 8, 2024
3351243
pre-commit fix
ahmedhosssam Jul 8, 2024
eb2ffc8
Merge branch 'main' into gsoc24-hek
ahmedhosssam Jul 10, 2024
e9d5394
test_parse_times
ahmedhosssam Jul 11, 2024
750090c
update hek.rst
ahmedhosssam Jul 16, 2024
c38af73
fix hek.rst
ahmedhosssam Jul 16, 2024
95f24f7
add notes and references sections
ahmedhosssam Jul 18, 2024
ab49202
remove line
ahmedhosssam Jul 18, 2024
05c7da6
add FL tests
ahmedhosssam Jul 18, 2024
ab8ef29
test_flares_pyhton_logical_ops
ahmedhosssam Jul 18, 2024
0b62d26
add obstime to event_coord
ahmedhosssam Jul 18, 2024
20b480c
fix typo
ahmedhosssam Jul 18, 2024
3d2e7a9
Merge branch 'main' into gsoc24-hek
ahmedhosssam Jul 18, 2024
364db09
add changelog
ahmedhosssam Jul 18, 2024
e1a8617
fix test_merging_event_coords
ahmedhosssam Jul 18, 2024
2cac890
add changelog for event_coord
ahmedhosssam Jul 19, 2024
37a1bf3
HEKClient docstring
ahmedhosssam Jul 19, 2024
510a6f6
Update sunpy/net/hek/utils.py
ahmedhosssam Jul 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog/7619.breaking.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
In `~sunpy.net.hek` the columns ``event_Coord1`` ``event_Coord2`` and ``event_Coord3`` are megred into one `astropy.coordinates.SkyCoord` object called ``event_coord``.
1 change: 1 addition & 0 deletions changelog/7619.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
`~sunpy.net.hek` now fully utilizes Astropy units and Sunpy coordinates to express feature/event properties.
111 changes: 56 additions & 55 deletions docs/tutorial/acquiring_data/hek.rst
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ The first thing we need to do is access the results from the HEK client, the onl

.. code-block:: python

>>> len(result['hek']) # doctest: +REMOTE_DATA
>>> len(result["hek"]) # doctest: +REMOTE_DATA
19

This object is an `astropy.table.Table` object with the columns which correspond to the parameters listed at http://www.lmsal.com/hek/VOEvent_Spec.html.
Expand Down Expand Up @@ -234,43 +234,43 @@ and as a check:

>>> result["hek"]["fl_peakflux"] # doctest: +REMOTE_DATA
<QueryResponseColumn name='fl_peakflux' dtype='object' length=17>
None
None
None
None
None
None
None
2326.86
1698.83
None
None
2360.49
3242.64
1375.93
6275.98
923.984
1019.83

>>> result["hek"]["event_coord1"] # doctest: +REMOTE_DATA
<QueryResponseColumn name='event_coord1' dtype='float64' length=17>
51.0
51.0
51.0
924.0
924.0
924.0
69.0
883.2
883.2
69.0
69.0
883.2
883.2
883.2
883.2
883.2
883.2
None
None
None
None
None
None
None
2326.86 DN / (pix s)
1698.83 DN / (pix s)
None
None
2360.49 DN / (pix s)
3242.64 DN / (pix s)
1375.93 DN / (pix s)
6275.98 DN / (pix s)
923.984 DN / (pix s)
1019.83 DN / (pix s)

>>> result["hek"]["event_coord"] # doctest: +REMOTE_DATA
<QueryResponseColumn name='event_coord' dtype='object' length=17>
<SkyCoord (Helioprojective: obstime=2011-08-08 01:30:04.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (51., 151.)>
<SkyCoord (Helioprojective: obstime=2011-08-08 01:30:04.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (51., 151.)>
<SkyCoord (Helioprojective: obstime=2011-08-08 01:30:04.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (51., 151.)>
<SkyCoord (Helioprojective: obstime=2011-08-09 02:30:04.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (924., 217.)>
<SkyCoord (Helioprojective: obstime=2011-08-09 02:30:04.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (924., 217.)>
<SkyCoord (Helioprojective: obstime=2011-08-09 02:30:04.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (924., 217.)>
<SkyCoord (ICRS): (ra, dec) in deg\n (69., 15.)>
<SkyCoord (Helioprojective: obstime=2011-08-09 07:22:38.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 268.8)>
<SkyCoord (Helioprojective: obstime=2011-08-09 07:22:44.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 268.8)>
<SkyCoord (ICRS): (ra, dec) in deg\n (69., 17.)>
<SkyCoord (ICRS): (ra, dec) in deg\n (69., 14.)>
<SkyCoord (Helioprojective: obstime=2011-08-09 07:55:59.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 268.8)>
<SkyCoord (Helioprojective: obstime=2011-08-09 07:59:49.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 192.)>
<SkyCoord (Helioprojective: obstime=2011-08-09 08:00:03.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 268.8)>
<SkyCoord (Helioprojective: obstime=2011-08-09 08:00:20.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 268.8)>
<SkyCoord (Helioprojective: obstime=2011-08-09 08:00:53.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 268.8)>
<SkyCoord (Helioprojective: obstime=2011-08-09 08:01:21.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 192.)>

Note that some of the fluxes are returned as "None".
This is because some feature recognition methods for flares do not report the peak flux.
Expand All @@ -283,23 +283,24 @@ Let's say we want all the flares west of 50 arcseconds AND have a peak flux over
>>> result = Fido.search(a.Time(tstart,tend), a.hek.EventType(event_type), (a.hek.Event.Coord1 > 50) and (a.hek.FL.PeakFlux > 1000)) # doctest: +REMOTE_DATA

>>> result["hek"]["fl_peakflux"] # doctest: +REMOTE_DATA
<QueryResponseColumn name='fl_peakflux' dtype='float64' length=7>
2326.86
1698.83
2360.49
3242.64
1375.93
6275.98
1019.83
>>> result["hek"]["event_coord1"] # doctest: +REMOTE_DATA
<QueryResponseColumn name='event_coord1' dtype='float64' length=7>
883.2
883.2
883.2
883.2
883.2
883.2
883.2
<QueryResponseColumn name='fl_peakflux' dtype='object' length=7>
2326.86 DN / (pix s)
1698.83 DN / (pix s)
2360.49 DN / (pix s)
3242.64 DN / (pix s)
1375.93 DN / (pix s)
6275.98 DN / (pix s)
1019.83 DN / (pix s)

>>> result["hek"]["event_coord"] # doctest: +REMOTE_DATA
<QueryResponseColumn name='event_coord' dtype='object' length=7>
<SkyCoord (Helioprojective: obstime=2011-08-09 07:22:38.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 268.8)>
<SkyCoord (Helioprojective: obstime=2011-08-09 07:22:44.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 268.8)>
<SkyCoord (Helioprojective: obstime=2011-08-09 07:55:59.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 268.8)>
<SkyCoord (Helioprojective: obstime=2011-08-09 07:59:49.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 192.)>
<SkyCoord (Helioprojective: obstime=2011-08-09 08:00:03.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 268.8)>
<SkyCoord (Helioprojective: obstime=2011-08-09 08:00:20.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 268.8)>
<SkyCoord (Helioprojective: obstime=2011-08-09 08:01:21.000, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec\n (883.2, 192.)>

In this case none of the peak fluxes are returned with the value `None`.
Since we are using an ``and`` logical operator we need a result from the ``(a.hek.FL.PeakFlux > 1000)`` filter.
Expand Down
12 changes: 1 addition & 11 deletions examples/plotting/overplot_hek_polygon.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,13 @@
import numpy as np

import astropy.units as u
from astropy.coordinates import SkyCoord
from astropy.time import TimeDelta

import sunpy.data.sample
import sunpy.map
from sunpy.coordinates import frames
from sunpy.net import attrs as a
from sunpy.net import hek
from sunpy.physics.differential_rotation import solar_rotate_coordinate
from sunpy.time import parse_time

###############################################################################
# We start with the sample data.
Expand Down Expand Up @@ -48,19 +45,12 @@
# Next let's get the boundary of the coronal hole.

ch = responses[response_index]
p1 = ch["hpc_boundcc"][9:-2]
p2 = p1.split(',')
p3 = [v.split(" ") for v in p2]
ch_date = parse_time(ch['event_starttime'])
ch_boundary = responses[response_index]["hpc_boundcc"].vertices

##############################################################################
# The coronal hole was detected at different time than the AIA image was
# taken so we need to rotate it to the map observation time.

ch_boundary = SkyCoord(
[(float(v[0]), float(v[1])) * u.arcsec for v in p3],
obstime=ch_date, observer="earth",
frame=frames.Helioprojective)
rotated_ch_boundary = solar_rotate_coordinate(ch_boundary, time=aia_map.date)

##############################################################################
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ net = [
"python-dateutil>=2.8.0",
"tqdm>=4.64.0",
"zeep>=4.1.0",
"regions>=0.8",
]
scikit-image = ["scikit-image>=0.19.0"]
spice = ["spiceypy>=5.0.0"]
Expand Down
Loading
Loading