Skip to content
Read & Plot THEMIS ASI 256x256 "high resolution" GBO ground-based imager data
Python MATLAB IDL
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github ci=> actions, black style Oct 2, 2019
Examples ci=> actions, black style Oct 2, 2019
archive
data
idl Rename sav2nc.pro to idl/sav2nc.pro Feb 8, 2016
matlab
tests note cdflib is now up to do Oct 2, 2019
themisasi ci=> actions, black style Oct 2, 2019
.codeclimate.yml
.coveragerc ci template, pytest parametrize Jan 20, 2019
.flake8
.gitattributes template Dec 1, 2017
.gitignore
DascThemisFOV.py ci=> actions, black style Oct 2, 2019
DascThemisSlice.py ci=> actions, black style Oct 2, 2019
DownloadThemis.py
LICENSE.txt update cdflib api Jun 17, 2019
PlotCameraResponse.py
PlotThemis.py
PlotThemisPixels.py
README.md
ThemisRadec.py
mypy.ini
pyproject.toml
pytest.ini
setup.cfg
setup.py ci=> actions, black style Oct 2, 2019

README.md

Zenodo DOI

Actions Status

pypi versions PyPi Download stats

THEMIS GBO ASI Reader

Read & plot 256x256 "high resolution" THEMIS ASI ground-based imager data from Python. THEMIS ASI data are collected with the original 2002 design, using Starlight-Xpress Lodestar MX716 cameras with monochrome Sony ICX249AL imaging chips. A subregion from full-size 752 x 582 pixels (512 x 512 pixels) are 2x2 binned to 256 x 256 pixels and retrieved over USB 1.1 for disk storage.

This package also reads the THEMIS ASI star registered plate scale, giving azimuth and elevation for each pixel.

Install

  1. clone this repository
git clone https://github.com/space-physics/themisasi
  1. install this program
    pip install -e themisasi

You can test the basic functionality by from the top cdflib directory:

pytest

Usage

One of the main ways analysts might use THEMIS-ASI data is by loading it into a 3-D array (time, x, y).

Single time

This example is where the ASI video files are in ~/data/themis, and the Gakona site is selected at the time shown.

import themisasi as ta

dat = ta.load('~/data/themis', site='gako', treq='2011-01-06T17:00:03')

This returns the camera image from Gakona camera closest to the requested time, and the 'az', 'el' calibration data, if available.

THEMIS-ASI output xarray.Dataset, which is used throughout geosciences and astronomy. Xarray may be thought of as a "smart" Numpy array, or a multidimensional Pandas array. A THEMIS image data stack is obtained by:

dat = ta.load(...)

imgs = dat['imgs']
  • dat.time contains the approximate time of each image (consider the finite exposure time).
  • dat.x and dat.y are simple pixel indices, perhaps not often needed.

Image + Azimuth, Elevation

Loading calibration data gives azimuth, elevation for each pixel and lat, lon of each camera.

import themisasi as ta

dat = ta.load('~/data/themis', site='gako', treq='2011-01-06T17:00:03')

If an appropriate calibration file exists, dat additionally contains 'az', 'el', 'lat', 'lon' and so on to allow using data for multi-camera analyses.

Coordinate conversion (optional)

If desired, convert azimuth/elevation to ra/dec using pymap3d.

pip install pymap3d

and then from within Python:

import pymap3d as pm

rasc, decl = pm.azel2radec(dat.az, dat.el, dat.lat, dat.lon, dat.time)

Download, Read and Plot THEMIS ASI Data

The data is downloaded concurrently using asyncio and aiohttp_requests. There is one concurrent worker launched per site, that downloads one time at a time concurrently across sites. Thus if downloading for one site, one time downloads at a time. If downloading for 5 sites, five files download at a time across requested times by site.

Get video data from Themis all-sky imager data repository. The plate scale data is also downloaded. The calibration files are named *asc*.cdf or *skymap*.sav.

Example: February 4, 2012, 8 UT Gakona

DownloadThemis 2012-02-04T08 gako ~/data

or via the API:

import themisasi as ta

ta.download('2012-02-04T08', 'gako', '~/data')

with the API, it is convenient to for-loop over many sites at the same time(s):

sites = ['fykn', 'gako']

for site in sites:
   ta.download('2012-02-04T08', site, '~/data')

With the calibration data, verify that the time range of the calibration data is appropriate for the time range of the image data. For example, calibration data from 1999 may not be valid for 2018 if the camera was ever moved in the enclosure during maintanence.https://github.com/dib-lab/khmer/pull/1430

You can optionally download from within Python:

import themisasi as ta

ta.download('2012-03-12T12', 'fykn', '~/data')

get times in a file

the convenience function themisasi.io.filetimes(filename) returns a list of Python datetime in a file

Video Playback / PNG conversion

This example plays the video content.

Use the -o option to dump the frames to individual PNGs for easier back-and-forth viewing. The calibration file (second filename) is optional.

PlotThemis ~/data/themis/thg_l1_asf_fykn_2013041408_v01.cdf

Plot time series of pixel(s)

Again, be sure the calibration file is appropriate for the time range of the video--the camera may have been moved / reoriented during maintenance.

The pixels can be specified by (azimuth, elevation) or (lat, lon, projection altitude [km])

Azimuth / Elevation:

PlotThemisPixels tests/thg_l1_ast_gako_20110505_v01.cdf -az 65 70 -el 48 68

Latitude, Longitude, Projection Altitude [kilometers]: Typically the brightest aurora is in the 100-110 km altitude range, so a common approximate is to assume "all" of the brightness comes from a single altitude in this region.

PlotThemisPixels tests/thg_l1_ast_gako_20110505_v01.cdf -lla 65 -145 100.

Notes

Themis site map (2009)

Themis site map

THEMIS GBO ASI spectral response:

Themis spectral response

Articles

These articles give vital descriptions of THEMIS GBO ASI.

Resources

Matlab

The Matlab code is obsolete, the Python version has so much more:

readTHEMIS('thg_l1_asf_fykn_2013041408_v01.cdf')

data corruption

I discovered that IDL 8.0 had a problem saving structured arrays of bytes. While current versions of IDL can read these corrupted .sav files, GDL 0.9.4 and SciPy 0.16.1 cannot. I submitted a patch to SciPy to allow reading these files, which was incorporated into SciPy 0.18.0.

As a fallback, read and rewrite the corrupted file with the IDL script in the idl directory.

You can’t perform that action at this time.