🚧 This repo is a WIP but the general idea is to get Astropy running on AWS Lambda.
This repository is an example of a directory that can be zipped up and deployed to Lambda.
- https://github.com/vitolimandibhrata/aws-lambda-numpy
- https://stackoverflow.com/questions/34749806/using-moviepy-scipy-and-numpy-in-amazon-lambda
- https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example-deployment-pkg.html#with-s3-example-deployment-pkg-python
Once the dependencies are installed (e.g. numpy
and astropy
) you need to defined the function for Lambda to execute. In this case we have a file named process.py
and a handler
function. We configure Lambda to execute process.handler
Currently we have the FITS file location hard-coded into the processing code but this could be dynamically passed.
Yeah, pretty much. Here's what the output from Lambda looks like:
START RequestId: 3096ba02-0c3e-11e8-be1d-b39e9f31e05f Version: $LATEST
Downloading https://hla.stsci.edu/cgi-bin/ecfproxy?file_id=hag_j004524.96+403851.8_j8hpdcaoq_v01.drizzle.fits [Done]
Filename: /tmp/tmpt3EFwI
No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 PrimaryHDU 23 ()
1 SCI_F850LP 1 ImageHDU 30 (100, 100) float32
2 WHT_F850LP 1 ImageHDU 24 (100, 100) float32
None
END RequestId: 3096ba02-0c3e-11e8-be1d-b39e9f31e05f
REPORT RequestId: 3096ba02-0c3e-11e8-be1d-b39e9f31e05f Duration: 173.98 ms Billed Duration: 200 ms Memory Size: 512 MB Max Memory Used: 63 MB
The following code downloads a FITS file from the Hubble Legacy Archive and inspects the contents of the file using astropy.io.fits
import os
import subprocess
import uuid
libdir = os.path.join(os.getcwd(), 'lib')
FITS_LOCATION = "https://hla.stsci.edu/cgi-bin/ecfproxy?file_id=hag_j004524.96+403851.8_j8hpdcaoq_v01.drizzle.fits"
import warnings
from astropy.utils.data import CacheMissingWarning
warnings.simplefilter('ignore', CacheMissingWarning)
from astropy.io import fits
def do_science(fits_location):
hdul = fits.open(fits_location)
print(hdul.info())
def handler(event, context):
do_science(FITS_LOCATION)
if __name__ == "__main__":
handler('', '')
Use something like astroquery.mast
to query for a collection of files and call the Lambda function for each file (i.e. dynamically pass the location of the FITS file). Then do something clever and write the result to S3?