Instructions on How to Generate ITS_LIVE Datacubes and Corresponding Composites

This page provides instructions on how to generate ITS_LIVE datacubes and corresponding composites.

Datacubes

All instructions rely on the datacube definition file as generated by the define_cube_polygons.py script. The definition file used for the v02 production is datacubes_definition.json (stored in the archive file).

All AWS related scripts are located in src/aws subdirectory of the code base. All helper utilities are located in src/tools subdirectory of the code base.

Please make sure to generate temporary access tokens to the AWS by running JPL Access Key Generation script to have an access to the kh9 AWS account for the project before running any AWS related (run_*.py) scripts.

You can generate datacubes for all regions at once or by providing a set of datacubes to generate.

All Regions

To generate datacubes globally, please use the following command:

python aws/run_batch.py -c tools/data/datacubes_definition.json -o v02_cubes.json

Specific Regions

To generate datacubes for a specific region, you have to provide one of the following options:

• Region definition as geojson file using --processCubesWithinPolygon command-line argument
• A JSON list of datacubes of interest through --processCubes command-line argument
• A file that lists datacubes of interest through --processCubesFile command-line argument

Alaska

To generate datacubes for Alaska using region polygon as defined in regions/Alaska.geojson in YAML format and EPSG code of 3413:

Command to run:

python aws/run_batch.py -c tools/data/datacubes_definition.json -o v02_ALA_cubes.json --processCubesWithinPolygon 'regions/Alaska.geojson' --epsgCode '[3413]'

• Outputs datacube filenames as submitted to the AWS Batch processing in filenames_v02_ALA_cubes.json (appends filenames_ to the v02_ALA_cubes.json output file)
• Stores output from the script and verbose information about each AWS Batch job submitted in v02_ALA_cubes.json

HMA

At first, extract datacubes that correspond to the RGI codes of 13, 14, 15 from the datacube catalog using extract_region_cubes.py script

• NOTE: Alex updated datacube catalog with RGI code and region notation metadata as these are not part of the tools/data/datacubes_definition.json catalog

python tools/extract_region_cubes.py -c aws/regions/catalog_v02_rgi.geojson --rgi_code '[13,14,15] -o HMA_datacubes_RGI_13_14_15.json

Command to run:

python aws/run_batch.py -c tools/data/datacubes_definition.json -o v02_HMA_cubes.json --processCubesFile HMA_datacubes_RGI_13_14_15.json

• Outputs datacube filenames as submitted to the AWS Batch processing in filenames_v02_HMA_cubes.json (appends filenames_ to the v02_HMA_cubes.json output file)
• Stores output from the script and verbose information about each AWS Batch job submitted in v02_HMA_cubes.json