Arch #2 Design

Hyrax DMR++ Handler for Arch #2

This is the design for a Hyrax data format handler that reads data using the DMR++ metadata file for HDF5 data in the Amazon cloud.

Design Assumptions

• The BES can get the DMR++ files
• It's clear to the BES how to get the granule that matches a given DMR++
• The name and/or offset info in the DMR++ can be given to a HTTP Range GET directly.
• This handlers only needs to build a DMR and can generate the needed stuff for a DAP2 response one way or another.
• Simple code can parse the DMR++ to make a DMR in memory.

Implementation Steps

This is a pretty simple handler; it's substantially simpler than the FreeForm handler but uses the same design, so there's no real need to draw up a formal design. Each of the steps below will result in for which code that we can write integration/regression. The first three steps in this simple plan assume data files that are neighter compressed or chunked (although compressed is not a huge issue; chunking might be).

Build the DMR response

• Use the DMR++ and the BES DMR parser.
• Strip out the extra elements in the DMR++
• This can also make the DDS & DAS responses too (this might be easier to do in XSLT than in C++).

Make new BaseType specializations

• The new BaseTypes hold the offsets and sizes for variables

Extend the types read() methods to use the offset/size info

• We can implement this using both HTTP Range GET and the Unix kernel file I/O layer
• The Unix file I/O layer will speed testing

The next steps will add support for chunked and compressed data

• Steps TBD