Architecture 3

The code for Architecture 3 is now up and running. As I was writing A3 I realized that the code was (modulo a wee touch of parser mods) the same as the A2 code. And I realized that the only difference was really in the inputs: In A2 I added a top level XML attribute, xml:base, to the dmr++ files that identified the location of the (in the A2 case) HDF5 file against which the range-gets would be made, as described by each h4:byteStream content. In A3 each h4:byteStream is associated with a different S3 resource, so I add an href element to every h4:byteStream that identifies either a locally held resource or an HTTP accessible resource holding the h4:byteStream's data content. It was easy to tweak my dmr++ modification scripts to apply the A2 use-case using the href element addition to the h4:byteStream elements - the difference being that in A2 all the href attributes point to the same resource and the offset and nBytes attributes are used to locate the specific content required. In A3 all of the href attributes point to (mostly) different resources, the offset value is set to 0, and nBytes remains the same (and in theory should be same as the size of the referenced object).

Why is the A2 use-case the "same" code as before? Because under the hood in the C++ code the H4ByteStream objects used by the implementation always had a data_url. In the code we originally ran for A2 the parser collected the top level xml:base attribute and propagated it into each instance of H4ByteStream. In the new code the parser collects each h4:byteStream/@href and propagates it into the associated instance of H4ByteStream. After that, All Of The Downstream Processing Is The Same!

whew

That was a bunch. Here's how it works. On the test server I have two sets of dmr++ files, one for A2 and one for A3. Also, this server is running the S3 Gateway code which means the it does A1 too.

Please find the associated Hyrax server:

Architecture 1: http://a3cft.opendap.org:8080/s3/cloudydap/ Architecture 2: http://a3cft.opendap.org:8080/opendap/arch2/ Architecture 3: http://a3cft.opendap.org:8080/opendap/arch3/