Skip to content

Use Case Grooming

Jump to bottom
Aleksandar Jelenak edited this page Jan 26, 2017 · 16 revisions

Why?

In order for us to use the same set of use cases for testing the 3 proposed cloudydap architectures we need to modify and/or drop many of the existing use cases in order to migrate the DAP2 requests to DAP4. I personally think that testing things like the ascii data response and the fileout-netcdf response are not typical for this work and that we should stick to just the DAP4 responses in order to further simplify our testing.

Tagging Requests

A tag needs to be added to the various requests sent to Hyrax so that the S3 requests can be associated with their concomitant UseCase runs. This can be done by adding a query parameter to every DAP4 URL:

cloudydap={UseCase}_{Arch}_STARTED_{seconds-since-epoch}.h5

where:

  • {UseCase} is the use case identifier, e.g. UC1 for the Use Case 1.
  • {Arch} is the architecture identifier, e.g. A1CFT for Architecture #1 CF=True Hyrax server.
  • seconds-since-epoch would be replaced with the output of a date +%s command (ex: 1485208202) which should be the same for every request in a particular run of a collection of the use cases.

And when a BES XML command is being utilized the tag must be added as a new context command:

<bes:setContext name="cloudydap">
    {UseCase}_{Arch}_STARTED_{seconds-since-epoch}.h5
</bes:setContext>

I'll use a concrete example for the rest of the examples: UC1_A1CFT_STARTED_1485208202.h5

TODO

Here's what I see that needs to be done:

  1. Edit all existing requests and convert them (if possible and without creating duplicates) from a DAP2 request to the related DAP4 request.

  2. For requests that utilize traditional Hyrax request URLs like this: http://a2cft.opendap.org:8080/s3/dap/cloudydap/airs/AIRS.2015.01.01.L3.RetStd_IR001.v6.0.11.0.G15013155825.nc.h5.dmrpp.dap?dap4.ce=Topography[0:20:179][0:20:359]

    The URL will need the new cloudy dap parameter: http://a2cft.opendap.org:8080/s3/dap/cloudydap/airs/AIRS.2015.01.01.L3.RetStd_IR001.v6.0.11.0.G15013155825.nc.h5.dmrpp.dap?dap4.ce=Topography[0:20:179][0:20:359]&cloudydap=UC1_A1CFT_STARTED_1485208202.h5

  3. The requests that utilize BES XML commands like this:

<?xml version="1.0" encoding="UTF-8"?>
<request reqID="some_unique_value" >
    <setContext name="dap_format">dap2</setContext>
    <setContainer name="MERRA2_400.tavgM_2d_int_Nx.201312" space="catalog">
        /s3/dap/cloudydap/merra2/MERRA2_400.tavgM_2d_int_Nx.201312.nc4
    </setContainer>
    <define name="d">
        <container name="MERRA2_400.tavgM_2d_int_Nx.201312" >
            <constraint>
                PRECCU[0:1:0][180:1:180][288:1:288]
            </constraint>
        </container>
    </define>
    <get type="dods" definition="d" />
</request>

will each need to be changed to employ the cloudy dap context tags

<?xml version="1.0" encoding="UTF-8"?>
<request reqID="some_unique_value" >
    <setContext name="dap_format">dap2</setContext>
    <setContext name="cloudydap">UC1_A1CFT_STARTED_1485208202.h5</setContext>
    <setContainer name="MERRA2_400.tavgM_2d_int_Nx.201312" space="catalog">
        /s3/dap/cloudydap/merra2/MERRA2_400.tavgM_2d_int_Nx.201312.nc4
    </setContainer>
    <define name="d">
        <container name="MERRA2_400.tavgM_2d_int_Nx.201312" >
            <constraint>
                PRECCU[0:1:0][180:1:180][288:1:288]
            </constraint>
        </container>
    </define>
    <get type="dods" definition="d" />
</request>

Notes

  1. The URLs used for the different architectures will also change, as will the setContainer objects sent to the BES.
  2. I have made most of these changes to the scripts in cloudydap/shell/aggEmulator

Clone this wiki locally