Permalink
What's new for Hyrax 1.16.3 | |
---------------------------- | |
CF option: | |
1. Enhance the support of handling HDF-EOS2 swath multiple dimension map pairs. | |
The enhancement includes the support of multiple swaths. | |
This fix solves the MOD09/MYD09 issue docoumented in HFRHANDLER-332. | |
Note for this enhancement: | |
1) Limitations | |
(1) The Latitude and Longitude must be 2 dimensional arrays. | |
(2) The number of dimension maps must be an even number in a swath. | |
(3) The handling of MODIS level 1B is still kept as the old way. | |
(4) When there is a one pair of dimension maps in a swath and the | |
the geo-dimensions defined in the dimension maps are only used | |
by 2-D Latitude and Longitude fields, we still keep the old way. | |
2) Variable/dimension name conventions | |
(1) The HDF-EOS2 file contains only one swath | |
The swath name is not included in the variable names. | |
For Latitude and Longitude, the interpolated Latitude and Longitude variable | |
names are named as "Latitude_1","Latitude_2","Longitude_1","Longitude_2". | |
The dimension and other variable names are just modified by following the | |
CF conventions. | |
A DDS example can be found under https://github.com/OPENDAP/hdf4_handler/ | |
Go to the following directory: | |
/bes-testsuite/h4.nasa.with_hdfeos2/MYD09.dds.bescmd.baseline | |
(2) The HDF-EOS2 file contains multiple swaths | |
The swath name are included in the variable and dimension names to | |
avoid name clashings. | |
The swath names are added as suffix for variable and dimension names. | |
Examples are like: | |
"temperature_swath1","Latitude_swath1","Latitude_swath1_1" etc. | |
A DDS example can be found under https://github.com/OPENDAP/hdf4_handler/ | |
Go to the following directory: | |
/bes-testsuite/h4.with_hdfeos2/swath_3_3d_dimmap.dds.bescmd.baseline | |
3) For applications that don't want to handle dimension maps, one can change | |
the BES key "H4.DisableSwathDimMap=false" at h4.conf.in to | |
"H4.DisableSwathDimMap= true". | |
2. Add a BES key to turn off the handling of HDF-EOS2 swath dimension map. | |
What's new for Hyrax 1.16.2 | |
---------------------------- | |
Clean up some compiler warnings. | |
What's new for Hyrax 1.16.1 | |
---------------------------- | |
Default option: | |
Fix the memory leaking caused by handling vdata. | |
What's new for Hyrax 1.15.4 | |
---------------------------- | |
CF option: | |
1. Map the AIRS version 6 HDF-EOS Grid/Swath attributes to DAP2. | |
What's new for Hyrax 1.15.3 | |
---------------------------- | |
CF option: | |
1. Enhance the support of handling the scale_factor and add_offset to | |
follow the CF. The scale_factor and add_offset rule for the MOD/MYD21 | |
product is different than other MODIS products. We make an exception | |
for this product only to ensure the scale_factor and add_offset follow | |
the CF conventions. | |
What's new for Hyrax 1.15.0 | |
---------------------------- | |
CF option: | |
1. Enhance the support of handling the scale_factor and add_offset to | |
follow the CF. The scale_factor and add_offset rule for the MOD16A3 | |
product is different than other MODIS products. We make an exception | |
for this product only to ensure the scale_factor and add_offset follow | |
the CF conventions. | |
What's new for Hyrax 1.14.0 | |
---------------------------- | |
CF option: | |
1. Enhance the support of handling the scale_factor and add_offset to | |
follow the CF. The scale_factor and add_offset rule for the MOD16A2 | |
product is different than other MODIS products. We make an exception | |
for this product only to ensure the scale_factor and add_offset follow | |
the CF conventions. | |
Default option(Not developed by The HDF Group): | |
1. We fixed several coding issues discovered by the coverity scan. We | |
also fixed quite a few memory leaking issues discovered by valgrind. | |
What's new for Hyrax 1.13.4 | |
---------------------------- | |
This is a maintenance release. No new features are added and no | |
outstanding bugs are fixed. | |
***************************************************** | |
********Special note about the version number******** | |
***************************************************** | |
Since Hyrax 1.13.2, Hyrax just treats each handler as a module. | |
So We stop assigning an individual version number for the HDF4 handler. | |
Updated for version 3.12.2(November 16 2016, Hyrax 1.13.2) | |
1) Minor code re-arrangement by removing the DAP4 macro and other misc. minor code updates. | |
2) The [Known Issues] is updated to reflect the current findings of some issues that either not | |
appropriate or not worth the effort for the handler to handle. | |
[Known Issues] | |
1) For AMSR E level-3 data, the variable like SI_12km_N_18H_DSC doesn't have any CF scale/offset, | |
_FillValue, unit attributes. These attributes have to be retrieved from the product document. | |
Therefore, the plot generated directly from the OPeNDAP handler may not be correct. | |
If AMSR_E level 3 HDF4 products are still served by NASA data centers, the Hyrax's NcML module | |
can be used with the HDF4 handler to provide the missing CF attributes to generate the correct | |
plot. | |
More information on how to use NcML to provide the missing CF attributes, | |
please check http://hdfeos.org/examples/ncml.php | |
2) For the HDF-EOS2 swath, the HDF4 handler identifies the latitude and longitude coordinate variables | |
based on the variable names rather than via the CF units attribute since one cannot use the HDF-EOS2 | |
library to add attributes for an HDF-EOS2 field. Therefore, the HDF-EOS2 swath field attributes | |
can only be added and retrieved by using the HDF4 APIs. It is difficult and inefficient for the | |
handler to use HDF4 APIs when handling the HDF-EOS2 swath fields. Given that all NASA HDF-EOS2 | |
swath files we observed simply use name "Latitude" for the latitude field and "Longitude" for the | |
longitude field, we believe using variable names "Latitude" and "Longitude" to identify latitude | |
and longitude under swath geo-location fields is sufficient to serve NASA HDF-EOS2 products. Even | |
if the handler cannot identify the latitude and longitude as coordinate variables, the handler | |
will still generate the correct DDS and DAS and Data responses although they may not follow the | |
CF conventions. | |
3) DataCache may not work on MacOS for rare cases. | |
Some HDF4 variables can be cached in disk. The cached file name uses the variable name as the key | |
to distinguish each other. On MacOS system that is not configured to be case-sensitive | |
(using diskutil info to check your OS), when the DataCache BES key is turned on, two legal variable | |
names like TIME and Time with the same shape may share the same cached file name. Thus this may | |
cause the data inconsistent. | |
Given that the DataCache feature is turned off by default and the usage of this feature is only | |
on Linux, the handler just documents this as an known issue. | |
Updated for version 3.12.1(April 29 2016, Hyrax 1.13.1)) | |
1) Improve the calculation of XDim and YDim for the sinusoidal projection. | |
2) Improve the handling of BES keys by moving the initialization of BES keys to the Handler constructor. | |
3) Bug Fixes: | |
(1) Remove the character dot(.) when the _FillValue is NaN. | |
In the previous version, the character dot(.) was added at the end of NaN. This prevents the NetCDF Java clients to access the DAS. | |
(2) Correct the (0,360) longitude value when the HDF-EOS2 EnableEOSGeoCacheFile is turned on. | |
Some HDF-EOS2 products make the longitude range from 0 to 360. When the HDF-EOS2 lat/lon cache is on, the longtiude values were not retrieved | |
correctly in the previous versions. This was discovered by testing an AMSR-E product when updating the testsuite for the handler. | |
[Known Issues] | |
1) DataCache may not work on MacOS for rare cases. | |
Some HDF4 variables can be cached in disk. The cached file name uses the variable name as the key to distinguish each other. | |
On MacOS system that is not configured to be case-sensitive(using diskutil info to check your OS), | |
when the DataCache BES key is turned on, two legal variable names like TIME and Time with the same shape may share the same cached file name. | |
Thus this may cause the data inconsistent. | |
Given that the DataCache feature is turned off by default and the usage of this feature is only on Linux, | |
the handler just documents this as an known issue. | |
2) Note for version 3.12.0: no new features or bug fixes for version 3.12.0. Only several warnings are removed. | |
The version was accidently bumped from 3.11.7 to 3.12.0. | |
Updated for version 3.11.7 (15 September 2015,Hyrax 1.13.0,1.12.2,1.12.1) | |
New Features: | |
1) We added 1-D coordinate variables and CF grid_mapping attributes for HDF-EOS2 grid with Sinusoidal projection. | |
This request is from LP DAAC. | |
2) We added the DDS and DAS cache for AIRS version 6 products and the data cache for HDF4 data read via SDS interfaces. | |
Several other improvement is also made for AIRS. This request is from GES DISC. | |
Bug Fixes: | |
1) We fixed a bug caused by casting values via pointers between different datatypes. | |
This is first reported by NSIDC and then from OPeNDAP. | |
2) We fixed the attribute missing issue for the HDF-EOS2 swath geo-location variables.(variables under Geolocation Fields) | |
3) We also made the correct representation of scale_factor and add_offset CF attributes for HDF-EOS2 swath geo-location variables. | |
****** | |
You may need to read the information about BES keys in the file h4.conf.in to see if the default values | |
need to be changed for your service. | |
****** | |
Updated for version 3.11.6 (15 November 2014,Hyrax 1.11.2,1.11.1,1.11.0,1.10.1,1.10.0) | |
In this version, we added the following features: | |
1) Implement an option to cache HDF-EOS2 grid latitude and longitude values to improve performance. | |
2) Implement an option not to pass file ids for the compatibility with NcML modules. | |
3) Improve the access of DDS by using the file structure obtained in the stage of building DAS. | |
4) Add the CF support of AIRS version 6 level 2(swath). | |
5) Support the mapping of vgroup attributes to DAP. | |
6) Support the mapping of HDF-EOS2 swath and grid object(like vgroup) attributes to DAP. | |
Bug fixes: | |
1) Obtained the CF add_offset values for MOD/MYD08_M3 products. | |
2) Fixed the wrong dimension size when a dimension is unlimited. | |
3) Make vdata field attributes mapped to correct DAS container. | |
NASA HDF4 and HDF-EOS2 products that are supported up to this release: | |
HDF-EOS2: AIRS, MODIS, some MISR, some Merra, some MOPITT | |
HDF4: TRMM version 6 and version 7, some CERES, some OBPG | |
Performance enhancement: | |
AIRS version 6 and MODIS O8_M3-like products | |
HDF-EOS2 grid lat/lon are calculated via cache | |
You may need to read the information about BES keys in the file h4.conf.in to see if the default values | |
need to be changed for your service. | |
Updated for version 3.11.5 (25 April 2014,Hyrax 1.9.7) | |
In this version, we fix the following datatype mapping issues: | |
1) We make HDF4 DFNT_CHAR array map to DAP string for variables. | |
In the previous versions, DFNT_CHAR is mapped to DAP BYTE for variables. | |
This may cause some NASA HDF4 vdata DFNT_CHAR array to DAP BYTE array, which is not right. | |
The fix makes the vdata fields of some NASA MISR (MISR_AEROSOL etc.) products map correctly to DAP. | |
2) We fix the mapping of DFNT_INT8 attribute value to DAP. | |
In the previous versions, DFNT_INT8(signed 8-bit integer) is mapped to DAP BYTE(unsigned 8-bit integer). | |
This will cause misrepresentation when the value is < 0. | |
This fix will make some attribute values of some MODIS or MISR non-physical fields map correctly to DAP. | |
3) For attribute _FillValue, we enforce that it is always a number even if the attribute datatype is | |
DFNT_CHAR or DFNT_UCHAR. In this release,we make the string representation of a _FillValue to a number. | |
We also make turn on the H4.DisableStructMetaAttr key to be 'true'the default setting of h4.conf.in. This | |
will improve the performance to generate DAS. | |
Updated for version 3.11.4 (1 April 2014,Hyrax 1.9.3) | |
This version improves I/O by reducing the number of HDF4/HDF-EOS2 file open / | |
close requests. | |
The default BES key settings are changed. See USAGE document. | |
Support to TRMM version 7 (3B43, 3B42, 3B31, 3A26, 3A25, 3A12, 3A11, 2B31, | |
2A25, 2A23, 2A21, 2A12, 1C21, 1B21, 1B11 and 1B01) and some TRMM level 3 | |
version 6(CSH and 3A46) products are added. The emphasis is on level 3 grid | |
products. | |
Some memory leaks detected by valgrind are fixed. | |
Error handling is greatly improved:resources are released properly when errors | |
occur. | |
The testsuite is updated accordingly. | |
All the updates are solely applied to the CF option. | |
[Known Issues] | |
Note:HFRHANDLER-??? means The HDF Group's JIRA ID. | |
HFRHANDLER-223:(U)CHAR8 type _FillValue attribute value conversion produces | |
string representation, instead of real value when the dataset is in numeric | |
type. | |
HFRHANDLER-224:Some attributes of Lat/Lon geolocation fields in HDF-EOS2 | |
swath are dropped in DAS output. | |
Updated for version 3.11.3 (1 February 2014,Hyrax 1.9.2) | |
This version optimizes handler for MOD08 M3 and AIRS version 6 products with | |
new BES keys. See the USAGE document for the new BES keys for details. | |
Updated for version 3.11.2 (18 October 2013) | |
This version changes BaseType::read(). | |
Updated for version 3.11.1 (10 September 2013) | |
This version addresses some issues in the code base. | |
Updated for version 3.11.0 (30 May 2013) | |
Limitation of handling MOD14 product is documented under USAGE. MOD14 is not | |
an HDF-EOS2 product so having MOD03 geo-location file will not change the | |
output of the HDF4 handler. | |
This version fixes the 7.6 of "Known Issues" documented in README for version | |
3.10.0. Search for "7.6" in this document for details. | |
Updated for version 3.10.1 (27 Nov 2012) | |
This version fixes a bug for reading Int16 type dataset from NASA MODIS 09 | |
products. | |
Updated for version 3.10.0 (30 Sep 2012) | |
1. Introduction | |
The HDF4 handler version 3.10.0 is a major update for the CF support. The | |
README for this version is *solely* for the CF support. | |
The README consists of several sections: Usage, New Features, Bug fixes, | |
Code Improvement, Limitations, and Known Issues. | |
2. Usage | |
The current version uses BES Keys to make the configuration of the CF option | |
more flexible and easier. The current version also doesn't require | |
the HDF-EOS2 library to support the CF conventions for non-HDFEOS2 HDF4 files. | |
Check the USAGE for the detailed information. | |
3. New Features | |
3.1 Testsuite Expansion | |
If you configure the handler with '--with-hdfeos2', the 'make check' will | |
test a new set of HDF-EOS2 test files that are added. Please make sure to | |
clean everything with 'make distclean' if you want to test with a different | |
configuration option each time. | |
Source codes are also provided for the new set of HDF-EOS2 test files. | |
3.2 Naming conventions follow HDF5 handler naming conventions for consistency. | |
Again, this is for the CF support only. | |
3.2.1 Non-CF-compliant characters | |
Any non-CF-compliant character for objects and their attribute names will be | |
changed to '_'. There is only one exception: if the first character of a name | |
is '/', handler will ignore the first '/'. This is a request from NASA data | |
centers since it means that the a path is prefixed to the name. The first '/' | |
is ignored for better readability. | |
The object names include all HDF-EOS2 and HDF4 objects supported by the CF | |
option. They are HDF-EOS2 swath fields and grid fields, HDF4 SDS, HDF4 Vdata, | |
HDF4 Vdata fields and corresponding attributes. | |
3.2.2 Multiple HDF-EOS2 swath and grid | |
For multiple HDF-EOS2 swath and grid files, since we only find a bunch of | |
AIRS and MODIS grid products in this category and so far the field names under | |
these grids can be distinguished by themselves, so we simply keep the original | |
field names. Grid names are not prefixed. | |
For example, in AIRS.2002.08.24.L3.RetStd_H008.v4.0.21.0.G06104133343.hdf | |
there are two fields with similar names: | |
1) Field under vgroup "ascending": TotalCounts_A | |
2) Field under vgroup "descending": TotalCounts_D | |
As you can see, the field "TotalCounts" in each vgroup can be distinguished | |
by their field names anyway. Prefixing a grid name to field name such as | |
"ascending_TotalCounts_A" and "descending_TotalCounts_D" makes the field name | |
difficult to read. | |
3.2.3 Non-HDFEOS2 SDS objects | |
For pure HDF4 SDS objects, the handler prefixes the vgroup path for SDS object | |
names. | |
3.2.4 SDS objects in Hybrid HDF-EOS2 files | |
For SDS objects in hybrid HDF-EOS2 files, to make these SDS objects | |
distinguished from the HDF-EOS2 grid or swath field names, handler adds | |
"_NONEOS" to the object name. This is based on the following fact: | |
These added SDS objects often share the same name as the HDF-EOS2 fields. | |
For example, band_number can be found both under HDF-EOS2 swath and under the | |
root vgroup. Thus, name clashing needs to be done. Adding "_NONEOS" is a better | |
way to handle the name clashing than simply adding a "_1". | |
3.2.5 Vdata object name conventions | |
The vdata name conventions are subject to change. We will evaluate the current | |
name conventions after hearing feedback from users. Since the main NASA HDF4 | |
object is SDS, hopefully this Vdata name conventions may not be critical. | |
3.2.5.1 Vdata object mapped to DAP array | |
Since we would like to make sure that users can easily figure out the DAP | |
variables mapped from vdata, we use the following name conventions. | |
For example, a vdata "level1B" has a field "scan" under the group "g1" | |
The mapped DAP variable name is "vdata_g1_level1B_vdf_scan." The "vdata_" | |
prefix tells the user that this is an original vdata. The "_vdf_" tells the | |
user that the string followed by "_vdf_" is the vdata field name. | |
The handler also adds the following attributes to DAS to serve the similar | |
purpose: | |
std::string VDdescname = "hdf4_vd_desc"; | |
std::string VDdescvalue = "This is an HDF4 Vdata."; | |
std::string VDfieldprefix = "Vdata_field_"; | |
These attributes will be generated if the corresponding BES key is turned on. | |
See USAGE for details. | |
3.2.5.2 Vdata fields mapped to DAP attributes | |
For the current version, if the number of vdata records is less than 10, | |
the vdata field will be mapped to DAP attributes. This is based on the fact | |
that small number of vdata records is mostly metadata (like attributes), | |
so this kind of mapping may be more reasonable. However, the attribute name | |
that are mapped from vdata field simply follows the conventions below: | |
<vdata path>_<vdata name>_<vdata field name> | |
The above name convention is simply following the handling of SDS. The | |
rationale here is that distinguishing between SDS and Vdata fields in DAP | |
attribute(DAS) is not as critical as distinguishing between SDS and Vdata | |
fields in DAP array(DDS). However, this may cause user's confusion. | |
We will evaluate this approach in the future based on user's feedback. | |
3.3 Name clashings | |
Name clashing is handled similarly like the HDF5 handler. Only the name that | |
has name clashing will be changed to a different name. This is according to | |
NASA user's request. Previously, all corresponding names will be changed if | |
there is a name clashing found in the file. | |
See 3.2 for the details about the naming conventions. | |
3.4 New BES Keys | |
There are six BES keys are newly added. | |
H4.EnableCF | |
H4.EnableMODISMISRVdata | |
H4.EnableVdata_to_Attr | |
H4.EnableCERESMERRAShortName | |
H4.DisableVdataNameclashingCheck | |
H4.EnableVdataDescAttr | |
H4.EnableCF is the most important key. It must be set to be true if CF | |
option is used. Other five keys are valid only if H4.EnableCF is set to be | |
true. | |
For more information, check the USAGE file. | |
3.5 Dimension maps and scale/offset handling for MODIS swath | |
3.5.1 Interpolating more fields according to the dimension maps | |
Previously, we only supported the interpolation of latitude and longitude. | |
In this release, we add the support of other fields such as solar_zenith, etc. | |
if dimension map is used to store the field. Additionally, we also provide an | |
option to directly read these fields (latitude, longitude, solar zenith etc.) | |
from MOD03 or MYD03 files distributed by LAADS. For more information on this | |
option, check the USAGE file. | |
3.5.2 Enhance the scale/offset handling for MODIS level 1B fields | |
According to the discussions with NASA users, we found that for fields such | |
as EV_1KM_RefSB and EV_1KM_Emissive, for values greater than 65500 are special | |
values that should not be considered as the real data signal. So in this | |
release, we keep values between 65500 and 65535 when applying the scale and | |
offset function. Furthermore, we also calculated valid_min and valid_max for | |
EV_???_RefSB and EV_???_Emissive to assure that users can plot data easily | |
with Panoply and IDV. | |
3.6 Dimensions in MERRA products | |
In this release, the handler uses HDF-EOS2 latitude and longitude fields as | |
the coordinate variables. The added HDF4 SDS XDim and YDim are used inside the | |
file as coordinate variables. | |
3.7 Vdata mapping | |
The new handler can handle vdatas robustly either as an attribute in DAS or | |
as an array in DDS. Review the previous 3.2.5.2 section for details. | |
Several BES keys are also provided for users to have options on how to | |
generate vdata DAP output. See USAGE for details. | |
Vdata subsetting is handled robustly except for some HDF-EOS2 Vdata objects. | |
See section 7.5 for the known issue. | |
3.8 Handling general HDF4 files with the CF option | |
The previous release listed the NASA HDF4 products we handled specially to | |
follow the CF convention. For other HDF4 products (we call them general HDF4 | |
files), we do the following: | |
A. Make the object and attribute names follow the CF conventions | |
B. Follow the default handler's mapping to map SDS. In this way, some SDS | |
objects that have dimension scales can be visualized by Panoply and IDV. | |
C. Map Vdata according to 3.7. | |
4. Bug Fixes | |
4.1 Attribute names are cleaned up. If an attribute name contains non alpha | |
numeric characters like '(' or '%', they are replaced with '_' to meet the | |
CF naming conventions. | |
4.2 Products that use SOM projection are handled correctly and Panoply can | |
display MISR data in block-by-block basis. However, please read 7.11 for | |
the known issue for the interpretation of final visualization output. | |
4.3 We continued correcting attributes related to scale/offset. For example, | |
the handler corrected "SCALE_FACTOR" and "OFFSET" attributes for AMSR_E L2A | |
product by renaming them to "scale_factor" and "add_offset". It also cleaned | |
any extra spaces in attribute names. We also corrected the rule on how to | |
applying scale/offset equations in MODIS product (e.g., Solar_Zenith dataset). | |
Finally, the handler renames Number_Type attributes to Number_Type_Orig if | |
data field's type is changed by applying scale/offset by the handler (e.g., | |
LPDAAC MOD11_L2 LST). | |
4.4 Strange ECS metadata sequences are handled. Some MODIS products have | |
metadata name sequence "coremetadata.0, coremetadata.0.1, ..." instead of | |
"coremdatadata.0, coremetadata.1, ..." | |
4.5 Mismatched valid_range attribute is removed from CERES ES4 product. | |
Panoply fails to visualize the product if the valid_range attribute in | |
lat/lon dataset doesn't match the calculated coordinate variable values | |
returned by the handler. Thus, the handler removes the valid_attribute from | |
coordinate variables. | |
4.6 There was a bug regarding subsetting the vdata field when the stride is | |
greater than 1. It was fixed in this release. We also found a similar bug | |
inside the HDF-EOS2 library regarding the subsetting of 1-D HDF-EOS2 swath. | |
This should be fixed in the next HDF-EOS2 release. | |
5. Code Improvement | |
5.1 Refactored codes | |
There were huge code overlap between hdfdesc.cc and HE2CF.cc. Those code | |
lines are combined and are moved to HDFCFUtil.cc. | |
5.2 Error handlings and debugging | |
Error handlings are improved to ensure the closing of all opened HDF4 API | |
handles when error occurs. DEBUG macros are replaced with BESDEBUG or BESLog. | |
6. Limitations | |
Again, all the limitations here are for the CF support (when the CF option is | |
enabled) only. | |
6.1 Unmapped objects | |
The handler ignores the mapping of image, palette, annotation, vgroup | |
attributes and HDF-EOS2 swath group and grid group attributes. Note HDF4 global | |
attributes(attributes from SD interfaces), HDF4 SDS objects, HDF4 Vdata | |
attributes, HDF-EOS2 global(ECS metadata etc.) and field attributes are mapped | |
to DAP. | |
6.2 The handler doesn't handle unlimited dimension. | |
The result may be unexpected. (e.g., some fields in CER_ES8_TRMM-PFM_Edition2 | |
product cannot be handled.) | |
6.3 Non-printable vdata (unsigned) character type data will not appear in DAS. | |
If vdata char type column has a non-printable value like '\\005', it will not | |
appear in DAS when vdata is mapped to attribute because the BES key, | |
H4.EnableVdata_toAttr, is enabled. See the file USAGE for the usage of the key. | |
6.4 Vdata with string type is handled in character-by-character basis in 2D | |
array. | |
For example, when the vdata is a string of characters like | |
"2006-1--01T16:17:12.693310Z", | |
the handler represents it as | |
Byte vdata_PerBlockMetadataTime_vdf_BlockCenterTime[VDFDim0_vdata_PerBlockMetadataTime_vdf_BlockCenterTime = 2][VDFDim1_vdata_PerBlockMetadataTime_vdf_BlockCenterTime = 28] = {{50, 48, 48, 54, 45, 49, 48, 45, 48, 49, 84, 49, 54, 58, 49, 55, 58, 49, 50, 46, 54, 57, 51, 51, 49, 48, 90, 0},{50, 48, 48, 54, 45, 49, 48, 45, 48, 49, 84, 49, 54, 58, 49, 55, 58, 51, 51, 46, 52, 51, 56, 57, 50, 54, 90, 0}}; | |
7. Known Issues | |
7.1 CER_SRBAVG3_Aqua-FM3-MODIS_Edition2A products have many blank spaces in | |
long_name attribute. | |
These products have datasets with really long_name attribute with size 277. | |
However, most of them are blank spaces in the middle. For example, you'll see | |
DAS output like below: | |
String long_name "1.0 Degree Regional Monthly Hourly (200+ blank spaces) | |
CERES Cloud Properties"; | |
String units "unitless"; | |
This is not a bug in the handler. The data product itself has such long | |
attribute. | |
7.2 Longitude values for products that use LAMAZ projection will differ in i386 | |
platform. | |
For i386 machines, handler will generate different longitude values from | |
x86_64 machines for the products that use Lambert azimuthal projection (LAMAZ) | |
near North Pole or South Pole. For example, the handler will return 0 | |
for 64-bit machine while it'll return -135 for 32-bit machine in the middle | |
point of longitude in the NSIDC AMSR_E_L3_5DaySnow_V09_20050126.hdf product: | |
< Float64 Longitude[YDim = 3][XDim = 3] = {{-135, 180, 135},{-90, 0, 90}, | |
{-45, 0, 45}}; | |
--- | |
> Float64 Longitude[YDim = 3][XDim = 3] = {{-135, -180, 135},{-90, -135, 90}, | |
{-45, -1.4320576915337e-15, 45}}; | |
This is due to the calculations in the current GCTP library that HDF-EOS2 | |
library uses. However, this will not affect the final visualization because, | |
for North Pole or South Pole, the longitude can be anything from -180 to 180. | |
So depending on floating point accuracy, handler may get different results for | |
longitude of this pixel from GCTP. The longitude value is irrelevant at North | |
Pole or South Pole for visualization clients. | |
7.3 IDV can't visualize SOM projection | |
MISR products that use SOM projection have 3D lat/lon. Although Panoply can | |
visualize them but IDV cannot. Handler doesn't treat the 3rd dimension as | |
a separate coordinate variable and coordinate attribute on dataset includes | |
only latitude and longitude variable names. | |
7.4 Vdata is mapped to attribute if there are less than or equal to 10 records | |
For example, the DAS output of TRMM data 1B21 will show Vdata as an attribute: | |
DATA_GRANULE_PR_CAL_COEF { | |
String hdf4_vd_desc "This is an HDF4 Vdata."; | |
Float32 Vdata_field_transCoef -0.5199999809; | |
Float32 Vdata_field_receptCoef 0.9900000095; | |
Float32 Vdata_field_fcifIOchar 0.000000000, 0.3790999949, 0.000000000, | |
-102.7460022, 0.000000000, 24.00000000, 0.000000000, 226.0000000, 0.000000000, | |
0.3790999949, 0.000000000, -102.7460022, 0.000000000, 24.00000000, 0.000000000, | |
226.0000000; | |
} | |
This is part of the vdata handling convention, not an issue. However, we | |
list here just for user's convenience. See 3.7 for more information. | |
7.5 Vdata subsetting in HDF-EOS2 data products may not work. | |
Subsetting HDF-EOS2 vdata with large step index (e.g. a_vdata[0:999:999]) | |
may not work due to a bug in HDF-EOS2 library. The bug has been reported to | |
HDF-EOS2 developer and should be fixed in the next HDF-EOS2 release. Reading | |
the entire Vdata is OK. | |
7.6 DDX generation will fail on PO.DAAC AVHHR product. | |
For example, handler can't generate DDX output for NASA JPL PO.DAAC AVHRR | |
product 2006001-2006005.s0454pfrt-bsst.hdf. Please see the OPeNDAP ticket #1930 | |
for details. | |
7.7 It is possible to have name clashing between dimension names and variable | |
names. Currently, the handler only checks the name clashing for the variables | |
and the name clashing for the dimensions, not combined. Here's a reason: | |
Many good COARDS files will have the following layout: | |
lat[lat=180] | |
lon[lon=360] | |
If we want to check the name clashings for the combined set, this kind of good | |
files will always have name clashings. And depending on the code flow, the | |
final layout may become something like: | |
lat_1[lat_1=180] | |
lon_1[lon_1=360]. | |
These are absolutely bad names for normal users. Instead, if we don't consider | |
the combined set, the chance of the name clashing due to changing the | |
conflicted coordinate variable names is very rare. So we may not do this at | |
all until we really find a typical product that causes a problem. | |
7.8 long_name attribute for <variable name>_NONEOS | |
The handler generates long_name attribute to indicate the original variable | |
name for the SDS objects after renaming them with _NONEOS suffix. Those | |
_NONEOS variables appear in hybrid files --- files that have additional HDF4 | |
objects written by the HDF4 APIs on top of the existing HDF-EOS2 files. This | |
addition of long_name attribute cannot be turned off using the | |
H4.EnableVdataDescAttr=false key described in USAGE document. | |
7.9 Empty dimension name creates a variable with empty name. | |
It is possible to create dataset with no dimension name in HDF-EOS2 library. | |
In such case, the handler generates fake dimension variable without | |
dataset name in DDS like below: | |
Int32 [4]; | |
Since there's no dataset name, data reading will also fail. | |
7.10 Special CF handlings for some products | |
The handler doesn't correct scale_factor/add_offset/_FillValue for every HDF4 | |
product to make it follow CF conventions. | |
For example, the handler doesn't apply the scale and offset function(log | |
equation) for OBPG CZCS Level 3 products. | |
The handler doesn't insert or correct fill value attribute for some OBPG | |
products so their plots may include fill values if you visualize them. OPeNDAP | |
server administrator can fix these easily by using the NcML handler. | |
PO.DAAC AVHRR product 2006001-2006005.s0454pfrt-bsst.hdf has "add_off" and | |
doesn't specify fill value in attribute. Therefore, the final visualization | |
image will not be correct for such product. | |
7.11 Bit shifting required in MISR product is not handled. | |
Some datasets in MISR products combine two datasets into one, which requires | |
bit shifting for the correct interpretation of the data. Handler doesn't | |
perform such operation so the final visualization image may not be correct. | |
(e.g., Blue Radiance in MISR_AM1_GRP_ELLIPSOID_GM_P117_O058421_BA_F03_0024.hdf) | |
7.12 Subsetting through Hyrax HTML interface will not work on LAMAZ products. | |
You cannot subset Latitude_1 and Longitude_1 datasets using the HTML form. | |
Checking the check box will not insert any array subscription text into the | |
text box. Please see the OPeNDAP ticket #2075 for details. | |
Kent Yang (myang6@hdfgroup.org) | |
Updated for version 3.9.4 (19 Jan, 2011) | |
If your system is non-i368 such as 64-bit architecture, please read | |
IMPORTANT NOTE on INSTALL document regarding '--with-pic' configuration option | |
(i.e., '-fPIC' compiler option). You need to install both HDF4 (and HDF-EOS2) | |
library with '--with-pic' option if you encounter a linking problem. | |
1. Fixed the bug in uint16/uint32 type attribute handling. | |
2. The following bug fix applies to only --with-hdfeos2 configuration option. | |
2.1 Corrected the handling the scale/offset for MODIS products because the | |
MODIS scale/offset equation is quite different from the CF standard. | |
There are three different "scale_factor" and "add_offset" equations in MODIS | |
data files: | |
1) For MODIS L1B, MODIS 03,05,06,07,08,09A1,17 and ATML2 level 2 swath | |
products, MCD43B4, MCD43C1, MOD and MYD 43B4 level 3 grid files, the scale | |
offset equation is | |
correct_data_value = scale * (raw_data_value - offset). | |
2) For MODIS 13, MODIS 09GA, and MODIS 09GHK, the scale offset equation is | |
correct_data_value=(raw_data_value -offset)/scale_factor. | |
3) For MODIS 11 level 2 swath products, the equation is | |
correct_data_value = scale * raw_data_value + offset. | |
We decide the type based on the group name. | |
If the group name consists of "L1B", "GEO", "BRDF", "0.05Deg", "Reflectance", | |
"MOD17A2","North", "mod05", "mod06", "mod07", "mod08", or "atm12", | |
it is type 1. | |
If the group name consists of "LST", it is type 2. | |
If group name consists of "VI", "1km_2D", "L2g_2d", it is type 3. | |
For type 1, use (raw_value-offset)*scale. | |
For type 2, use (raw_value*scale+offset). | |
For type 3, use (raw_value-offset)/scale. | |
For recalculation of MODIS, one of the following conditions must meet: | |
"radiance_scales" and "radiance_offsets" attributes are available, or | |
"reflectance_scales" and "reflectance_offsets" attributes are available, or | |
"scale_factor" and "add_offset" attributes are available. | |
If any of the above conditions meet, recalculation will be applied, | |
otherwise nothing will happen. If scale is 1 and offset is 0, we don't | |
perform recalculation to improve performance. | |
Data values are adjusted based on it type. If "scale_factor" and "add_offset" | |
attributes are not available, "radiance_scales" and "radiance_offsets" | |
attributes, or "reflectance_scales" and "reflectance_offsets" attributes, are | |
used instead. | |
After adjustment, the data type is converted uniformly to float not to lose | |
precision even if its original data type is integer. The "valid_range" | |
attribute is removed accordingly as it does not reflect the actual values | |
any more. | |
Since some netCDF visualization tools will apply the linear scale and offset | |
equation to the data value if the CF "scale_factor" and "add_offset" attributes | |
appear, these two attributes are renamed as "orig_scale_factor" and | |
"orig_add_offset" respectively to prevent the second adjustment. | |
2.2 Latitude and longitude are provided for HDF-EOS2 grid files that use | |
Space-Oblique Mercator(SOM) and Lambert Azimuthal Equal Area(LAMAZ) | |
projections. | |
We added the support for LAMAZ projection data such as MOD29 from NSIDC. | |
For grid files using HDF-EOS2 Lambert Azimuthal Equal Area(LAMAZ) projection, | |
the latitude and longitude values retrieved from the HDF-EOS2 library include | |
infinite numbers. Those infinite numbers are removed and replaced with new | |
values through interpolation. Therefore, an HDF-EOS2 grid file with LAMAZ | |
projection can be served correctly. | |
2.3 Fixed memory release error that occurs on iMac (OS X Lion) with the STL | |
string:string Map. | |
2.4 For OBPG L3m products, two additional CF attributes, "scale_factor" and | |
"add_offset", are added if their scaling function is linear. The values of | |
these two attributes are copied directly from file attributes, "Slope" and | |
"Intercept". | |
2.5 Known Bugs: | |
1) Attribute names are not sanitized if they contain non-CF compliant | |
characters such as '('. NSIDC MOD29 data product is a good example. | |
2) If different scale/offset rules should be applied to different datasets | |
like MOD09GA product, the current handler cannot handle them properly. | |
We apply scale/offset rule globally on per file basis, not on per | |
individual dataset basis and CF visualization clients like IDV and | |
Panoply will not display some datasets correctly since they'll apply | |
scale/offset rule according to the CF-convention which doesn't match | |
MODIS's scale/offset rule. | |
Updated for 3.9.3 (21 Aug. 2011) | |
Fixed a lingering issue with the processing of HDF-EOS attributes when | |
the handler is not compiled with the HDF-EOS library. The handler was | |
returning an error because those attributes were not parsing | |
correctly. In the end, it appears there were two problems. The first | |
was that some files contain slightly malformed EOS attributes: the | |
'END' token is missing the 'D'. The second was that this triggered a | |
bogus error message. | |
Fixed the nagging 'dim_0' bug where some files with variables that use | |
similar names for arrays trigger a bug in the code that merges the das | |
into the dds. The results was that sometimes the code tried to add a | |
dim_0 attribute to a variable that already had one. This I fixed by | |
correcting an error in the way the STL's string::find() method was | |
used. | |
Updated for 3.9.2 (17 Mar. 2011) | |
In this patch, we added the following three features: | |
1. We add the mapping of the SDS objects added by using HDF4 APIs to | |
an HDF-EOS2 file. These SDS objects are normally NOT physical fields, | |
so they are not supposed to be plotted by Java tools such as IDV and Panoply. | |
The attributes and values of these SDS objects may be useful for end | |
users. | |
2. We also fix the bug of handling MERRA data. In the previous | |
release, the unit of time is not handled correctly. This release fixes | |
this bug under the condition that the file name of MERRA data must | |
start with MERRA. | |
3. We also enhance the support of mapping HDF4 files that uses HDF4 | |
SDS dimension scales. Especially we make a patch specially for P.O. | |
DAAC's AVHRR files. Now with enough Heap space, IDV can visualize | |
AVHRR files via OPeNDAP. | |
What we haven't done: | |
1. We haven't mapped the Vdata objects added by using HDF4 APIs to an | |
HDF-EOS2 file. | |
2. We haven't handled the plotting of the vertical profile files (Such | |
as MOP Level 2 data). More investigation needs to be done on how IDV | |
can handle things. | |
3. Other limitations listed in Section III of 3.9.1 that are not | |
addressed above. | |
Kent Yang (myang6@hdfgroup.org) | |
Updated for 3.9.1 (14 Sep. 2010) | |
In this release, we greatly enhance the support of the access of NASA | |
HDF-EOS2 and HDF4 products. The whole note for 3.9.0 includes three | |
sections. | |
Section I. Configuration | |
The handler is enhanced to support the access of many NASA HDF-EOS2 | |
products and some NASA pure HDF4 products by many CF-compliant | |
visualization clients such as IDV and Panoply. To take advantage of | |
this feature, one MUST use HDF-EOS2 library and configure with the | |
following option: | |
./configure --with-hdf4=<Your HDF4 library path> | |
--with-hdfeos2=<Your HDF-EOS2 library path> | |
--prefix=<Your installation path> | |
Without specifying the option "--with-hdfeos2" will result in | |
configuring the default HDF4 OPeNDAP handler. The HDF4 handler with | |
the default options can NOT make the NASA HDF-EOS2 products and some | |
NASA pure HDF4 products work with CF-compliant visualization clients. | |
Some variable paths are pretty long(>15 characters). COARDS conventions | |
require the number of characters in a field doesn't exceed 15 | |
characters. So the above configuration option may cause some OPeNDAP | |
clients that are still following COARDS conventions. To compensate | |
that, we provide a configuration option to shorten the name so that in | |
doesn't exceed 15 characters. To address the potential name clashing | |
issue, both options may make some variable names to change so that | |
unique variable names are present in the OPeNDAP output. To best | |
preserve the original variable names, we recommend not to use | |
--enable-short-name option if necessary. To configure the handler with | |
the short name option, do the following: | |
./configure --with-hdf4=<Your HDF4 library path> | |
--with-hdfeos2=<Your HDF-EOS2 library path> | |
--prefix=<Your installation path> --enable-short-name | |
To find the information on how to build the HDF-EOS2 library, please | |
check | |
http://hdfeos.org/software/hdfeos.php#ref_sec:hdf-eos2 | |
To build RPMs by yourself, check the directory 'build_rpms_eosoption'. | |
Section II. NASA products that are supported to be accessed via Java | |
and other OPeNDAP visualization clients | |
The following NASA HDF-EOS2 products are tested with IDV and Panoply, | |
check the Limitation section for the limitations: | |
1). NASA GES DISC | |
AIRS/MERRA/TOMS | |
2). NASA LAADS/LP DAAC/NSIDC | |
Many MODIS products | |
3). NASA NSIDC | |
AMSR_E/NISE products | |
4). NASA LaRC | |
MISR/MOPITT/CERES-TRMM | |
The following NASA special HDF4 products are tested with IDV Panoply, | |
check the Limitation section for the limitations: | |
1). NASA GES DISC | |
TRMM Level 1B, Level 2B Swath | |
TRMM Level 3 Grid 42B and 43B | |
2). OBPG(Ocean Color) | |
SeaWiFS/MODIST/MODISA/CZCS/OCTS level 2 and level 3m(l3m) | |
3). Some LaRC CERES products | |
CER_AVG_Aqua-FM3-MODIS,CER_AVG_Terra-FM1-MODIS | |
CER_ES4_Aqua-FM3_Edition1-CV or similar one | |
CER_ISCCP-D2like-Day_Aqua-FM3-MODIS or similar one | |
CER_ISCCP-D2like-GEO_ or similar one | |
CER_SRBAVG3_Aqua or similar one | |
CER_SYN_Aqua or similar one | |
CER_ZAVG or similar one | |
Section III. Limitations | |
1. Visualization clients and http header size | |
1). Visualization Slowness or even failures with IDV or panoply | |
clients for big size field We found that for big size variable | |
array(>50 MB), the visualization of the variable is very slow. | |
Sometimes, IDV or Panoply may even generate an "out of memory" error. | |
2). Some NASA HDF files(some CERES files e.g.) include many (a few | |
hundred) fields and the field names are long. This will cause the | |
maximum http header size to exceed the default maximum http header | |
size and a failure will occur. To serve those files, please increase | |
your max http header size by adding the following line at your | |
server.xml under the line containing <Connector port="8080" | |
protocol="HTTP/1.1" maxHttpHeaderSize="819200" | |
2. HDF-EOS2 files | |
1) HDF-EOS2 Lambert Azimuthal Equal Area(LAMAZ) projection grid For | |
LAMAZ projection data, the latitude and longitude values retrieved | |
from the HDF-EOS2 library include infinite numbers. So an HDF-EOS2 | |
grid file with LAMAZ projection can not be served correctly. | |
2) Latitude and longitude values that don't follow CF conventions | |
2.1) Missing (Fill) values inside latitude and longitude fields | |
Except the HDF-EOS2 geographic(also called equidirectional, | |
equirectangular, equidistant cylindrical) projection, clients may | |
NOT display the data correctly. | |
2.2) 3-D latitude and longitude fields Except some CERES products | |
listed at section 2, clients may NOT display the data correctly if | |
the latitude and longitude fields are 3-D arrays. | |
3) HDF-EOS2 files having additional HDF4 objects | |
Some HDF-EOS2 files have additional HDF4 objects. The object may be | |
vdata or SDS. That means, some contents are added to an HDF-EOS2 | |
file by using the HDF4 APIs directly after the HDF-EOS2 file is | |
created. The HDF-EOS2 API may not retrieve the added information. Up | |
to this release, we found those information are mainly related to | |
metadata and those metadata may not be critical to visualize and | |
analyze the real physical data variables. So in this release, those | |
objects are currently ignored. One should NOTE that attributes of | |
existing HDF-EOS2 data variables are NOT ignored. | |
4) Variables stored as 3-D or 4-D arrays | |
Some variables stored as 3-D or 4-D arrays are either missing or | |
hard to find the third or the fourth dimension's coordinate | |
variables. The handler will use integer number(1,2,3,......) to | |
represent the third or the fourth dimension as levels. Clients can | |
still visualize the data in a horizontal plane level by level. | |
3. Pure HDF4 files | |
All pure HDF4(e.g. non-HDFEOS2) products we've tested are listed | |
under Section II. Other pure HDF4 products are not tested and may | |
NOT be visualized by Java OPeNDAP clients. | |
4. Misc. | |
For pure HDF4 products, currently attributes in a vgroup(not SDS or SD) | |
are not mapped to DAP. | |
To speed up the performance, we choose not to generate | |
structMetadata, coreMetadata, archiveMetadata attributes for some | |
CERES products. For applications that need these, please contact | |
myang6@hdfgroup.org or eoshelp@hdfgroup.org. You can also post a | |
message at hdfeos.org/forums . | |
Kent Yang(myang6@hdfgroup.org) | |
Updated for 3.8.1 | |
The OPeNDAP HDF4 Data Handler is enhanced by adding two more configuration | |
options: | |
1) --enable-cf | |
2) --use-hdfeos2=/path/to/hdfeos2_library | |
The option 1) uses the StructMetadata parser. A valid HDF-EOS2 file always | |
has the StructMetadata information and the parser can infer geolocation | |
information from the StructMetadata. By retrieving such information, the | |
HDF4 handler can generate DAP Grids that OPeNDAP visualization clients can | |
display on a world map. | |
The option 2) REQUIRES option 1) and it uses the HDF-EOS2 library instead of | |
the StructMetadata parser. The benefit of using HDF-EOS2 library is tremendous. | |
It can support more HDF-EOS2 files by handling different projections like polar | |
and sinusoidal. In addition, it can detect any anomalies that are common in | |
some HDF-EOS2 files and handle them intelligently. Thus, we recommend the | |
server administrator to install HDF-EOS2 library first and configure the | |
handler with BOTH 1) and 2) options. | |
THE LIMITATIONS of the ENHANCED HDF4 HANDLER | |
Please note that the enhanced handler has some limitations. | |
o No support for using the HDF4 handler cache directory. | |
o No support for Grids other than geographic 1-D projection. However, | |
option 2) will make some Grids with other projections (polar, sinusoidal) | |
work. | |
o No vgroup to DAP structure mapping. Thus, the files that have same | |
dataset name under different vgroups will throw a DDS semantics violation | |
error. | |
o No support for files that have the same dimension names with different | |
dimension sizes. For example if a swath variable "A" has dimension lat=360 | |
and lon=1440 (e.g., A[lat=360][lon=1440]) and another swath variable "B" | |
has dimension lat=180 and lon=720 (e.g., B[lat=180][lon=720]), | |
the handler will throw an error for inconsistent dimension. | |
Updated for 3.7.12 (16 March 2009) | |
This is the OPeNDAP HDF4 Data Handler. It is used along with the OPeNDAP | |
DAP Server. | |
For information about building the OPeNDAP netCDF Data Handler, see the | |
INSTALL file. | |
This handler uses a configuration parameter, set in the bes.conf file, to | |
control where copies of some metadata objects built by the server are | |
cached. By default this cache is in /tmp - you are encouraged to change | |
that. Set the location using the 'HDF4.CacheDir' parameter. For example, | |
if you have set the BES.CacheDir parameter to /var/run/bes/cache you might | |
set HDF4.CacheDir to /var/run/bes/hdf4_cache. | |
A configuration edition helper script, `bes-hdf4-data.sh' is provided in | |
this package for easy configuration of the Hyrax BES server, designed to | |
edit bes.conf. The script is called using: | |
<code> | |
bes-hdf4-data.sh [<bes.conf file to modify> [<bes modules dir>]] | |
</code> | |
The `bes-conf' make target runs the script while trying to select paths | |
cleverly, and should be called using: | |
<code> | |
make bes-conf | |
</code> | |
Test data are also installed, so after installing this handler, Hyrax | |
will have data to serve providing an easy way to test your new | |
installation and to see how a working bes.conf should look. To use this, | |
make sure that you first install the bes, and that dap-server gets | |
installed too. Finally, every time you install or reinstall handlers, | |
make sure to restart the BES and OLFS. | |
This data handler is one component of the OPeNDAP DAP Server; the server | |
base software is designed to allow any number of handlers to be | |
configured easily. See the DAP Server README and INSTALL files for | |
information about configuration, including how to use this handler. | |
Copyright information: This software was originally written at JPL as | |
part of the DODS NASA ESIP Federation Cooperative Agreement Notice. The | |
original copyright described free use of the software 'for research | |
purposes' although it was not clear what exactly those were. In Spring | |
of 2003 we (OPeNDAP) sought clarification of that language and | |
JPL/CalTech asked us to change the copyright to the copyright text now | |
included in the code. | |
In Fall of 2005 we decided to release the software under the LGPL, based | |
on a previous discussion with personnel at JPL. | |
James Gallagher | |
Support for HDF data types: | |
<code> | |
HDF Version: This release of the server supports HDF4.2r1. It also | |
supports reading/parsing the HDF-EOS attribute | |
information which is then available to DAP clients. | |
SDS: This is mapped to a DAP2 Grid (if it has a dimension | |
scale) or Array (if it lacks a dim scale). | |
Raster image: This is read via the HDF 4.0 General Raster | |
interface and is mapped to Array. Each component of a | |
raster is mapped to a new dimension labeled | |
accordingly. For example, a 2-dimensional, 3-component | |
raster is mapped to an m x n x 3 Array. | |
Vdata: This is mapped to a Sequence, each element of | |
which is a Structure. Each subfield of the Vdata is | |
mapped to an element of the Structure. Thus a Vdata | |
with one field of order 3 would be mapped to a | |
Sequence of 1 Structure containing 3 base types. | |
Note: Even though these appear as Sequences, the data | |
handler does not support applying relational | |
constraints to them. You can use the array notation to | |
request a range of elements. | |
Attributes: HDF attributes on SDS, rasters are | |
straight-forwardly mapped to DAP attributes (HDF | |
doesn't yet support Vdata attributes). File | |
attributes (both SDS, raster) are mapped as attributes | |
of a DAP variable called "HDF_GLOBAL" (by analogy to | |
the way DAP handles netCDF global attributes, i.e., | |
attaching them to "NC_GLOBAL"). | |
Annotations: HDF file annotations mapped in the DAP to attribute | |
values of type "String" attached to the fake DAP | |
variable named "HDF_ANNOT". HDF annotations on | |
objects are currently not read by the server. | |
Vgroups: Vgroups are straight-forwardly mapped to | |
Structures. | |
</code> | |
Special characters in HDF identifiers: | |
A number of non-alphanumeric characters (e.g., space, #, +, -) used in HDF | |
identifiers are not allowed in the names of DAP objects, object components | |
or in URLs. The HDF4 data handler therefore deals internally with | |
translated versions of these identifiers. To translate the WWW convention | |
of escaping such characters by replacing them with "%" followed by the | |
hexadecimal value of their ASCII code is used. For example, "Raster Image | |
#1" becomes "Raster%20Image%20%231". These translations should be | |
transparent to users of the server (but they will be visible in the DDS, | |
DAS and in any applications which use a client that does not translate the | |
identifiers back to their original form). | |
Known problems: | |
Handling of floating point attributes: | |
Because the DAP software encodes attribute values as ASCII strings there | |
will be a loss of accuracy for floating point attributes. This loss of | |
accuracy is dependent on the version of the C++ I/O library used in | |
compiling/linking the software (i.e., the amount of floating point | |
precision preserved when outputting to ASCII is dependent on the library). | |
Typically it is very small (e.g., at least six decimal places are | |
preserved). | |
Handling of global attributes: | |
- The server will merge the separate global attributes for the SD, GR | |
interfaces with any file annotations into one set of global attributes. | |
These will then be available through any of the global attribute access | |
functions. | |
- If the client opens a constrained dataset (e.g., in SDstart), any global | |
attributes of the unconstrained dataset will not be accessible because the | |
constraint creates a "virtual dataset" which is a subset of the original | |
unconstrained dataset. | |
Todd Karakashian (Todd.K.Karakashian at jpl.nasa.gov) | |
Isaac Henry (ike at seanet.jpl.nasa.gov) | |
Jake Hamby (Jake.Hamby at jpl.nasa.gov) | |
NASA/JPL April 1998 | |