[GEOS-6895]: Improve NetCDF community module: NetCDF4 support, projec…

…tions

support output customization panel, CF support, dataPacking support

doc/en/user/source/community/netcdf-out/index.rst

@@ -15,13 +15,89 @@ Installing the GeoServer NetCDF Output format extension
 
 Getting a NetCDF output file
 ----------------------------
-Make sure to specify NetCDF as value of the format parameter within the getCoverage request.
+Make sure to specify application/x-netcdf as value of the format parameter within the getCoverage request using the proper constant.
 As an instance: 
 http://localhost:8080/geoserver/wcs?request=GetCoverage&service=WCS&version=2.0.1&coverageId=it.geosolutions__V&Format=application/x-netcdf...
 
+
 Current limitations
 -------------------
 
 * Only WGS84 output CRS is supported
 * Input coverages/slices should share the same bounding box (lon/lat coordinates are the same for the whole ND cube)
-* NetCDF output will be produced only when input coverages come from a StructuredGridCoverage2D reader (This will allows to query the GranuleSource to get the list of granules in order to setup dimensions slices for each sub-coverage)
+* NetCDF output will be produced only when input coverages come from a StructuredGridCoverage2D reader (This will allows to query the GranuleSource to get the list of granules in order to setup dimensions slices for each sub-coverage)
+
+
+Supporting NetCDF4-Classic output file
+--------------------------------------
+Starting with version 2.8 of GeoServer, NetCDF4-Classic output is supported in addition to NetCDF-3.
+NetCDF4-Classic leverages on the simpler data model of NetCDF-3 by supporting the HDF5-based storage capabilities of NetCDF-4. 
+See :ref:`Installing required NetCDF-4 Native libraries <nc4>` for more info on that.
+
+NetCDF Output customization
+---------------------------
+
+Global Settings configuration
++++++++++++++++++++++++++++++
+Starting with version 2.8 of GeoServer it is possible to define a few global settings for the NetCDF output format. A new section will be added to the *Global Settings* page.
+
+.. figure:: netcdfoutsettings.png
+   :align: center
+
+   *NetCDF Output Global settings section*
+
+From this panel, you may configure:
+
+* Data Packing (*NONE*, *BYTE*, *SHORT*, *INT*)
+* Variable attributes
+* NetCDF4-Classic output specific parameters (they will be taken into account only in case the format specified in the WCS 2.0 GetCoverage request is application/x-netcdf4).
+
+Layer configuration
+++++++++++++++++++++
+With version 2.8 of GeoServer it is also possible to add more customization to the layer in order to specify some properties of the NetCDF Output.
+You will notice an additional tab to the layer configuration.
+
+.. figure:: netcdfoutpanel.png
+   :align: center
+
+   *NetCDF Output settings panel*
+
+.. note:: This tab will be initialized with the parameters defined in the *Global Settings* page. 
+
+From this panel, you may configure the same parameters as for the global panel and also other 2 Layer-specific parameters:
+
+* Output variable name. 
+* variable's unit of measure.
+
+.. note:: This panel will be available for Raster data only.
+
+CF Standard names support
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Note that the output name can also be chosen from the list of CF Standard names.
+Check `CF standard names <http://cfconventions.org/standard-names.html>`_ page for more info on it.
+
+Once you click on the dropdown, you may choose from the set of available standard names.
+
+.. figure:: cfnames.png
+   :align: center
+
+   *NetCDF CF Standard names list*
+
+Note that once you specify the standard name, the unit will be automatically configured, using the canonical unit associated with that standard name.
+
+.. figure:: cfunit.png
+   :align: center
+
+   *NetCDF CF Standard names and canonical unit*
+
+The list of standard names is populated by taking the entries from a standard name table xml.
+At time of writing, a valid example is available `Here <http://cfconventions.org/Data/cf-standard-names/27/src/cf-standard-name-table.xml>`_
+
+You have three ways to provide it to GeoServer.
+
+#. Add a ``-DNETCDF_STANDARD_TABLE=/path/to/the/table/tablename.xml`` property to the startup script.
+#. Put that xml file within the ``NETCDF_DATA_DIR`` which is the folder where all NetCDF auxiliary files are located. (`More info <http://geoserver.geo-solutions.it/multidim/en/mosaic_config/netcdf_mosaic.html#customizing-netcdf-ancillary-files-location>`_)
+#. Put that xml file within the ``GEOSERVER_DATA_DIR``.
+
+.. note:: Note that for the 2nd and 3rd case, file name must be **cf-standard-name-table.xml**.

doc/en/user/source/community/netcdf-out/nc4.rst

@@ -0,0 +1,74 @@
+.. _nc4:
+
+Installing required NetCDF-4 Native libraries
+=============================================
+In order to write NetCDF-4 files, you must have the NetCDF-4 C library (version 4.3.1 or above) available on your system, along with all supporting libraries (HDF5, zlib, etc). 
+The following sections provide quick reference installation instructions.
+For more detailed info, please take a look at the `NetCDF-4 C Library Loading <http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/reference/netcdf4Clibrary.html>`_ page.
+
+
+Windows install
+---------------
+#. Download the latest NetCDF4 installer from the `NetCDF-C Windows Libraries <http://www.unidata.ucar.edu/software/netcdf/docs/winbin.html>`_ page.
+#. Install the executable
+#. Make sure to add the *bin* folder of the package you have extracted, to the ``PATH`` environment variable.
+
+Linux install
+-------------
+#. Download the latest required dependencies (SZIP, ZLIB, HDF5) from the `NetCDF-4 libraries section <ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4/>`_.
+
+   As an instance:
+     .. code-block:: bash
+    
+       wget ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4/hdf5-1.8.13.tar.gz
+       wget ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4/zlib-1.2.8.tar.gz
+
+#. Download the latest NetCDF-C source code from `here <https://github.com/Unidata/netcdf-c/releases/>`_.
+
+   As an instance:
+     .. code-block:: bash
+    
+       wget https://github.com/Unidata/netcdf-c/archive/v4.3.3.1.tar.gz
+       
+#. Build and install the required dependencies (The following instructions assume that you will install all NetCDF4 C libs on ``/work/libs/nc4libs``, as an instance).
+
+   #. ZLIB
+       .. code-block:: bash
+    
+         ./configure --prefix=/work/libs/nc4libs
+         
+         make check install
+   
+   #. HDF5
+       .. code-block:: bash
+       
+         ./configure --with-zlib=/work/libs/nc4libs --prefix=/work/libs/nc4libs --enable-threadsafe --with-pthread=/DIR/TO/PTHREAD
+         
+         make check install
+
+#. Build the NetCDF C Library.
+
+    .. code-block:: bash
+    
+       CPPFLAGS=-I/work/libs/nc4libs/include LDFLAGS=-L/work/libs/nc4libs/lib ./configure --prefix=/work/libs/nc4libs
+       
+       make check install
+
+#. Make sure to add the *lib* folder of the package you have extracted, to the ``PATH`` environment variable.
+
+GeoServer startup checks
+------------------------
+If everything has been properly configured, you may notice a similar log message during GeoServer startup:
+
+| ``NetCDF-4 C library loaded (jna_path='null', libname='netcdf').``
+| ``Netcdf nc_inq_libvers='4.3.1 of Jan 16 2014 15:04:00 $' isProtected=true``
+|
+
+In case the native libraries haven't been properly configured you will see a message like this, instead:
+
+``NetCDF-4 C library not present (jna_path='null', libname='netcdf').``
+
+
+Requesting a NetCDF4-Classic output file as WCS2.0 output
+=========================================================
+Specifying application/x-netcdf4 as Format, will return a NetCDF4-Classic output files, provided that the underlying libraries are available.

doc/en/user/source/community/netcdf/netcdf.rst

@@ -33,4 +33,61 @@ Configuring a NetCDF data store
    * - ``Enabled``
      -  
    * - ``URL``
-     - 
+     - 
+
+Supporting Custom NetCDF Coordinate Reference Systems
+-----------------------------------------------------
+Starting with GeoServer 2.8.x, NetCDF related modules (both NetCDF/GRIB store, imageMosaic store based on NetCDF/GRIB dataset and NetCDF output format) allow to support custom Coordinate Reference Systems and Projections.
+As reported in the `NetCDF CF documentation, Grid mappings section <http://cfconventions.org/Data/cf-conventions/cf-conventions-1.6/build/cf-conventions.html#appendix-grid-mappings>`_
+a NetCDF CF file may expose gridMapping attributes to describe the underlying projection. 
+
+The GeoTools NetCDF machinery will parse the attributes (if any) contained in the underlying NetCDF dataset to setup an OGC CoordinateReferenceSystem object.
+Once created, a CRS lookup will be made to identify a custom EPSG (if any) defined by the user to match that Projection.
+
+User defined NetCDF Coordinate Reference Systems with their custom EPSG need to be provided in :file:`user_projections\\netcdf.projections.properties` file inside your data directory (you have to create that file if missing).  
+
+A sample entry in that property file could look like this:
+
+      9718001=PROJCS["lambert_conformal_conic_1SP", GEOGCS["unknown", DATUM["unknown", SPHEROID["unknown", 6371229.0, 0.0]], PRIMEM["Greenwich", 0.0], UNIT["degree", 0.017453292519943295], AXIS["Geodetic longitude", EAST], AXIS["Geodetic latitude", NORTH]], PROJECTION["Lambert_Conformal_Conic_1SP"], PARAMETER["central_meridian", -95.0], PARAMETER["latitude_of_origin", 25.0], PARAMETER["scale_factor", 1.0], PARAMETER["false_easting", 0.0], PARAMETER["false_northing", 0.0], UNIT["m", 1.0], AXIS["Easting", EAST], AXIS["Northing", NORTH], AUTHORITY["EPSG","9718001"]]
+
+.. note:: Note the "unknown" names for GEOGCS, DATUM and SPHEROID elements. This is how the underlying NetCDF machinery will name custom elements.
+.. note:: Note the number that precedes the WKT.  This will determine the EPSG code.  So in this example, the EPSG code is 97180001.
+
+You may specify further custom NetCDF EPSG references by adding more lines to that file. 
+
+#. Insert the code WKT for the projection at the end of the file (on a single line or with backslash characters)::
+     
+      9718002=PROJCS["lambert_conformal_conic_2SP", \
+	    GEOGCS["unknown", \
+		  DATUM["unknown", \
+		    SPHEROID["unknown", 6377397.0, 299.15550239234693]], \
+	      PRIMEM["Greenwich", 0.0], \
+		  UNIT["degree", 0.017453292519943295], \
+		  AXIS["Geodetic longitude", EAST], \
+		  AXIS["Geodetic latitude", NORTH]], \
+		PROJECTION["Lambert_Conformal_Conic_2SP"], \
+		PARAMETER["central_meridian", 13.333333015441895], \
+		PARAMETER["latitude_of_origin", 46.0], \
+		PARAMETER["standard_parallel_1", 46.0], \
+		PARAMETER["standard_parallel_2", 49], \
+		PARAMETER["false_easting", 0.0], \
+		PARAMETER["false_northing", 0.0], 
+		UNIT["m", 1.0], \
+		AXIS["Easting", EAST], \
+		AXIS["Northing", NORTH], \
+		AUTHORITY["EPSG","97180002"]]
+
+#. Save the file.
+
+#. Restart GeoServer.
+
+#. Verify that the CRS has been properly parsed by navigating to the :ref:`srs_list` page in the :ref:`web_admin`.
+
+#. If the projection wasn't listed, examine the logs for any errors.
+
+Specify an external file through system properties
+--------------------------------------------------
+You may also specify the NetCDF projections definition file by setting a **Java system property** which links to the specified file.
+As an instance: :file:`-Dnetcdf.projections.file=/full/path/of/the/customfile.properties`
+
+

src/community/netcdf-out/pom.xml

@@ -25,6 +25,10 @@
     <groupId>org.geoserver</groupId>
     <artifactId>gs-main</artifactId>
   </dependency>
+  <dependency>
+    <groupId>org.geoserver.web</groupId>
+    <artifactId>gs-web-core</artifactId>
+  </dependency>
   <dependency>
     <groupId>org.geoserver</groupId>
     <artifactId>gs-main</artifactId>
@@ -40,6 +44,13 @@
     <artifactId>gs-ows</artifactId>
     <scope>test</scope>
   </dependency>
+  <dependency>
+    <groupId>org.geoserver.web</groupId>
+    <artifactId>gs-web-core</artifactId>
+    <classifier>tests</classifier>
+    <scope>test</scope>
+    <version>${project.version}</version>
+  </dependency>
   <dependency>
     <groupId>org.geoserver</groupId>
     <artifactId>gs-wcs1_1</artifactId>
@@ -57,6 +68,11 @@
     <groupId>org.geoserver</groupId>
     <artifactId>gs-wcs2_0</artifactId>
   </dependency>
+  <dependency>
+    <groupId>org.geoserver.community</groupId>
+    <artifactId>gs-netcdf</artifactId>
+	<version>${project.version}</version>
+  </dependency>
   <dependency>
     <groupId>org.geoserver</groupId>
     <artifactId>gs-wcs2_0</artifactId>
@@ -66,7 +82,6 @@
   <dependency>
     <groupId>org.geotools</groupId>
     <artifactId>gt-imagemosaic</artifactId>
-
     <scope>test</scope>
   </dependency>
 

src/community/netcdf-out/src/main/java/applicationContext.xml

@@ -10,4 +10,27 @@
     <bean id="netcdfCoverageResponseDelegate" class="org.geoserver.wcs.responses.NetCDFCoverageResponseDelegate">
         <constructor-arg ref="geoServer"/>
     </bean>
+    <bean id="netcdfSettingsPanel" class="org.geoserver.web.data.settings.SettingsPluginPanelInfo">
+        <description>This bean adds the necessary form fields to NetCDF Output Settings</description>
+        <property name="id" value="netcdfSettingsPanel"/>
+        <property name="titleKey" value="netcdfOutSettings"/>
+        <property name="componentClass" value="org.geoserver.web.netcdf.NetCDFOutSettingsPanel"/>
+        <property name="priority" value="100"/>
+    </bean>
+
+    <bean id="netcdfOutInitializer" class="org.geoserver.wcs.responses.NetCDFOutInitializer"/>
+
+    <bean id="netcdfOutXStreamInitializer" class="org.geoserver.web.netcdf.NetCDFContainerXStreamInitializer" />
+
+    <bean id="globalAttributeXStreamInitializer" class="org.geoserver.web.netcdf.GlobalAttributeXStreamInitializer" />
+
+    <bean id="netcdfParser" class="org.geoserver.web.netcdf.layer.NetCDFParserBean" />
+
+    <bean id="netcdfOutEditTabPanelInfo" class="org.geoserver.web.netcdf.layer.NetCDFOutEditTabPanelInfo">
+        <property name="id" value="netcdfOutEditTabPanelInfo"/>
+        <property name="titleKey" value="NetCDFOutTabPanel.title"/>
+        <property name="descriptionKey" value="NetCDFOutTabPanel.shortDescription"/>
+        <property name="componentClass" value="org.geoserver.web.netcdf.layer.NetCDFOutTabPanel"/>
+        <property name="order" value="400"/>
+    </bean>
 </beans>