CSW Module : Internal Catalog - ISO MetaData Profile

doc/en/user/source/community/csw/index.rst

@@ -0,0 +1,73 @@
+Catalogue Services For The Web
+==============================
+
+GeoServer supports retrieving MetaData information from the GeoServer Catalog using CSW (Catalogue Services for the Web). 
+More specifically, the following standard CSW Services are currently being supported:
+ * GetRecords
+ * GetRecordById
+ * GetDomain
+ * DescribeRecord
+ * GetCapabilities
+
+For more information on CSW services, we refer to OGC OpenGIS Implementation Specification 07-006r1 (see http://www.opengeospatial.org/standards/specifications/catalog) and http://www.ogcnetwork.net/node/630.
+
+Selecting Catalog Store
+-----------------------
+
+In order to direct GeoServer to use the right Catalog Store, we must set the Java system property 'DefaultCatalogStore'. To use the Internal Catalog Store, i.e. the catalog store that connects to the internal catalog of GeoServer, we must set this property to "org.geoserver.csw.store.internal.GeoServerInternalCatalogStore". Alternatively, there is also a SimpleCatalogStore which reads the Catalog Data directly from files (this is mainly used for testing).
+
+Currently the Internal Catalog Store supports two MetaData schemes: Dublin Core and ISO MetaData Profile.
+
+Mapping Files
+------------
+
+The mapping files are located in the 'csw' directory inside the GeoServer Data Directory.
+Every mapping file must have the exact name of the record type name. For example, 
+ * the Dublin Core mapping can be found in the file 'csw/Record' inside the Data Directory.
+ * the ISO MetaData mapping can be found in the file 'csw/MD_Metadata' inside the Data Directory.
+
+The mapping files take the syntax from Java properties files. On the left of the = sign, we specify the target field name or path in the MetaData record, paths beings separated with dots. On the right of the = sign, we can specify any CQL expression that specifies the value of the target property. The CQL expression is applied to each 'ResourceInfo' object in the catalog and can retrieve all properties from this object. These expressions can make use of literals, properties present in the 'ResourceInfo' object, and all normal CQL operators and functions. We will discuss some example mapping files below.
+
+Furthermore, we can place a @ symbol in front of the field we also wish to use as identifier for each MetaData record (this may be useful for ID filters).  We use a $ sign in front of fields that are required to make sure the mapping is aware of the requirement (in particular when selecting properties).
+
+Dublic Core Example
+-------------------
+
+Mapping File::
+
+ @identifier.value=id
+ title.value=title
+ creator.value='GeoServer Catalog'
+ subject.value=keywords
+ subject.scheme='http://www.digest.org/2.1'
+ abstract.value=abstract
+ description.value=strConcat('description about ' , title)
+ date.value="metadata.csw.date"
+ type.value='http://purl.org/dc/dcmitype/Dataset'
+ publisher.value='Niels Charlier'
+
+All fields have the form of <fieldname>.value (for the actual value in the field) and, additionally also <fieldname>.scheme can be specified (for the @scheme attribute) of this field.
+
+Examples of attributes extracted from the resource are “id”, “title”, “keywords”, etc... The attribute “metadata.csw.date” uses the MetaData (java.util.)Map from the ResourceInfo object. In this map, it searches for the keyword “csw”, which in return also returns a (java.util).Map from which the value for the keyword “date” is extracted. Note that the double quotes are necessary to preserve the dots to separate the property path.
+
+ISO MetaData Profile Example
+----------------------------
+
+For more information on the ISO MetaData standard, we refer to OGC Implementation Specification 07-045 (see http://www.opengeospatial.org/standards/specifications/catalog). Mapping File::
+
+ @fileIdentifier.CharacterString=id
+ identificationInfo.AbstractMD_Identification.citation.CI_Citation.title.CharacterString=title
+ identificationInfo.AbstractMD_Identification.descriptiveKeywords.MD_Keywords.keyword.CharacterString=keywords
+ identificationInfo.AbstractMD_Identification.abstract.CharacterString=abstract
+ $dateStamp.Date= if_then_else ( isNull("metadata.csw.date") , 'Unknown', "metadata.csw.date")
+ hierarchyLevel.MD_ScopeCode.@codeListValue='http://purl.org/dc/dcmitype/Dataset'
+ $contact.CI_ResponsibleParty.individualName.CharacterString='Niels Charlier'
+
+The full path of each field must be specified (separated with dots). Xml attributes are here specified with a @ symbol, as in the normal XML and X-path notation.
+
+Note that “dateStamp.Date” and ”contact.CI_ResponsibleParty.individualName.CharacterString” must be preceded by a $ sign to make sure that it is always included, even when using property selection (to keep the result XSD compliant).
+
+Filtering
+---------
+
+The Internal Catalog Store supports filtering on both full x-paths as well as the 'Queryables' specified in getCapabilities (see the standards).

doc/en/user/source/community/index.rst

@@ -26,3 +26,4 @@ officially part of the GeoServer releases. They are however built along with the
    scripting/index
    spatialite/index
    xslt/index
+   csw/index

src/community/csw/csw-api/pom.xml

@@ -44,9 +44,24 @@
         <version>${gt.version}</version>
     </dependency>
     <dependency>
-        <groupId>org.geotools</groupId>
-        <artifactId>gt-app-schema</artifactId>
-        <version>${gt.version}</version>
-    </dependency>
+	<groupId>org.geotools</groupId>
+	<artifactId>gt-complex</artifactId>
+	<version>${gt.version}</version>
+     </dependency>
+     <dependency>
+	  <groupId>org.geotools.schemas</groupId>
+	  <artifactId>ows-1.0</artifactId>
+	  <version>1.0.0-2</version>
+     </dependency>
+     <dependency>
+	  <groupId>org.geotools.schemas</groupId>
+	  <artifactId>filter-1.1</artifactId>
+	  <version>1.1.1-2</version>
+     </dependency>
+     <dependency>
+	  <groupId>org.geotools.schemas</groupId>
+	  <artifactId>gml-3.2</artifactId>
+	  <version>3.2.1-1</version>
+     </dependency>
   </dependencies>
 </project>

src/community/csw/csw-api/src/main/java/org/geoserver/csw/feature/AbstractFeatureCollection.java

@@ -4,13 +4,10 @@
  */
 package org.geoserver.csw.feature;
 
-import java.util.ArrayList;
 import java.util.Collection;
 import java.util.HashSet;
 import java.util.Iterator;
-import java.util.List;
 import java.util.Set;
-
 import org.geotools.feature.FeatureCollection;
 import org.geotools.feature.FeatureIterator;
 import org.geotools.feature.collection.DelegateFeatureIterator;
@@ -30,6 +27,8 @@
  */
 public abstract class AbstractFeatureCollection<T extends FeatureType, F extends Feature> implements
         FeatureCollection<T, F> {
+
+
     /**
      * id used when serialized to gml
      */
@@ -47,7 +46,7 @@ protected AbstractFeatureCollection(T memberType) {
     //
     @SuppressWarnings("unchecked")
     public FeatureIterator<F> features() {
-        FeatureIterator iter = new DelegateFeatureIterator(this, openIterator());
+        FeatureIterator iter = new DelegateFeatureIterator(openIterator());
         getOpenIterators().add(iter);
         return iter;
     }
@@ -286,10 +285,10 @@ public Object[] toArray() {
     }
 
     @SuppressWarnings("unchecked")
-    public <T> T[] toArray(T[] a) {
+    public <T2> T2[] toArray(T2[] a) {
         int size = size();
         if (a.length < size) {
-            a = (T[]) java.lang.reflect.Array.newInstance(a.getClass().getComponentType(), size);
+            a = (T2[]) java.lang.reflect.Array.newInstance(a.getClass().getComponentType(), size);
         }
         Iterator<F> it = iterator();
         try {

src/community/csw/csw-api/src/main/java/org/geoserver/csw/feature/MemoryFeatureCollection.java

@@ -14,7 +14,6 @@
 
 import org.geoserver.csw.feature.sort.ComplexComparatorFactory;
 import org.geotools.feature.FeatureCollection;
-import org.geotools.feature.FeatureIterator;
 import org.opengis.feature.Feature;
 import org.opengis.feature.type.FeatureType;
 import org.opengis.filter.Filter;

src/community/csw/csw-api/src/main/java/org/geoserver/csw/records/AbstractRecordDescriptor.java

@@ -0,0 +1,103 @@
+/* Copyright (c) 2012 TOPP - www.openplans.org. All rights reserved.
+ * This code is licensed under the GPL 2.0 license, available at the root
+ * application directory.
+ */
+package org.geoserver.csw.records;
+
+import java.util.LinkedList;
+import java.util.List;
+
+import org.geotools.factory.CommonFactoryFinder;
+import org.geotools.feature.NameImpl;
+import org.geotools.referencing.CRS;
+import org.opengis.feature.type.FeatureType;
+import org.opengis.feature.type.Name;
+import org.opengis.filter.FilterFactory2;
+import org.opengis.filter.expression.PropertyName;
+import org.opengis.referencing.crs.CoordinateReferenceSystem;
+import org.xml.sax.helpers.NamespaceSupport;
+
+/**
+ * Describes a record, its schema, its possible representations, in a pluggable way
+ * The Abstract class provides some default behaviour.
+ * 
+ * @author Niels Charlier
+ */
+public abstract class AbstractRecordDescriptor implements RecordDescriptor {
+
+    public static final FilterFactory2 ff = CommonFactoryFinder.getFilterFactory2();
+
+    public static final String DEFAULT_CRS_NAME = "urn:x-ogc:def:crs:EPSG:6.11:4326";
+
+    public static final CoordinateReferenceSystem DEFAULT_CRS;
+
+    static {
+        // build the default CRS
+        try {
+            DEFAULT_CRS = CRS.decode(DEFAULT_CRS_NAME);
+        } catch(Exception e) {
+            throw new RuntimeException("Failed to decode the default CRS, this should never happen!", e);
+        }
+    }
+
+    /**
+     * The GeoTools feature type representing this kind of record
+     * The default method retrieves type from the descriptor
+     * 
+     * @return the feature type
+     */
+    public FeatureType getFeatureType() {
+        return (FeatureType) getFeatureDescriptor().getType();
+    }   
+
+    /**
+     * Helper method to create a list of names from namespace support and a sequence of strings
+     * 
+     * @param ns Namespace Support
+     * @param names Sequence of name strings
+     * @return the List of Names
+     */
+    protected static List<Name> createNameList(NamespaceSupport ns, String... names) {
+        List<Name> result = new LinkedList<Name>();
+        for (String name : names) {
+            String[] splitted = name.split(":");
+            String uri, localName;
+            if (splitted.length == 1) {
+                uri = ns.getURI("");
+                localName = splitted[0];                
+            } else {
+                uri = ns.getURI(splitted[0]);
+                localName = splitted[1];                    
+            }
+            result.add(new NameImpl(uri, localName));
+        }
+
+        return result;
+    }
+
+    /**
+     * Helper method to build a property name from a simple name (not an x-path) with namespace support.
+     *   
+     * @param namespaces Namespace support
+     * @param name the Name
+     * @return the PropertyName
+     */
+    public static PropertyName buildPropertyName( NamespaceSupport namespaces, Name name) {
+        String ns = name.getNamespaceURI();
+        String localName = name.getLocalPart();
+
+        String prefix = namespaces.getPrefix(ns);
+        // build the xpath with the prefix, or not if we don't have one
+        String xpath;
+        if(prefix != null && !"".equals(prefix)) {
+            xpath = prefix + ":" + localName;
+        } else {
+            xpath = localName;
+        }
+
+        PropertyName property = ff.property(xpath, namespaces);
+        return property;
+    }
+
+
+}

src/community/csw/csw-api/src/main/java/org/geoserver/csw/records/CSWRecordBuilder.java

@@ -4,108 +4,37 @@
  */
 package org.geoserver.csw.records;
 
-import java.util.ArrayList;
-import java.util.List;
-
-import org.geotools.feature.AttributeBuilder;
-import org.geotools.feature.ComplexFeatureBuilder;
-import org.geotools.feature.LenientFeatureFactoryImpl;
-import org.geotools.geometry.jts.JTS;
-import org.geotools.geometry.jts.ReferencedEnvelope;
-import org.geotools.referencing.crs.DefaultGeographicCRS;
-import org.opengis.feature.Attribute;
-import org.opengis.feature.Feature;
-import org.opengis.feature.type.AttributeDescriptor;
-
-import com.vividsolutions.jts.geom.Geometry;
-import com.vividsolutions.jts.geom.Polygon;
-
 /**
  * A helper that builds CSW Dublin core records as GeoTools features
  * 
  * @author Andrea Aime - GeoSolutions
  */
-public class CSWRecordBuilder {
-
-    ComplexFeatureBuilder fb = new ComplexFeatureBuilder(CSWRecordDescriptor.RECORD);
-
-    AttributeBuilder ab = new AttributeBuilder(new LenientFeatureFactoryImpl());
-
-    List<ReferencedEnvelope> boxes = new ArrayList<ReferencedEnvelope>();
+public class CSWRecordBuilder extends GenericRecordBuilder{
 
+    public CSWRecordBuilder() {
+        super(CSWRecordDescriptor.getInstance());
+    }
+
     /**
      * Adds an element to the current record
      * 
      * @param name
      * @param values
      */
     public void addElement(String name, String... values) {
-        for (String value : values) {
-            AttributeDescriptor descriptor = CSWRecordDescriptor.getDescriptor(name);
-            ab.setDescriptor(descriptor);
-            ab.add(null, value, CSWRecordDescriptor.SIMPLE_LITERAL_VALUE);
-            Attribute element = ab.build();
-
-            fb.append(CSWRecordDescriptor.DC_ELEMENT_NAME, element);
-        }
-    }
-
-    public void addElementWithScheme(String name, String scheme, String value) {
-        AttributeDescriptor descriptor = CSWRecordDescriptor.getDescriptor(name);
-        ab.setDescriptor(descriptor);
-        ab.add(null, value, CSWRecordDescriptor.SIMPLE_LITERAL_VALUE);
-        ab.add(null, scheme, CSWRecordDescriptor.SIMPLE_LITERAL_SCHEME);
-        Attribute element = ab.build();
-
-        fb.append(CSWRecordDescriptor.DC_ELEMENT_NAME, element);
+        super.addElement(name + ".value", values);
     }
 
     /**
-     * Adds a bounding box to the record. The envelope must be in WGS84
+     * Adds an element to the current record with scheme
      * 
-     * @param env
-     */
-    public void addBoundingBox(ReferencedEnvelope env) {
-        boxes.add(env);
-    }
-
-    /**
-     * Builds a record and sets up to work on the next one
-     * 
-     * @param id
-     * @return
+     * @param name
+     * @param scheme
+     * @param values
      */
-    public Feature build(String id) {
-        // gather all the bounding boxes in a single geometry
-        Geometry geom = null;
-        for (ReferencedEnvelope env : boxes) {
-            try {
-                env = env.transform(CSWRecordDescriptor.DEFAULT_CRS, true);
-
-                Polygon poly = JTS.toGeometry(env);
-                poly.setUserData(CSWRecordDescriptor.DEFAULT_CRS);
-                if (geom == null) {
-                    geom = poly;
-                } else {
-                    geom = geom.union(poly);
-                }
-            } catch (Exception e) {
-                throw new IllegalArgumentException(
-                        "Failed to reproject one of the bounding boxes to WGS84, "
-                                + "this should never happen with valid coordinates", e);
-            }
-        }
-        if (geom instanceof Polygon) {
-            geom = geom.getFactory().createMultiPolygon(new Polygon[] { (Polygon) geom });
-        }
-
-        ab.setDescriptor(CSWRecordDescriptor.RECORD_BBOX_DESCRIPTOR);
-        Attribute element = ab.buildSimple(null, geom);
-        element.getUserData().put(CSWRecordDescriptor.ORIGINAL_BBOXES, new ArrayList<ReferencedEnvelope>(boxes));
-        fb.append(CSWRecordDescriptor.RECORD_BBOX_NAME, element);
-
-        boxes.clear();
-        return fb.buildFeature(id);
+    public void addElementWithScheme(String name, String scheme, String value) {
+        super.addElement(name + ".value", value);
+        super.addElement(name + ".scheme", scheme);
     }
 
 }