[GEOS-8879] Adding a GetDomainValues to get paged unique values of a …

…given domain, in WMTS multidimensional community module

doc/en/user/source/community/wmts-multidimensional/index.rst

@@ -120,6 +120,8 @@ This module adds three new operations to the WMTS service that are described in
      - Description
    * - DescribeDomains
      - Describes all the dimension domains in a compact document, along with the restricted bounding box of the two dimensional space intercepted by the request.
+   * - GetDomainValues
+     - Allows to page through domain values (supplements DescribeDomains in case the domain has too many values, and the client still wants to get all of them, one page at a time)
    * - GetHistogram
      - Given a scattered domain description and an interval, this operation divides the interval in regular buckets, and provides an item count for each bucket.
    * - GetFeature
@@ -266,6 +268,157 @@ and the result will be similar to this:
     </SpaceDomain>
   </Domains>
 
+GetDomainValues
+^^^^^^^^^^^^^^^
+
+This operation is useful to page through the values of a given domain, in case the "multidimensional" area of interest
+is too large for DescribeDomain to return them in a single shot.
+
+.. list-table::
+   :widths: 20 10 70
+   :header-rows: 1
+
+   * - Name
+     - Mandatory
+     - Description
+   * - Service=WMTS
+     - Yes
+     - Service type identifier
+   * - Request=GetDomainValues
+     - Yes
+     - Operation name
+   * - Version=1.0.0
+     - Yes
+     - Standard and schema version for this operation
+   * - Layer
+     - Yes
+     - Layer identifier
+   * - bbox=minx,miny,maxx,maxy
+     - No
+     - Bounding box corners (lower left, upper right) in CRS units
+   * - DimensionIdentifier
+     - No
+     - At most one per dimension, a range described as min/max, restricting the domain of this dimension
+   * - Domain
+     - Yes
+     - Name of the domain whose values will be returned (one cannot use "bbox", only single value dimensions can be enumerated by GetDomainValues, e.g., time, elevation).
+   * - FromValue
+     - No
+     - Sets the beginning of domain enumeration, for paging purposes. It's not included in the result
+   * - Sort
+     - No
+     - Can be "asc" or "desc", determines if the enumeration is from low to high, or from high to low
+   * - Limit
+     - No
+     - Maximum number of values returned by this call. The server assumes a built-in limit of 1000 in case not specified,
+       and allows client to specify a value up to 10000.
+
+For example, let's say a "elevation" domain has values 1,2,3 and 5, and that we are paging through
+it by pages of 2 elements. The client will start without providing a "fromValue", and will then continue
+using the last value of the previous page as a reference:
+
+.. code-block:: guess
+
+  http://localhost:8080/geoserver/gwc/service/wmts?request=GetDomainValues&Version=1.0.0&Layer=sampleLayer&domain=elevation&limit=2
+
+.. code-block:: xml
+
+    <DomainValues xmlns="http://demo.geo-solutions.it/share/wmts-multidim/wmts_multi_dimensional.xsd" xmlns:ows="http://www.opengis.net/ows/1.1">
+      <ows:Identifier>elevation</ows:Identifier>
+      <Limit>2</Limit>
+      <Sort>asc</Sort>
+      <Domain>1.0,2.0</Domain>
+      <Size>2</Size>
+    </DomainValues>
+
+.. code-block:: guess
+
+  http://localhost:8080/geoserver/gwc/service/wmts?request=GetDomainValues&Version=1.0.0&Layer=sampleLayer&domain=elevation&limit=2&fromValue=2
+          
+.. code-block:: xml
+
+    <DomainValues xmlns="http://demo.geo-solutions.it/share/wmts-multidim/wmts_multi_dimensional.xsd" xmlns:ows="http://www.opengis.net/ows/1.1">
+      <ows:Identifier>elevation</ows:Identifier>
+      <Limit>2</Limit>
+      <Sort>asc</Sort>
+      <FromValue>2.0</FromValue>
+      <Domain>3.0,5.0</Domain>
+      <Size>2</Size>
+    </DomainValues>
+
+.. code-block:: guess
+
+  http://localhost:8080/geoserver/gwc/service/wmts?request=GetDomainValues&Version=1.0.0&Layer=sampleLayer&domain=elevation&limit=2&fromValue=5
+          
+.. code-block:: xml
+
+    <DomainValues xmlns="http://demo.geo-solutions.it/share/wmts-multidim/wmts_multi_dimensional.xsd" xmlns:ows="http://www.opengis.net/ows/1.1">
+      <ows:Identifier>elevation</ows:Identifier>
+      <Limit>2</Limit>
+      <Sort>asc</Sort>
+      <FromValue>5.0</FromValue>
+      <Domain></Domain>
+      <Size>0</Size>
+    </DomainValues>
+
+For elevations it might not be uncommon to iterate backwards, from the top-most elevation down to the lowest value. The interaction
+between client and server migth then look as follows:
+
+.. code-block:: guess
+
+  http://localhost:8080/geoserver/gwc/service/wmts?request=GetDomainValues&Version=1.0.0&Layer=sampleLayer&domain=elevation&limit=2&sort=desc
+
+.. code-block:: xml
+
+    <DomainValues xmlns="http://demo.geo-solutions.it/share/wmts-multidim/wmts_multi_dimensional.xsd" xmlns:ows="http://www.opengis.net/ows/1.1">
+      <ows:Identifier>elevation</ows:Identifier>
+      <Limit>2</Limit>
+      <Sort>asc</Sort>
+      <Domain>5.0,3.0</Domain>
+      <Size>2</Size>
+    </DomainValues>
+
+.. code-block:: guess
+
+  http://localhost:8080/geoserver/gwc/service/wmts?request=GetDomainValues&Version=1.0.0&Layer=sampleLayer&domain=elevation&limit=2&fromValue=3&sort=desc
+          
+.. code-block:: xml
+
+    <DomainValues xmlns="http://demo.geo-solutions.it/share/wmts-multidim/wmts_multi_dimensional.xsd" xmlns:ows="http://www.opengis.net/ows/1.1">
+      <ows:Identifier>elevation</ows:Identifier>
+      <Limit>2</Limit>
+      <Sort>asc</Sort>
+      <FromValue>3.0</FromValue>
+      <Domain>2.0,1.0</Domain>
+      <Size>2</Size>
+    </DomainValues>
+
+.. code-block:: guess
+
+  http://localhost:8080/geoserver/gwc/service/wmts?request=GetDomainValues&Version=1.0.0&Layer=sampleLayer&domain=elevation&limit=2&fromValue=1&sort=desc
+          
+.. code-block:: xml
+
+    <DomainValues xmlns="http://demo.geo-solutions.it/share/wmts-multidim/wmts_multi_dimensional.xsd" xmlns:ows="http://www.opengis.net/ows/1.1">
+      <ows:Identifier>elevation</ows:Identifier>
+      <Limit>2</Limit>
+      <Sort>asc</Sort>
+      <FromValue>1.0</FromValue>
+      <Domain></Domain>
+      <Size>0</Size>
+    </DomainValues>
+
+The paging approach might seem odd for those used to using "limit" and "offset". The main reason it's done
+this way it's performance, paging through unique values via limit and offset means that the data source
+has to compute and collect the unique values that are not needed (the ones in previous pages) in order to
+find the ones in the current page. With large domains (typical of time series) this quickly becomes too
+slow for interactive usage, as one moves forward in the domain.
+
+By giving a starting point, the unneeded data points can be skipped via index and the distinct value
+computation can be performed only on the current page data, stopping it as soon as the desired number
+of results has been computed. With an index on the dimension being queries, this results in nearly
+constant response times, regardless of the page being requested.
+
 GetHistogram
 ^^^^^^^^^^^^
 

src/community/wmts-multi-dimensional/src/main/java/org/geoserver/gwc/wmts/DescribeDomainsTransformer.java

@@ -9,6 +9,7 @@
 import java.util.Map;
 import java.util.stream.Collectors;
 import org.geoserver.wms.WMS;
+import org.geotools.data.Query;
 import org.geotools.geometry.jts.ReferencedEnvelope;
 import org.geotools.referencing.CRS;
 import org.geotools.xml.transform.TransformerBase;
@@ -58,7 +59,8 @@ public void encode(Object object) throws IllegalArgumentException {
                             dimension -> {
                                 Tuple<Integer, List<String>> dimensionValues =
                                         dimension.getDomainValuesAsStrings(
-                                                domains.getFilter(), domains.getExpandLimit());
+                                                new Query(null, domains.getFilter()),
+                                                domains.getExpandLimit());
                                 domainsValues.put(dimension.getDimensionName(), dimensionValues);
                             });
             if (domains.getSpatialDomain() != null && !domains.getSpatialDomain().isEmpty()) {

src/community/wmts-multi-dimensional/src/main/java/org/geoserver/gwc/wmts/Domains.java

@@ -18,6 +18,7 @@
 import org.geotools.feature.FeatureCollection;
 import org.geotools.geometry.jts.ReferencedEnvelope;
 import org.opengis.filter.Filter;
+import org.opengis.filter.sort.SortOrder;
 
 /**
  * Utility class for aggregating several dimensions. All the dimensions will share the same spatial
@@ -30,22 +31,70 @@ public class Domains {
     private final Filter filter;
 
     private final LayerInfo layerInfo;
-    private final int expandLimit;
+    private int expandLimit;
+    private int maxReturnedValues;
+    private SortOrder sortOrder;
 
     private String histogram;
     private String resolution;
+    private String fromValue;
 
     public Domains(
             List<Dimension> dimensions,
             LayerInfo layerInfo,
             ReferencedEnvelope boundingBox,
-            Filter filter,
-            int expandLimit) {
+            Filter filter) {
         this.dimensions = dimensions;
         this.layerInfo = layerInfo;
         this.spatialDomain = boundingBox;
         this.filter = filter;
+    }
+
+    public Domains withExpandLimit(int expandLimit) {
         this.expandLimit = expandLimit;
+        return this;
+    }
+
+    public Domains withMaxReturnedValues(int maxReturnedValues) {
+        this.maxReturnedValues = maxReturnedValues;
+        return this;
+    }
+
+    public Domains withSortOrder(SortOrder sortOrder) {
+        this.sortOrder = sortOrder;
+        return this;
+    }
+
+    public Domains withFromValue(String fromValue) {
+        this.fromValue = fromValue;
+        return this;
+    }
+
+    /**
+     * The maximum number of returned values in a GetDomainValues request
+     *
+     * @return
+     */
+    public int getMaxReturnedValues() {
+        return maxReturnedValues;
+    }
+
+    /**
+     * Returns the "fromValue" parameter in a GetDomainValues request
+     *
+     * @return
+     */
+    public String getFromValue() {
+        return fromValue;
+    }
+
+    /**
+     * The sort direction in a GetDomainValues request
+     *
+     * @return
+     */
+    public SortOrder getSortOrder() {
+        return sortOrder;
     }
 
     public List<Dimension> getDimensions() {

src/community/wmts-multi-dimensional/src/main/java/org/geoserver/gwc/wmts/GetDomainValuesTransformer.java

@@ -0,0 +1,71 @@
+/* (c) 2016 Open Source Geospatial Foundation - all rights reserved
+ * This code is licensed under the GPL 2.0 license, available at the root
+ * application directory.
+ */
+package org.geoserver.gwc.wmts;
+
+import java.util.List;
+import java.util.stream.Collectors;
+import org.geoserver.gwc.wmts.dimensions.Dimension;
+import org.geoserver.wms.WMS;
+import org.geotools.data.Query;
+import org.geotools.xml.transform.TransformerBase;
+import org.geotools.xml.transform.Translator;
+import org.opengis.filter.sort.SortOrder;
+import org.xml.sax.Attributes;
+import org.xml.sax.ContentHandler;
+
+/** XML transformer for the describe domains operation. */
+class GetDomainValuesTransformer extends TransformerBase {
+
+    public GetDomainValuesTransformer(WMS wms) {
+        setIndentation(2);
+        setEncoding(wms.getCharSet());
+    }
+
+    @Override
+    public Translator createTranslator(ContentHandler handler) {
+        return new TranslatorSupport(handler);
+    }
+
+    class TranslatorSupport extends TransformerBase.TranslatorSupport {
+
+        public TranslatorSupport(ContentHandler handler) {
+            super(handler, null, null);
+        }
+
+        @Override
+        public void encode(Object object) throws IllegalArgumentException {
+            if (!(object instanceof Domains)) {
+                throw new IllegalArgumentException(
+                        "Expected domains info but instead got: "
+                                + object.getClass().getCanonicalName());
+            }
+            Domains domains = (Domains) object;
+            Attributes nameSpaces =
+                    createAttributes(
+                            new String[] {
+                                "xmlns",
+                                        "http://demo.geo-solutions.it/share/wmts-multidim/wmts_multi_dimensional.xsd",
+                                "xmlns:ows", "http://www.opengis.net/ows/1.1"
+                            });
+            start("DomainValues", nameSpaces);
+            Dimension dimension = domains.getDimensions().get(0);
+            element("ows:Identifier", dimension.getDimensionName());
+            element("Limit", String.valueOf(domains.getMaxReturnedValues()));
+            element("Sort", domains.getSortOrder() == SortOrder.ASCENDING ? "asc" : "desc");
+            if (domains.getFromValue() != null) {
+                element("FromValue", domains.getFromValue());
+            }
+            Query query = new Query(null, domains.getFilter());
+            List<String> values =
+                    dimension.getPagedDomainValuesAsStrings(
+                                    query, domains.getMaxReturnedValues(), domains.getSortOrder())
+                            .second;
+            element("Domain", values.stream().collect(Collectors.joining(",")));
+            element("Size", String.valueOf(values.size()));
+
+            end("DomainValues");
+        }
+    }
+}