[GEOS-8223] Allow ordering layer contents in WMS using a "sortBy" ven…

…dor parameter, as well as mosaic contents in WCS

doc/en/user/source/services/wcs/vendor.rst

@@ -3,6 +3,9 @@
 WCS Vendor Parameters
 =====================
 
+namespace
+---------
+
 Requests to the WCS GetCapabilities operation can be filtered to only return layers corresponding to a particular namespace.
 
 Sample code:
@@ -15,4 +18,31 @@ Sample code:
       request=GetCapabilities&
       namespace=topp
 
-Using an invalid namespace prefix will not cause any errors, but the document returned will not contain information on any layers.
+Using an invalid namespace prefix will not cause any errors, but the document returned will not contain information on any layers.
+
+cql_filter
+----------
+
+The ``cql_filter`` parameter is similar to same named WMS parameter, and allows expressing a filter using ECQL (Extended Common Query Language).
+The filter is sent down into readers exposing a ``Filter`` read parameter.
+
+For example, assume a image mosaic has a tile index with a ``cloudCover`` percentage attribute, then it's possible to mosaic only
+granules with a cloud cover less than 10% using:
+
+   cql_filter=cloudCover < 10
+
+For full details see the :ref:`filter_ecql_reference` and :ref:`cql_tutorial` tutorial.
+
+sortBy
+------
+
+The ``sortBy`` parameter allows to control the order of granules being mosaicked, using the same
+syntax as WFS 1.0, that is:
+
+* ``&sortBy=att1 A|D,att2 A|D, ...``
+
+This maps to a "SORTING" read parameter that the coverage reader might expose (image mosaic exposes such parameter).
+
+In image mosaic, this causes the first granule found in the sorting will display on top, and then the others will follow.
+
+Thus, to sort a scattered mosaic of satellite images so that the most recent image shows on top, and assuming the time attribute is called ``ingestion`` in the mosaic index, the specification will be ``&sortBy=ingestion D``.

doc/en/user/source/services/wms/vendor.rst

@@ -69,6 +69,27 @@ An example of a simple CQL filter is::
    cql_filter=INTERSECTS(the_geom,%20POINT%20(-74.817265%2040.5296504))
    
 
+sortBy
+------
+
+The ``sortBy`` parameter allows to control the order of features/rasters displayed in the map, using the same
+syntax as WFS 1.0, that is:
+
+* ``&sortBy=att1 A|D,att2 A|D, ...`` for a single layer request
+* ``&sortBy=(att1Layer1 A|D,att2Layer1 A|D, ...)(att1Layer2 A|D,att2Layer2 A|D, ...)...`` when requesting multiple layers
+
+Care should be taken when using it as it has different behavior for raster layers, vector layers, and layer groups.
+In particular:
+
+* | For **raster layers**, ``sortBy`` maps to a "SORTING" read parameter that the reader might expose (image mosaic exposes such parameter).
+  | In image mosaic, this causes the first granule found in the sorting will display on top, and then the others will follow.
+  | Thus, to sort a scattered mosaic of satellite images so that the most recent image shows on top, and assuming the time attribute is called ``ingestion`` in the mosaic index, the specification will be ``&sortBy=ingestion D``.
+* | For **vector layers**, ``sortBy`` maps to a sort by clause in the vector data source, and then painting happens using the normal "painter model" rules, so the first item returned is painted first, and then all others on top of it.
+  | Thus, to sort a set of event points so that the most recent event is painted on top, and assuming the attribute is called "date" in the vector layer, the specification will be ``&sortBy=date`` or ``&sortBy=date A`` (ascending direction being the default one).
+* | For **layer groups**, the sort specification is going to be copied over all internal layers, so the spec has to be valid for all of them, or an error will be reported. 
+  | An empty spec can be used for layer groups in this case, for example, ``layers=theGroup,theLayer&sortBy=(),(date A)``
+ 
+
 env
 ---
 

src/gwc/src/main/java/org/geoserver/gwc/GWC.java

@@ -910,6 +910,11 @@ boolean isCachingPossible(TileLayer layer, GetMapRequest request,
                 return false;
             }
         }
+        if (null != request.getSortBy() && !request.getSortBy().isEmpty()) {
+            if (!filterApplies(filters, request, "SORTBY", requestMistmatchTarget)) {
+                return false;
+            }
+        }
         if (null != request.getPalette()) {
             if (!filterApplies(filters, request, "PALETTE", requestMistmatchTarget)) {
                 return false;

src/kml/src/main/java/org/geoserver/kml/builder/SuperOverlayNetworkLinkBuilder.java

@@ -30,6 +30,7 @@
 import org.geotools.referencing.crs.DefaultGeographicCRS;
 import org.geotools.styling.Style;
 import org.opengis.filter.Filter;
+import org.opengis.filter.sort.SortBy;
 import org.opengis.referencing.crs.CoordinateReferenceSystem;
 
 import com.vividsolutions.jts.geom.Envelope;
@@ -455,6 +456,12 @@ private boolean isRequestGWCCompatible(GetMapRequest request, int layerIndex, WM
         if (filters != null && filters.size() > 0 && filters.get(layerIndex) != Filter.INCLUDE) {
             return false;
         }
+
+        // no extra sorts
+        List<List<SortBy>> sortBy = request.getSortBy();
+        if (sortBy != null && sortBy.size() > 0) {
+            return false;
+        }
 
         // no fiddling with antialiasing settings
         String antialias = (String) request.getFormatOptions().get("antialias");

src/wfs/src/main/java/org/geoserver/wfs/kvp/SortByKvpParser.java → src/main/src/main/java/org/geoserver/ows/kvp/SortByKvpParser.java

@@ -3,7 +3,7 @@
  * This code is licensed under the GPL 2.0 license, available at the root
  * application directory.
  */
-package org.geoserver.wfs.kvp;
+package org.geoserver.ows.kvp;
 
 import org.geoserver.ows.NestedKvpParser;
 import org.opengis.filter.FilterFactory;
@@ -13,7 +13,8 @@
 
 /**
  * Parses kvp of the form 'sortBy=Field1 {A|D},Field2 {A|D}...' into a
- * list of {@link org.opengis.filter.sort.SortBy}.
+ * list of {@link org.opengis.filter.sort.SortBy} (WFS style syntax, as opposed
+ * to the CSW one, which is slightly different)
  *
  * @author Justin Deoliveira, The Open Planning Project
  *
@@ -31,7 +32,7 @@ public SortByKvpParser(FilterFactory filterFactory) {
      * {@link SortBy}.
      */
     protected Object parseToken(String token) throws Exception {
-        String[] nameOrder = token.split(" ");
+        String[] nameOrder = token.trim().split(" ");
         String propertyName = nameOrder[0];
 
         SortOrder order = SortOrder.ASCENDING;

src/ows/src/main/java/org/geoserver/ows/kvp/EMFKvpRequestReader.java

@@ -88,13 +88,7 @@ public Object read(Object request, Map kvp, Map rawKvp) throws Exception {
 
             if (EMFUtils.has(eObject, property)) {
                 try {
-                    //check for a collection
-                    if ( EMFUtils.isCollection(eObject, property) ) {
-                        EMFUtils.add(eObject, property, value);
-                    }
-                    else {
-                        EMFUtils.set(eObject, property, value);    
-                    }
+                    setValue(eObject, property, value);
                 } catch(Exception ex) {
                     throw new ServiceException("Failed to set property " + property 
                             + " in request object using value " + value 
@@ -106,4 +100,17 @@ public Object read(Object request, Map kvp, Map rawKvp) throws Exception {
 
         return request;
     }
+
+    /**
+     * Sets a value in the target EMF object, adding it to a collection if the target is a collection,
+     * setting it otherwise. Subclasses can override this behavior
+     */
+    protected void setValue(EObject eObject, String property, Object value) {
+        // check for a collection
+        if (EMFUtils.isCollection(eObject, property)) {
+            EMFUtils.add(eObject, property, value);
+        } else {
+            EMFUtils.set(eObject, property, value);
+        }
+    }
 }

src/wcs2_0/src/main/java/applicationContext.xml

@@ -113,6 +113,12 @@
     <property name="service" value="WCS" />
     <property name="version" value="2.0.1" />
   </bean>
+  <bean id="wcs201SortByKvpParser" class="org.geoserver.ows.kvp.SortByKvpParser">
+    <constructor-arg ref="filterFactory"/>
+    <property name="service" value="WCS"/>
+    <property name="version" value="2.0.1" />
+  </bean>
+
 
 
   <bean id="wcs20getCoverageKvpParser" class="org.geoserver.wcs2_0.kvp.WCS20GetCoverageRequestReader" />

src/wcs2_0/src/main/java/org/geoserver/wcs2_0/GetCoverage.java

@@ -20,6 +20,7 @@
 import java.util.Set;
 import java.util.logging.Level;
 import java.util.logging.Logger;
+import java.util.stream.Collectors;
 
 import javax.media.jai.BorderExtender;
 import javax.media.jai.Interpolation;
@@ -584,6 +585,7 @@ private WCSDimensionsSubsetHelper parseGridCoverageRequest(CoverageInfo ci, Grid
         gcr.setElevationSubset(requestSubset.getElevationSubset());
         gcr.setDimensionsSubset(requestSubset.getDimensionsSubset());
         gcr.setFilter(request.getFilter());
+        gcr.setSortBy(request.getSortBy());
         gcr.setOverviewPolicy(overviewPolicy);
         subsetHelper.setGridCoverageRequest(gcr);
         return subsetHelper;
@@ -865,9 +867,24 @@ private GridCoverage2D readCoverage(CoverageInfo cinfo, GridCoverageRequest requ
         }
 
         // handle filter
-        if(request.getFilter() != null) {
-            List<GeneralParameterDescriptor> descriptors = readParametersDescriptor.getDescriptor().descriptors();
-            readParameters = CoverageUtils.mergeParameter(descriptors, readParameters, request.getFilter(), "Filter");
+        if (request.getFilter() != null) {
+            List<GeneralParameterDescriptor> descriptors = readParametersDescriptor.getDescriptor()
+                    .descriptors();
+            readParameters = CoverageUtils.mergeParameter(descriptors, readParameters,
+                    request.getFilter(), "Filter");
+        }
+
+        // handle sorting
+        if (request.getSortBy() != null) {
+            List<GeneralParameterDescriptor> descriptors = readParametersDescriptor.getDescriptor()
+                    .descriptors();
+            String sortBySpec = request.getSortBy().stream()
+                    .map(sb -> sb.getPropertyName().getPropertyName() + " "
+                            + sb.getSortOrder().name().charAt(0))
+                    .collect(Collectors.joining(","));
+
+            readParameters = CoverageUtils.mergeParameter(descriptors, readParameters, sortBySpec,
+                    "SORTING");
         }
 
         // handle additional dimensions through dynamic parameters

src/wcs2_0/src/main/java/org/geoserver/wcs2_0/GridCoverageRequest.java

@@ -14,6 +14,7 @@
 import org.geotools.util.DateRange;
 import org.geotools.util.NumberRange;
 import org.opengis.filter.Filter;
+import org.opengis.filter.sort.SortBy;
 import org.opengis.referencing.crs.CoordinateReferenceSystem;
 
 /**
@@ -42,6 +43,8 @@ public class GridCoverageRequest {
     OverviewPolicy overviewPolicy;
 
     Filter filter;
+
+    List<SortBy> sortBy;
 
     public WCSEnvelope getSpatialSubset() {
         return spatialSubset;
@@ -98,6 +101,14 @@ public Filter getFilter() {
     public void setFilter(Filter filter) {
         this.filter = filter;
     }
+
+    public List<SortBy> getSortBy() {
+        return sortBy;
+    }
+
+    public void setSortBy(List<SortBy> sortBy) {
+        this.sortBy = sortBy;
+    }
 
     public Map<String, List<Object>> getDimensionsSubset() {
         return dimensionsSubset;

src/wcs2_0/src/main/java/org/geoserver/wcs2_0/kvp/WCS20GetCoverageRequestReader.java

@@ -1,9 +1,20 @@
 package org.geoserver.wcs2_0.kvp;
 
 import java.util.Arrays;
+import java.util.Collection;
 import java.util.List;
 import java.util.Map;
 
+import org.eclipse.emf.ecore.EObject;
+import org.geoserver.ows.kvp.EMFKvpRequestReader;
+import org.geoserver.ows.util.KvpUtils;
+import org.geoserver.platform.OWS20Exception;
+import org.geoserver.wcs2_0.WCS20Const;
+import org.geoserver.wcs2_0.exception.WCS20Exception;
+import org.geotools.wcs.v2_0.Interpolation;
+import org.geotools.wcs.v2_0.RangeSubset;
+import org.geotools.wcs.v2_0.Scaling;
+
 import net.opengis.wcs20.DimensionSubsetType;
 import net.opengis.wcs20.ExtensionItemType;
 import net.opengis.wcs20.GetCoverageType;
@@ -14,13 +25,6 @@
 import net.opengis.wcs20.ScalingType;
 import net.opengis.wcs20.Wcs20Factory;
 
-import org.geoserver.ows.kvp.EMFKvpRequestReader;
-import org.geoserver.ows.util.KvpUtils;
-import org.geoserver.wcs2_0.WCS20Const;
-import org.geotools.wcs.v2_0.Interpolation;
-import org.geotools.wcs.v2_0.RangeSubset;
-import org.geotools.wcs.v2_0.Scaling;
-
 /**
  * KVP reader for WCS 2.0 GetCoverage request
  * 
@@ -155,4 +159,23 @@ private void parseOverviewPolicyExtension(GetCoverageType gc, Map kvp) {
             }
         }
     }
+
+    @Override
+    protected void setValue(EObject eObject, String property, Object value) {
+        if ("sortBy".equalsIgnoreCase(property)) {
+            // we get an arraylist of arraylists
+            List sorts = (List) value;
+            final int sortsSize = sorts.size();
+            if (sortsSize != 1) {
+                throw new OWS20Exception(
+                        "Invalid sortBy specification, expecting sorts for just one coverage, but got "
+                                + sortsSize + " instead",
+                        WCS20Exception.WCS20ExceptionCode.InvalidParameterValue, "sortBy");
+            }
+            final GetCoverageType getCoverage = (GetCoverageType) (eObject);
+            getCoverage.getSortBy().addAll((List) sorts.get(0));
+        } else {
+            super.setValue(eObject, property, value);
+        }
+    }
 }

src/wcs2_0/src/main/java/org/geoserver/wcs2_0/response/WCSDefaultValuesHelper.java

@@ -157,7 +157,12 @@ private GridCoverageRequest setDefaultsFromStructuredReader(GridCoverageRequest
             Query query = new Query();
 
             // Set sorting order (default Policy is using Max... therefore Descending order)
-            sortBy(query, timeDimension, elevationDimension);
+            final List<SortBy> requestedSort = subsettingRequest.getSortBy();
+            if (requestedSort == null) {
+                sortBy(query, timeDimension, elevationDimension);
+            } else {
+                query.setSortBy(requestedSort.toArray(new SortBy[requestedSort.size()]));
+            }
             query.setFilter(finalFilter);
 
             // Returning a single feature matching the filtering

src/wcs2_0/src/main/java/org/geoserver/wcs2_0/response/WCSDimensionsSubsetHelper.java

@@ -887,6 +887,7 @@ public GridCoverageRequest createGridCoverageRequestSubset() throws IOException
         subsettingRequest.setTemporalSubset(temporalSubset);
         subsettingRequest.setDimensionsSubset(dimensionsSubset);
         subsettingRequest.setFilter(request.getFilter());
+        subsettingRequest.setSortBy(request.getSortBy());
 
         // Handle default values and update subsetting values if needed
         String coverageName =  getCoverageName();

src/wcs2_0/src/test/java/org/geoserver/wcs2_0/kvp/GetCoverageKvpTest.java

@@ -253,6 +253,20 @@ public void testCqlFilterGreen() throws Exception {
         assertOriginPixelColor(response, new int[] {0, 255, 0});
     }
 
+    @Test
+    public void testSortByLocationAscending() throws Exception {
+        MockHttpServletResponse response = getAsServletResponse("wcs?request=GetCoverage&service=WCS&version=2.0.1&coverageId=sf__mosaic&sortBy=location");
+        // green is the lowest, lexicographically
+        assertOriginPixelColor(response, new int[] {0, 255, 0});
+    }
+
+    @Test
+    public void testSortByLocationDescending() throws Exception {
+        MockHttpServletResponse response = getAsServletResponse("wcs?request=GetCoverage&service=WCS&version=2.0.1&coverageId=sf__mosaic&sortBy=location D");
+        // yellow is the highest, lexicographically
+        assertOriginPixelColor(response, new int[] {255, 255, 0});
+    }
+
     private void assertOriginPixelColor(MockHttpServletResponse response, int[] expected)
             throws DataSourceException, IOException {
         assertEquals("image/tiff", response.getContentType());

src/wfs/src/main/java/applicationContext.xml

@@ -382,7 +382,7 @@
     </bean>
 
 	<bean id="srsNameKvpParser" class="org.geoserver.wfs.kvp.SrsNameKvpParser"/>
-	<bean id="sortByKvpParser" class="org.geoserver.wfs.kvp.SortByKvpParser">
+	<bean id="sortByKvpParser" class="org.geoserver.ows.kvp.SortByKvpParser">
 		<constructor-arg ref="filterFactory"/>
 	</bean>
 	<bean id="expiryKvpParser" class="org.geoserver.ows.util.NumericKvpParser">