poolmanager,pool: Fix precision underflow in WASS

Fixes a problem in which a large number of writers on a pool (such as what you
would see on a tape read pool) would cause the scale of a double to underflow,
causing the weigthed acailable space to become zero.  If all pools have such
high load, no pool would be availble for selection and pool manager would
report that no pool is available for staging.

The fix uses the fact that only the relative weighted available space is
relevant. Since the weighted space is calculated by dividing unweighted space
with two to the power of the load, we can subtract a constant from the load of
all pools and maintain the relative available space of each pool. The patch
first finds the lowest load of all non-full pools in the selection and
subtracts that from the load of all pools, thus normalizing the calculation.

Other pools may have even high load, still causing the precision of a double to
be insufficient, however the likelihood of selection such a pool is virtuallt
zero anyway, so this barely affects the write distribution. The normalization
does however ensure that at least one pool does not underflow (assuming
non-full pools exist at all) and thus that a pool can be selected.

Target: 2.6
Ticket: http://rt.dcache.org/Ticket/Display.html?id=7993
Require-notes: yes
Require-book: no

Author: gbehrmann

modules/dcache/src/main/java/org/dcache/poolmanager/WassPartition.java

@@ -249,39 +249,65 @@ protected double getAvailable(PoolSpaceInfo space, long filesize)
      * become random. This it the same behaviour as with the classic
      * partition.
      */
-    protected double getWeightedAvailable(PoolCostInfo info, long filesize)
+    protected double getWeightedAvailable(PoolCostInfo info, double available, double load)
     {
-        double available = getAvailable(info.getSpaceInfo(), filesize);
-        double load = _performanceCostFactor *
-            info.getMoverCostFactor() * info.getWriters();
-        return Math.pow(available, _spaceCostFactor) / Math.pow(2.0, load);
+        return (available == 0) ? 0 : (Math.pow(available, _spaceCostFactor) / Math.pow(2.0, load));
+    }
+
+    private double getLoad(PoolCostInfo info)
+    {
+        return _performanceCostFactor * info.getMoverCostFactor() * info.getWriters();
     }
 
     /**
      * Selects a pool from a list using the WASS algorithm.
-     *
+     * <p/>
      * Returns null if all pools are full.
      */
-    protected PoolInfo
-        selectByAvailableSpace(List<PoolInfo> pools, long size)
+    public PoolInfo selectByAvailableSpace(List<PoolInfo> pools, long filesize)
     {
-        double[] available = new double[pools.size()];
-        double sum = 0.0;
+        int length = pools.size();
+        double[] available = new double[length];
 
-        for (int i = 0; i < available.length; i++) {
-            sum += getWeightedAvailable(pools.get(i).getCostInfo(), size);
+        /* Calculate available space adjusted by space cost factor. Determine the smallest
+         * load of all pools able to hold the file.
+         */
+        double minLoad = Double.POSITIVE_INFINITY;
+        for (int i = 0; i < length; i++) {
+            PoolCostInfo info = pools.get(i).getCostInfo();
+            double free = getAvailable(info.getSpaceInfo(), filesize);
+            if (free > 0) {
+                available[i] = free;
+                minLoad = Math.min(minLoad, getLoad(info));
+            }
+        }
+
+        if (minLoad == Double.POSITIVE_INFINITY) {
+            return null;
+        }
+
+        /* Weight available space by normalized load. Load is normalized to ensure that at least
+         * for one pool we maintain enough precision to not reduce available space to zero.
+         */
+        double sum = 0.0;
+        for (int i = 0; i < length; i++) {
+            PoolCostInfo info = pools.get(i).getCostInfo();
+            double normalizedLoad = getLoad(info) - minLoad;
+            double weightedAvailable = getWeightedAvailable(info, available[i], normalizedLoad);
+            sum += weightedAvailable;
             available[i] = sum;
         }
 
+        /* Randomly choose one of the pools.
+         */
         double threshold = random() * sum;
-
-        for (int i = 0; i < available.length; i++) {
+        for (int i = 0; i < length; i++) {
             if (threshold < available[i]) {
                 return pools.get(i);
             }
         }
 
-        return null;
+        throw new RuntimeException("Unreachable statement");
     }
 
     @Override