Perf: Improve caching of stones

[sunnyflunk]

This mainly affects boulder, where you already have some packages downloaded already. Currently it starts fetching missing stones and caching them as they arrive. However, any stone that is already fetched does not start caching until all files have been downloaded (and cached).

Ideally moss should be queueing already fetched packages while missing files are downloaded.

[sunnyflunk]

So I've had a think about this one and the simplest (and quite effective) way to handle this is to abuse the fetch queue by adding cache only jobs to it. Currently if a stone exists, it skips it (adds to precacheItems) to cache after all files have been fetched. This is inefficient and we need it to be part of the queue.

By making expSize 0 (and making moss-fetcher not fetch when 0), then when the queue is sorted, the worker starting from the smallest size will not download the file and then cache them while the 3 other workers fetch the large files.

There are implications as we use expSize 0 to signify we don't know the size. As an alternative we could make job.remoteURI empty/null (or both conditions) to signify there's nothing to fetch (which is probably a better approach).

Something like this (with moss-fetcher changes needed) and a bit of cleanup no doubt to threadCompletionHandler

diff --git a/source/moss/client/impl.d b/source/moss/client/impl.d
index 12e5ead..07524ea 100644
--- a/source/moss/client/impl.d
+++ b/source/moss/client/impl.d
@@ -250,7 +250,6 @@ public final class MossClient
         renderer = new Renderer();
 
         auto st = stateDB.createState(tx, application);
-        string[] precacheItems;
 
         /* For all packages in state - form job queue */
         foreach (pkg; application)
@@ -284,11 +283,15 @@ public final class MossClient
 
             if (job.destinationPath.exists)
             {
-                precacheItems ~= job.remoteURI;
-                continue;
+                expSize = 0;
+                job.remoteURI = null;
+            }
+            else
+            {
+                downloadProgress.total = downloadProgress.total + 1;
+                dlTotal += expSize;
             }
 
-            dlTotal += expSize;
             cacheTotal++;
 
             /* TODO: Set closure for caching + hash verification! */
@@ -301,15 +304,13 @@ public final class MossClient
                 cachePackage(f.sourceURI);
             }
 
-            downloadProgress.total = downloadProgress.total + 1;
-
-            /* Only fetch what is missing. */
+            /* Add all jobs, will only fetch where job.remoteURI still exists. */
             auto fc = Fetchable(job.remoteURI, job.destinationPath, expSize,
                     FetchType.RegularFile, &threadCompletionHandler);
             fetchContext.enqueue(fc);
         }
 
-        immutable bool haveWork = workQueue.length > 0 || !precacheItems.empty;
+        immutable bool haveWork = workQueue.length > 0;
 
         /* Can't have progress on no work.. */
         if (haveWork)
@@ -347,11 +348,6 @@ public final class MossClient
             fetchContext.fetch();
         }
 
-        foreach (pkg; precacheItems)
-        {
-            cachePackage(pkg);
-        }
-
         /* Lets get ourselves a state ID */
         stateDB.save(st);