Use Celerity to version all static resources

Summary:
We don't use versioned URIs for images, so when they change users may get old versions.

This was a particular issue with the recent logo change, which several users reported cache-related issues from.

Instead, use Celerity to manage image URI versions in addition to CSS/JS.

This is complicated, because we need to rewrite image URIs inside of CSS, which means the hash of a CSS file has to be derived from the current image data. Otherwise, when we updated an image the CSS wouldn't update, so we wouldn't be any better off.

So basically we:

  - Find all the "raw" files, and put them into the map.
  - Find all the CSS/JS, perform content-altering transformations on it (i.e., not minification) based on the partial map, and then put it into the map based on transformed hashes.

(If we wanted, we could now do CSS variables or whatever for "free", more or less.)

Test Plan:
  - Regenerated celerity map, browsed site, verified images generated with versioned URIs.
  - Moved "blue" flag image over "green" flag image, regenerated map, verified "green" flag image and the associated CSS changed hashes.
  - Added transformation unit tests; ran unit tests.

Reviewers: btrahan, vrana, jungejason

Reviewed By: vrana

CC: aran

Maniphest Tasks: T1073

Differential Revision: https://secure.phabricator.com/D2146

Author: epriestley

scripts/celerity_mapper.php

@@ -144,7 +144,43 @@
 
 $root = Filesystem::resolvePath($argv[1]);
 
-echo "Finding static resources...\n";
+$resource_hash = PhabricatorEnv::getEnvConfig('celerity.resource-hash');
+$runtime_map = array();
+
+echo "Finding raw static resources...\n";
+$raw_files = id(new FileFinder($root))
+  ->withType('f')
+  ->withSuffix('png')
+  ->withSuffix('jpg')
+  ->withSuffix('gif')
+  ->withSuffix('swf')
+  ->withFollowSymlinks(true)
+  ->setGenerateChecksums(true)
+  ->find();
+
+echo "Processing ".count($raw_files)." files";
+foreach ($raw_files as $path => $hash) {
+  echo ".";
+  $path = '/'.Filesystem::readablePath($path, $root);
+  $type = CelerityResourceTransformer::getResourceType($path);
+
+  $hash = md5($hash.$path.$resource_hash);
+  $uri  = '/res/'.substr($hash, 0, 8).$path;
+
+  $runtime_map[$path] = array(
+    'hash' => $hash,
+    'uri'  => $uri,
+    'disk' => $path,
+    'type' => $type,
+  );
+}
+echo "\n";
+
+$xformer = id(new CelerityResourceTransformer())
+  ->setMinify(false)
+  ->setRawResourceMap($runtime_map);
+
+echo "Finding transformable static resources...\n";
 $files = id(new FileFinder($root))
   ->withType('f')
   ->withSuffix('js')
@@ -155,26 +191,31 @@
 
 echo "Processing ".count($files)." files";
 
-$resource_hash = PhabricatorEnv::getEnvConfig('celerity.resource-hash');
-
 $file_map = array();
-foreach ($files as $path => $hash) {
+foreach ($files as $path => $raw_hash) {
   echo ".";
-  $name = '/'.Filesystem::readablePath($path, $root);
-  $file_map[$name] = array(
-    'hash' => md5($hash.$name.$resource_hash),
+  $path = '/'.Filesystem::readablePath($path, $root);
+  $data = Filesystem::readFile($root.$path);
+
+  $data = $xformer->transformResource($path, $data);
+  $hash = md5($data);
+  $hash = md5($hash.$path.$resource_hash);
+
+  $file_map[$path] = array(
+    'hash' => $hash,
     'disk' => $path,
   );
 }
 echo "\n";
 
-$runtime_map = array();
 $resource_graph = array();
 $hash_map = array();
 
 $parser = new PhutilDocblockParser();
 foreach ($file_map as $path => $info) {
-  $data = Filesystem::readFile($info['disk']);
+  $type = CelerityResourceTransformer::getResourceType($path);
+
+  $data = Filesystem::readFile($root.$info['disk']);
   $matches = array();
   $ok = preg_match('@/[*][*].*?[*]/@s', $data, $matches);
   if (!$ok) {
@@ -202,11 +243,6 @@
 
   $provides = reset($provides);
 
-  $type = 'js';
-  if (preg_match('/\.css$/', $path)) {
-    $type = 'css';
-  }
-
   $uri = '/res/'.substr($info['hash'], 0, 8).$path;
 
   $hash_map[$provides] = $info['hash'];