Implement font fallback

Prior to this change, if none of the fonts specified in CSS contained a
glyph for a codepoint, we tried only one fallback font. If that font
didn't contain the glyph, we'd give up.

With this change, we try multiple fonts in turn. The font names we try
differ across each platform, and based on the codepoint we're trying to
match. The current implementation is heavily inspired by the analogous
code in Gecko, but I've used to ucd lib to make it more readable,
whereas Gecko matches raw unicode ranges.

This fixes some of the issues reported in #17267, although colour emoji
support is not implemented.

== Notes on changes to WPT metadata ==

=== css/css-text/i18n/css3-text-line-break-opclns-* ===

A bunch of these have started failing on macos when they previously
passed.

These tests check that the browser automatically inserts line breaks
near certain characters that are classified as "opening and closing
punctuation". The idea is that if we have e.g. an opening parenthesis,
it does not make sense for it to appear at the end of a line box; it
should "stick" to the next character and go into the next line box.

Before this change, a lot of these codepoints rendered as a missing
glyph on Mac and Linux. In some cases, that meant that the test was
passing.

After this change, a bunch of these codepoints are now rendering glyphs
on Mac (but not Linux). In some cases, the test should continue to pass
where it previously did when rendering with the missing glyph.

However, it seems this has also exposed a layout bug. The "ref" div in
these tests contains a <br> element, and it seems that this, combined
with these punctuation characters, makes the spacing between glyphs ever
so slightly different to the "test" div. (Speculation: might be
something to do with shaping?)

Therefore I've had to mark a bunch of these tests failing on mac.

=== css/css-text/i18n/css3-text-line-break-baspglwj-* ===

Some of these previously passed on Mac due to a missing glyph. Now that
we're rendering the correct glyph, they are failing.

=== css/css-text/word-break/word-break-normal-bo-000.html ===

The characters now render correctly on Mac, and the test is passing. But
we do not find a suitable fallback font on Linux, so it is still failing
on that platform.

=== css/css-text/word-break/word-break-break-all-007.html ===

This was previously passing on Mac, but only because missing character
glyphs were rendered. Now that a fallback font is able to be found, it
(correctly) fails.

=== mozilla/tests/css/font_fallback_* ===

These are new tests added in this commit. 01 and 02 are marked failing
on Linux because the builders don't have the appropriate fonts installed
(that will be a follow-up).

Fix build errors from rebase

FontTemplateDescriptor can no longer just derive(Hash). We need to
implement it on each component part, because the components now
generally wrap floats, which do not impl Hash because of NaN. However in
this case we know that we won't have a NaN, so it is safe to manually
impl Hash.

components/gfx/Cargo.toml

@@ -42,6 +42,7 @@ unicode-bidi = {version = "0.3", features = ["with_serde"]}
 unicode-script = {version = "0.2", features = ["harfbuzz"]}
 webrender_api = {git = "https://github.com/servo/webrender", features = ["ipc"]}
 xi-unicode = "0.1.0"
+ucd = "0.1.1"
 
 [target.'cfg(target_os = "macos")'.dependencies]
 byteorder = "1.0"

components/gfx/font.rs

@@ -9,12 +9,14 @@ use font_template::FontTemplateDescriptor;
 use ordered_float::NotNaN;
 use platform::font::{FontHandle, FontTable};
 use platform::font_context::FontContextHandle;
+pub use platform::font_list::fallback_font_families;
 use platform::font_template::FontTemplateData;
 use servo_atoms::Atom;
 use smallvec::SmallVec;
 use std::borrow::ToOwned;
 use std::cell::RefCell;
 use std::collections::HashMap;
+use std::iter;
 use std::rc::Rc;
 use std::str;
 use std::sync::Arc;
@@ -117,7 +119,7 @@ pub struct FontMetrics {
 /// template at a particular size, with a particular font-variant-caps applied, etc. This contrasts
 /// with `FontTemplateDescriptor` in that the latter represents only the parameters inherent in the
 /// font data (weight, stretch, etc.).
-#[derive(Clone, Debug, PartialEq)]
+#[derive(Clone, Debug, Eq, Hash, PartialEq)]
 pub struct FontDescriptor {
     pub template_descriptor: FontTemplateDescriptor,
     pub variant: font_variant_caps::T,
@@ -329,6 +331,7 @@ pub type FontRef = Rc<RefCell<Font>>;
 pub struct FontGroup {
     descriptor: FontDescriptor,
     families: SmallVec<[FontGroupFamily; 8]>,
+    last_matching_fallback: Option<FontRef>,
 }
 
 impl FontGroup {
@@ -337,10 +340,14 @@ impl FontGroup {
 
         let families =
             style.font_family.0.iter()
-                .map(|family| FontGroupFamily::new(descriptor.clone(), family.clone()))
+                .map(|family| FontGroupFamily::new(descriptor.clone(), &family))
                 .collect();
 
-        FontGroup { descriptor, families }
+        FontGroup {
+            descriptor,
+            families,
+            last_matching_fallback: None,
+        }
     }
 
     /// Finds the first font, or else the first fallback font, which contains a glyph for
@@ -352,35 +359,78 @@ impl FontGroup {
         mut font_context: &mut FontContext<S>,
         codepoint: char
     ) -> Option<FontRef> {
-        self.find(&mut font_context, |font| font.borrow().has_glyph_for(codepoint))
-            .or_else(|| self.first(&mut font_context))
+        let has_glyph = |font: &FontRef| font.borrow().has_glyph_for(codepoint);
+
+        let font = self.find(&mut font_context, |font| has_glyph(font));
+        if font.is_some() {
+            return font
+        }
+
+        if let Some(ref fallback) = self.last_matching_fallback {
+            if has_glyph(&fallback) {
+                return self.last_matching_fallback.clone()
+            }
+        }
+
+        let font = self.find_fallback(&mut font_context, Some(codepoint), has_glyph);
+        if font.is_some() {
+            self.last_matching_fallback = font.clone();
+            return font
+        }
+
+        self.first(&mut font_context)
     }
 
+    /// Find the first available font in the group, or the first available fallback font.
     pub fn first<S: FontSource>(
         &mut self,
         mut font_context: &mut FontContext<S>
     ) -> Option<FontRef> {
         self.find(&mut font_context, |_| true)
+            .or_else(|| self.find_fallback(&mut font_context, None, |_| true))
     }
 
     /// Find a font which returns true for `predicate`. This method mutates because we may need to
     /// load new font data in the process of finding a suitable font.
     fn find<S, P>(
         &mut self,
         mut font_context: &mut FontContext<S>,
-        mut predicate: P
+        predicate: P,
     ) -> Option<FontRef>
     where
         S: FontSource,
-        P: FnMut(&FontRef) -> bool
+        P: FnMut(&FontRef) -> bool,
     {
         self.families.iter_mut()
             .filter_map(|family| family.font(&mut font_context))
-            .find(|f| predicate(f))
-            .or_else(|| {
-                font_context.fallback_font(&self.descriptor)
-                    .into_iter().find(predicate)
-            })
+            .find(predicate)
+    }
+
+    /// Attempts to find a suitable fallback font which matches the `predicate`. The default
+    /// family (i.e. "serif") will be tried first, followed by platform-specific family names.
+    /// If a `codepoint` is provided, then its Unicode block may be used to refine the list of
+    /// family names which will be tried.
+    fn find_fallback<S, P>(
+        &mut self,
+        font_context: &mut FontContext<S>,
+        codepoint: Option<char>,
+        predicate: P,
+    ) -> Option<FontRef>
+    where
+        S: FontSource,
+        P: FnMut(&FontRef) -> bool,
+    {
+        iter::once(FontFamilyDescriptor::default())
+            .chain(
+                fallback_font_families(codepoint).into_iter().map(|family| {
+                     FontFamilyDescriptor::new(
+                         FontFamilyName::from(family),
+                         FontSearchScope::Local,
+                     )
+                })
+            )
+            .filter_map(|family| font_context.font(&self.descriptor, &family))
+            .find(predicate)
     }
 }
 
@@ -389,17 +439,22 @@ impl FontGroup {
 /// only if actually needed.
 #[derive(Debug)]
 struct FontGroupFamily {
-    descriptor: FontDescriptor,
-    family: SingleFontFamily,
+    font_descriptor: FontDescriptor,
+    family_descriptor: FontFamilyDescriptor,
     loaded: bool,
     font: Option<FontRef>,
 }
 
 impl FontGroupFamily {
-    fn new(descriptor: FontDescriptor, family: SingleFontFamily) -> FontGroupFamily {
+    fn new(font_descriptor: FontDescriptor, family: &SingleFontFamily) -> FontGroupFamily {
+        let family_descriptor = FontFamilyDescriptor::new(
+            FontFamilyName::from(family),
+            FontSearchScope::Any
+        );
+
         FontGroupFamily {
-            descriptor,
-            family,
+            font_descriptor,
+            family_descriptor,
             loaded: false,
             font: None,
         }
@@ -410,7 +465,7 @@ impl FontGroupFamily {
     /// subsequent calls.
     fn font<S: FontSource>(&mut self, font_context: &mut FontContext<S>) -> Option<FontRef> {
         if !self.loaded {
-            self.font = font_context.font(&self.descriptor, &self.family);
+            self.font = font_context.font(&self.font_descriptor, &self.family_descriptor);
             self.loaded = true;
         }
 
@@ -451,3 +506,74 @@ pub fn get_and_reset_text_shaping_performance_counter() -> usize {
     TEXT_SHAPING_PERFORMANCE_COUNTER.store(0, Ordering::SeqCst);
     value
 }
+
+/// The scope within which we will look for a font.
+#[derive(Clone, Debug, Deserialize, Eq, Hash, PartialEq, Serialize)]
+pub enum FontSearchScope {
+    /// All fonts will be searched, including those specified via `@font-face` rules.
+    Any,
+
+    /// Only local system fonts will be searched.
+    Local,
+}
+
+/// A font family name used in font selection.
+#[derive(Clone, Debug, Deserialize, Eq, Hash, PartialEq, Serialize)]
+pub enum FontFamilyName {
+    /// A specific name such as `"Arial"`
+    Specific(Atom),
+
+    /// A generic name such as `sans-serif`
+    Generic(Atom),
+}
+
+impl FontFamilyName {
+    pub fn name(&self) -> &str {
+        match *self {
+            FontFamilyName::Specific(ref name) => name,
+            FontFamilyName::Generic(ref name) => name,
+        }
+    }
+}
+
+impl<'a> From<&'a SingleFontFamily> for FontFamilyName {
+    fn from(other: &'a SingleFontFamily) -> FontFamilyName {
+        match *other {
+            SingleFontFamily::FamilyName(ref family_name) =>
+                FontFamilyName::Specific(family_name.name.clone()),
+
+            SingleFontFamily::Generic(ref generic_name) =>
+                FontFamilyName::Generic(generic_name.clone()),
+        }
+    }
+}
+
+impl<'a> From<&'a str> for FontFamilyName {
+    fn from(other: &'a str) -> FontFamilyName {
+        FontFamilyName::Specific(Atom::from(other))
+    }
+}
+
+/// The font family parameters for font selection.
+#[derive(Clone, Debug, Deserialize, Eq, Hash, PartialEq, Serialize)]
+pub struct FontFamilyDescriptor {
+    pub name: FontFamilyName,
+    pub scope: FontSearchScope,
+}
+
+impl FontFamilyDescriptor {
+    fn new(name: FontFamilyName, scope: FontSearchScope) -> FontFamilyDescriptor {
+        FontFamilyDescriptor { name, scope }
+    }
+
+    fn default() -> FontFamilyDescriptor {
+        FontFamilyDescriptor {
+            name: FontFamilyName::Generic(atom!("serif")),
+            scope: FontSearchScope::Local,
+        }
+    }
+
+    pub fn name(&self) -> &str {
+        self.name.name()
+    }
+}