Unified in-memory caches

[brson]

There are many caches all up and down the browser stack, for things like images, fonts, glyphs, text runs, and numerous unimagined future things. Additionally servo may have dual caches for any particular thing, one remote cache in a dedicated task and another task-local cache. As control over memory usage is critical in web browsers we should plan on making this robust from the outset. This is concerned with memory caches, not disk.

Implement a generic caching framework and replace all the existing caches.

Things to consider:

• What strategies do we need for cache eviction? LRU-SP has been suggested as best-of-breed. Is it sufficient for all use cases?
• There are scenarios (with image loading at least, maybe more in the future) where we first consult a task-local cache then, if the resource we want isn't available, ask another task for it, which in turn maintains its own cache. Do these two caches need to communicate?
• In a low memory situation the browser should ideally be able to immediately free all unnecessary memory. This seems to imply some chain of communication between all caches and the OS. Consider though that it isn't simply enough to free memory at the language level - the allocator must actually return memory to the operating system. How do we do this?
• Related to the above, some Linux kernels have features that allow the OS itself to discard application memory when it needs to (either ashmem on Android or the work-in-progress volatile pages patches). Integrating this in a comprehensive way could be a big win. At the least we can design our API's with these in mind.

Related: #51

[brson]

Linux also has an upcoming feature for informing the client about general memory pressure, vmpressure_fd(): http://lwn.net/SubscriberLink/531077/d317d5bf9ab1855a/

[brson]

Here's how the linux kernel's LRU cache shrinking subsystem is designed: https://lwn.net/Articles/550463/