docs: turn README into full articles

Documentation/drivers-and-data-sources.md

@@ -0,0 +1,87 @@
+# Understanding drivers, their data sources, and creating your own
+
+Clair is organized into many different software components all of which are dynamically registered at compile time.
+All of these components can be found in the `ext/` directory.
+
+## Driver Types
+
+| Driver Type  | Functionality                                                                      | Example       |
+|--------------|------------------------------------------------------------------------------------|---------------|
+| featurefmt   | parses features of a particular format out of a layer                              | apk           |
+| featurens    | identifies whether a particular namespaces is applicable to a layer                | alpine 3.5    |
+| imagefmt     | determines the location of the root filesystem location for a layer                | docker        |
+| notification | implements the transport used to notify of vulnerability changes                   | webhook       |
+| versionfmt   | parses and compares version strings                                                | rpm           |
+| vulnmdsrc    | fetches vulnerability metadata and appends them to vulnerabilities being processed | nvd           |
+| vulnsrc      | fetches vulnerabilities for a set of namespaces                                    | alpine-sec-db |
+
+## Data Sources for the built-in drivers
+
+| Data Source                   | Data Collected                                                           | Format | License         |
+|-------------------------------|--------------------------------------------------------------------------|--------|-----------------|
+| [Debian Security Bug Tracker] | Debian 6, 7, 8, unstable namespaces                                      | [dpkg] | [Debian]        |
+| [Ubuntu CVE Tracker]          | Ubuntu 12.04, 12.10, 13.04, 14.04, 14.10, 15.04, 15.10, 16.04 namespaces | [dpkg] | [GPLv2]         |
+| [Red Hat Security Data]       | CentOS 5, 6, 7 namespaces                                                | [rpm]  | [CVRF]          |
+| [Oracle Linux Security Data]  | Oracle Linux 5, 6, 7 namespaces                                          | [rpm]  | [CVRF]          |
+| [Alpine SecDB]                | Alpine 3.3, Alpine 3.4, Alpine 3.5 namespaces                            | [apk]  | [MIT]           |
+| [NIST NVD]                    | Generic Vulnerability Metadata                                           | N/A    | [Public Domain] |
+
+[Debian Security Bug Tracker]: https://security-tracker.debian.org/tracker
+[Ubuntu CVE Tracker]: https://launchpad.net/ubuntu-cve-tracker
+[Red Hat Security Data]: https://www.redhat.com/security/data/metrics
+[Oracle Linux Security Data]: https://linux.oracle.com/security/
+[NIST NVD]: https://nvd.nist.gov
+[dpkg]: https://en.wikipedia.org/wiki/dpkg
+[rpm]: http://www.rpm.org
+[Debian]: https://www.debian.org/license
+[GPLv2]: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
+[CVRF]: http://www.icasi.org/cvrf-licensing/
+[Public Domain]: https://nvd.nist.gov/faq
+[Alpine SecDB]: http://git.alpinelinux.org/cgit/alpine-secdb/
+[apk]: http://git.alpinelinux.org/cgit/apk-tools/
+[MIT]: https://gist.github.com/jzelinskie/6da1e2da728424d88518be2adbd76979
+
+## Adding new drivers
+
+In order to allow programmers to add additional behavior, Clair follows a pattern that Go programmers may recognize from the standard `database/sql` library.
+Each Driver Type defines an interface that must be implemented by drivers.
+
+```go
+type DriverInterface interface {
+	Action() error
+}
+
+func RegisterDriver(name, DriverInterface) { ... }
+```
+
+Create a new Go package containing an implementation of the driver interface.
+In the source file that implements this custom interface, create an `init()` function that registers the driver.
+
+```go
+func init() {
+	drivers.RegisterDriver("mydrivername", myDriverImplementation{})
+}
+
+// This line causes the Go compiler to enforce that myDriverImplementation
+// implements the the DriverInterface at compile time.
+var _ drivers.DriverInterface = myDriverImplementation{}
+
+type myDriverImplementation struct{}
+
+func (d myDriverImplementation) Action() error {
+	fmt.Println("Hello, Clair!")
+	return nil
+}
+```
+
+The final step is to import the new driver in `main.go` as `_` in order ensure that the `init()` function executes, thus registering your driver.
+
+```go
+import (
+	...
+
+	_ "github.com/you/yourclairdriver"
+)
+```
+
+If you believe what you've created can benefit others outside of your organization, please consider open sourcing it and creating a pull request to get it included by default.

Documentation/notifications.md

@@ -1,13 +1,14 @@
 # Notifications
 
-Notifications are a way for Clair to inform an endpoint that changes to tracked vulnerabilities have occurred.
+Notifications are a way for Clair to inform another service that changes to tracked vulnerabilities have occurred.
+Because changes to vulnerabilities also contain the set of affected images, Clair sends only the name of the notification to another service, then depends on that service read and mark the notification as read using Clair's API.
 Because notification data can require pagination, Clair should only send the name of a notification.
-It is expected that the receiving endpoint calls the Clair API for reading notifications and marking them as read after being notified.
-If the notification is never marked as read, Clair will continue attempting to send the same notification to the endpoint indefinitely.
+If a notification is not marked as read, Clair will resend notifications at a configured interval.
 
-## Webhook
+# Webhook
 
-Webhook is an out-of-the-box notifier that sends the following JSON object via an HTTP POST:
+Notifications are an extensible component of Clair, but out of the box Clair supports [webhooks].
+The webhooks look like the following:
 
 ```json
 {
@@ -17,11 +18,7 @@ Webhook is an out-of-the-box notifier that sends the following JSON object via a
 }
 ```
 
-## Custom Notifiers
+If you're interested in adding your own notification senders, read the documentation on [adding new drivers].
 
-Clair can also be compiled with custom notifiers by importing them in `main.go`.
-Custom notifiers are any Go package that implements the `Notifier` interface and registers themselves with the `notifier` package.
-Notifiers are registered in [init()] similar to drivers for Go's standard [database/sql] package.
-
-[init()]: https://golang.org/doc/effective_go.html#init
-[database/sql]: https://godoc.org/database/sql
+[webhooks]: https://en.wikipedia.org/wiki/Webhook
+[adding new drivers]: /Documentation/drivers-and-data-sources.md#adding-new-drivers

Documentation/presentations.md

@@ -0,0 +1,17 @@
+# Presentations
+
+This document tracks projects that integrate with Clair. [Join the community](https://github.com/coreos/clair/), and help us keep the list up-to-date.
+
+## Clair v1
+
+| Title                                                                     | Event                         | Video                 | Slides                |
+|---------------------------------------------------------------------------|-------------------------------|-----------------------|-----------------------|
+| Clair: The Container Image Security Analyzer                              | [ContainerDays Boston 2016]   | https://goo.gl/ey7QQR | https://goo.gl/WmNiwA |
+| Identifying Common Vulnerabilities and Exposures in Containers with Clair | [CoreOS Fest 2016]            | https://goo.gl/fGtb9s | https://goo.gl/35gixV |
+| Clair: A Container Image Security Analyzer                                | [Microservices NYC]           | https://goo.gl/WZaCU2 | https://goo.gl/sCXGcH |
+| Clair: A Container Image Security Analyzer                                | [Container Orchestration NYC] | https://goo.gl/wxi24C | https://goo.gl/VfRxv2 |
+
+[ContainerDays Boston 2016]: http://dynamicinfradays.org/events/2016-boston/
+[CoreOS Fest 2016]: https://coreos.com/fest/#2016
+[Microservices NYC]: https://www.meetup.com/Microservices-NYC/events/230023492/
+[Container Orchestration NYC]: https://www.meetup.com/Kubernetes-Cloud-Native-New-York/events/229779466/

Documentation/production-users.md

@@ -4,5 +4,4 @@ This document tracks people and use cases for Clair in production. [Join the com
 
 ## [Quay.io](https://quay.io/)
 
-Quay is CoreOS' enterprise-ready container registry. It displays the results of a Clair security scan on each hosted image's page.
-
+Quay.io is hosted and enterprise-ready container registry. It displays the results of a Clair security scan for each image uploaded to the registry.

Documentation/running-clair.md

@@ -0,0 +1,111 @@
+# Running Clair
+
+The following document outlines possible ways to deploy Clair both on your local machine and to a cluster of machines.
+
+## Official Container Repositories
+
+Clair is officially packaged and released as a container.
+
+* [quay.io/coreos/clair] - Stable releases
+* [quay.io/coreos/clair-jwt] - Stable releases with an embedded instance of [jwtproxy]
+* [quay.io/coreos/clair-git] - Development releases
+
+[quay.io/coreos/clair]: https://quay.io/repository/coreos/clair
+[jwtproxy]: https://github.com/coreos/jwtproxy
+[quay.io/coreos/clair-jwt]: https://quay.io/repository/coreos/clair-jwt
+[quay.io/coreos/clair-git]: https://quay.io/repository/coreos/clair-git
+
+## Common Architecture
+
+### Registry Integration
+
+Clair can be integrated directly into a container registry such that the registry is responsible for interacting with Clair on behalf of the user.
+This type of setup avoids the manual scanning of images and creates a sensible location to which Clair's vulnerability notifications can be propagated.
+The registry can also be used for authorization to avoid sharing vulnerability information about images to which one might not have access.
+
+![Simple Clair Diagram](https://cloud.githubusercontent.com/assets/343539/21630809/c1adfbd2-d202-11e6-9dfe-9024139d0a28.png)
+
+### CI/CD Integration
+
+Clair can be integrated into a CI/CD pipeline such that when a container image is produced, the step after pushing the image to a registry is to compose a request for Clair to scan that particular image.
+This type of integration is more flexible, but relies on additional components to be setup in order to secure.
+
+## Deployment Strategies
+
+**NOTE:** These instructions demonstrate running HEAD and not stable versions.
+
+The following are community supported instructions to run Clair in a variety of ways.
+A [PostgreSQL 9.4+] database instance is required for all instructions.
+
+[PostgreSQL 9.4+]: https://www.postgresql.org
+
+### Cluster
+
+#### Kubernetes
+
+If you don't have a local Kubernetes cluster already, check out [minikube].
+
+[minikube]: https://github.com/kubernetes/minikube
+
+```
+git clone https://github.com/coreos/clair
+cd clair/contrib/k8s
+kubectl create secret generic clairsecret --from-file=./config.yaml
+kubectl create -f clair-kubernetes.yaml
+```
+
+### Local
+
+#### Docker Compose
+
+```sh
+$ curl -L https://raw.githubusercontent.com/coreos/clair/master/docker-compose.yml -o $HOME/docker-compose.yml
+$ mkdir $HOME/clair_config
+$ curl -L https://raw.githubusercontent.com/coreos/clair/master/config.example.yaml -o $HOME/clair_config/config.yaml
+$ $EDITOR $HOME/clair_config/config.yaml # Edit database source to be postgresql://postgres:password@postgres:5432?sslmode=disable
+$ docker-compose -f $HOME/docker-compose.yml up -d
+```
+
+Docker Compose may start Clair before Postgres which will raise an error.
+If this error is raised, manually execute `docker-compose start clair`.
+
+#### Docker
+
+```sh
+$ mkdir $PWD/clair_config
+$ curl -L https://raw.githubusercontent.com/coreos/clair/master/config.example.yaml -o $PWD/clair_config/config.yaml
+$ docker run -d -e POSTGRES_PASSWORD="" -p 5432:5432 postgres:9.6
+$ docker run -d -p 6060-6061:6060-6061 -v $PWD/clair_config:/config quay.io/coreos/clair-git:latest -config=/config/config.yaml
+```
+
+#### Source
+
+To build Clair, you need to latest stable version of [Go] and a working [Go environment].
+In addition, Clair requires some additional binaries be installed on the system [$PATH] as runtime dependencies:
+
+* [git]
+* [bzr]
+* [rpm]
+* [xz]
+
+[Go]: https://github.com/golang/go/releases
+[Go environment]: https://golang.org/doc/code.html
+[git]: https://git-scm.com
+[bzr]: http://bazaar.canonical.com/en
+[rpm]: http://www.rpm.org
+[xz]: http://tukaani.org/xz
+[$PATH]: https://en.wikipedia.org/wiki/PATH_(variable)
+
+```sh
+$ go get github.com/coreos/clair
+$ go install github.com/coreos/clair/cmd/clair
+$ $EDITOR config.yaml # Add the URI for your postgres database
+$ ./$GOPATH/bin/clair -config=config.yaml
+```
+
+## Troubleshooting
+
+### I just started up Clair and nothing appears to be working, what's the deal?
+
+During the first run, Clair will bootstrap its database with vulnerability data from the configured data sources.
+It can take several minutes before the database has been fully populated, but once this data is stored in the database, subsequent updates will take far less time.

Documentation/terminology.md

@@ -0,0 +1,15 @@
+# Terminiology
+
+## Container
+
+- *Container* - the execution of an image
+- *Image* - a set of tarballs that contain the filesystem contents and run-time metadata of a container
+- *Layer* - one of the tarballs used in the composition of an image, often expressed as a filesystem delta from another layer
+
+## Specific to Clair
+
+- *Ancestry* - the Clair-internal representation of an Image
+- *Feature* - anything that when present in a filesystem could be an indication of a *vulnerability* (e.g. the presence of a file or an installed software package)
+- *Feature Namespace* (featurens) - a context around *features* and *vulnerabilities* (e.g. an operating system or a programming language)
+- *Vulnerability Source* (vulnsrc) - the component of Clair that tracks upstream vulnerability data and imports them into Clair's database
+- *Vulnerability Metadata Source* (vulnmdsrc) - the component of Clair that tracks upstream vulnerability metadata and associates them with vulnerabilities in Clair's database