Create a public registry interface and separate out HTTP exposition

General context and approch
===========================

This is the first part of the long awaited wider refurbishment of
`client_golang/prometheus/...`. After a lot of struggling, I decided
to not go for one breaking big-bang, but cut things into smaller steps
after all, mostly to keep the changes manageable and easy to
review. I'm aiming for having the invasive breaking changes
concentrated in as few steps as possible (ideally one). Some steps
will not be breaking at all, but typically there will be breaking
changes that only affect quite special cases so that 95+% of users
will not be affected. This first step is an example for that, see
details below.

What's happening in this commit?
================================

This step is about finally creating an exported registry
interface. This could not be done by simply export the existing
internal implementation because the interface would be _way_ too
fat. This commit introduces a very lean `Registry` interface. Most of
the existing functionality that is not part of that interface is
provided by helper functions, not by methods
(e.g. `MustRegisterWith`). The functions that act on the default
registry are retained (with very few exceptions) so that most use
cases won't see a change.

The default registry is kept in the public variable
`DefaultRegistry`. This follows the example of the http package in the
standard library (cf. `http.DefaultServeMux`, `http.DefaultClient`)
with the same implications.

Another important part in making the registry lean is the extraction
of the HTTP exposition, which also allows for customization of the
HTTP exposition.

The following issues are fixed by this commit (some solved "on the
fly" now that I was touching the code anyway and it would have been
stupid to port the bugs):

#46
#100
#170
#205

What future changes does this commit enable?
============================================

The following items are not yet implemented, but this commit opens the
possibility of implementing these independently.

- The separation of the HTTP exposition allows the implementation of
  other exposition methods based on the Registry interface, as known
  from other Prometheus client libraries, e.g. sending the metrics to
  Graphite.
  Cf. #197

- The public `Registry` interface allows the implementation of
  convenience tools for testing metrics collection. Those tools can
  inspect the collected MetricFamily protobufs and compare them to
  expectation. Also, tests can use their own testing instance of a
  registry.
  Cf. #58

Notable non-goals of this commit
================================

Non-goals that will be tackled later
------------------------------------

The following two issues are quite closely connected to the changes in
this commit but the line has been drawn deliberately to address them
in later steps of the refurbishment:

- `InstrumentHandler` has many known problems. The plan is to create a
  saner way to conveniently intrument HTTP handlers and remove the old
  `InstrumentHandler` altogether. To keep breakage low for now, even
  the default handler to expose metrics is still using the old
  `InstrumentHandler`.
  Cf. #200

- There is work underway to make the whole handling of metric
  descriptors (`Desc`) more intuitive and transparent for the user
  (including an ability for less strict checking,
  cf. #47). That's
  quite invasive from the perspective of the internal code, namely the
  registry. I deliberately kept those changes out of this commit.

Non-goals that _might_ be tackled later
---------------------------------------

There is a strong and understandable urge to divide the `prometheus`
package into a number of sub-packages (like `registry`, `collectors`,
`http`, `metrics`, …). However, to not run into a multitude of
circular import chains, this would need to break every single existing
usage of the library. (As just one example, if the ubiquitious
`prometheus.MustRegister` (with more than 2,000 uses on GitHub alone)
is kept in the `prometheus` package, but the other registry concerns
go into a new `registry` package, then the `prometheus` package would
import the `registry` package (to call the actual register method),
while at the same time the `registry` package needs to import the
`prometheus` package to access `Collector`, `Metric`, `Desc` and
more. If we moved `MustRegister` into the `registry` package,
thousands of code lines would have to be fixed (which would be easy if
the world was a mono repo, but it is not).)

The main problem is really the top-level functions like
`MustRegister`, `Handler`, `Push`, …, which effectively pull
everything into one package. Those functions are however very
convenient for the easy and very frequent use-cases.

This problem has to be revisited later.

For now, I'm trying to keep the amount of exported names in the
package as low as possible (e.g. I unexported expvarCollector in this
commit because the NewExpvarCollector constructor is enough to export,
similar to the other exporters).

Non-goals that won't be tackled anytime soon
--------------------------------------------

Something that I have played with a lot is "streaming collection",
i.e. allow an implementation of the `Registry` interface that collects
metrics incrementally and serves them while doing so. As it has turned
out, this has many many issues and makes the `Registry` interface very
clunky. Eventually, I made the call that it is unlikely we will really
implement streaming collection, and making the interface more clunky
for something that might not even happen is really a big no-no. Note
that the `Registry` interface only creates the in-memory
representation of the metric family protobufs in one go. The
serializaton onto the wire can still be handled in a streaming
fashion.

What are the breaking changes?
==============================

- Signature of functions pushing to Pushgateway has changed to allow
  arbitrary grouping (long planned anyway, and now that I worked on it
  anyway, I did it,
  cf. #100).

- `SetMetricFamilyInjectionHook` is gone. A registry with a
  MetricFamily injection hook has to be created now with
  `NewRegistryWithInjectionHook`.

- `PanicOnCollectError` is gone. This behavior can now be configured
  when creating a custom HTTP handler.

- `EnableCollectChecks` is gone. A registry with those checks can now
  be created with `NewPedanticRegistry` (it is only ever used to
  test custom Collectors).

NOTICE

@@ -7,11 +7,6 @@ SoundCloud Ltd. (http://soundcloud.com/).
 
 The following components are included in this product:
 
-goautoneg
-http://bitbucket.org/ww/goautoneg
-Copyright 2011, Open Knowledge Foundation Ltd.
-See README.txt for license details.
-
 perks - a fork of https://github.com/bmizerany/perks
 https://github.com/beorn7/perks
 Copyright 2013-2015 Blake Mizerany, Björn Rabenstein

prometheus/collector.go

@@ -37,16 +37,16 @@ type Collector interface {
 	// executing this method, it must send an invalid descriptor (created
 	// with NewInvalidDesc) to signal the error to the registry.
 	Describe(chan<- *Desc)
-	// Collect is called by Prometheus when collecting metrics. The
-	// implementation sends each collected metric via the provided channel
-	// and returns once the last metric has been sent. The descriptor of
-	// each sent metric is one of those returned by Describe. Returned
-	// metrics that share the same descriptor must differ in their variable
-	// label values. This method may be called concurrently and must
-	// therefore be implemented in a concurrency safe way. Blocking occurs
-	// at the expense of total performance of rendering all registered
-	// metrics. Ideally, Collector implementations support concurrent
-	// readers.
+	// Collect is called by the Prometheus registry when collecting
+	// metrics. The implementation sends each collected metric via the
+	// provided channel and returns once the last metric has been sent. The
+	// descriptor of each sent metric is one of those returned by
+	// Describe. Returned metrics that share the same descriptor must differ
+	// in their variable label values. This method may be called
+	// concurrently and must therefore be implemented in a concurrency safe
+	// way. Blocking occurs at the expense of total performance of rendering
+	// all registered metrics. Ideally, Collector implementations support
+	// concurrent readers.
 	Collect(chan<- Metric)
 }
 

prometheus/doc.go

@@ -11,18 +11,14 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-// Package prometheus provides embeddable metric primitives for servers and
-// standardized exposition of telemetry through a web services interface.
+// Package prometheus provides metrics primitives to instrument code for
+// monitoring. It also offers a registry for metrics and ways to expose
+// registered metrics via an HTTP endpoint or push them to a Pushgateway.
 //
 // All exported functions and methods are safe to be used concurrently unless
 // specified otherwise.
 //
-// To expose metrics registered with the Prometheus registry, an HTTP server
-// needs to know about the Prometheus handler. The usual endpoint is "/metrics".
-//
-//     http.Handle("/metrics", prometheus.Handler())
-//
-// As a starting point a very basic usage example:
+// As a starting point, a very basic usage example:
 //
 //    package main
 //
@@ -44,6 +40,7 @@
 //    )
 //
 //    func init() {
+//      // Metrics have to be registered to be exposed:
 //    	prometheus.MustRegister(cpuTemp)
 //    	prometheus.MustRegister(hdFailures)
 //    }
@@ -52,6 +49,8 @@
 //    	cpuTemp.Set(65.3)
 //    	hdFailures.Inc()
 //
+//      // The Handler function provides a default handler to expose metrics
+//      // via an HTTP server. "/metrics" is the usual endpoint for that.
 //    	http.Handle("/metrics", prometheus.Handler())
 //    	http.ListenAndServe(":8080", nil)
 //    }
@@ -74,8 +73,8 @@
 // Those are all the parts needed for basic usage. Detailed documentation and
 // examples are provided below.
 //
-// Everything else this package offers is essentially for "power users" only. A
-// few pointers to "power user features":
+// Everything else this package and its sub-packages offer is essentially for
+// "power users" only. A few pointers to "power user features":
 //
 // All the various ...Opts structs have a ConstLabels field for labels that
 // never change their value (which is only useful under special circumstances,
@@ -84,9 +83,6 @@
 // The Untyped metric behaves like a Gauge, but signals the Prometheus server
 // not to assume anything about its type.
 //
-// Functions to fine-tune how the metric registry works: EnableCollectChecks,
-// PanicOnCollectError, Register, Unregister, SetMetricFamilyInjectionHook.
-//
 // For custom metric collection, there are two entry points: Custom Metric
 // implementations and custom Collector implementations. A Metric is the
 // fundamental unit in the Prometheus data model: a sample at a point in time
@@ -105,7 +101,24 @@
 // collection time, MetricVec to bundle custom Metrics into a metric vector
 // Collector, SelfCollector to make a custom Metric collect itself.
 //
-// A good example for a custom Collector is the ExpVarCollector included in this
+// A good example for a custom Collector is the expvarCollector included in this
 // package, which exports variables exported via the "expvar" package as
 // Prometheus metrics.
+//
+// The functions Register, Unregister, MustRegister, RegisterOrGet, and
+// MustRegisterOrGet all act on the default registry. They wrap other calls as
+// described in their doc comment. For advanced use cases, you can work with
+// custom registries (created by NewRegistry and similar) and call the wrapped
+// functions directly.
+//
+// The functions Handler and UninstrumentedHandler create an HTTP handler to
+// serve metrics from the default registry in the default way, which covers most
+// of the use cases. With HandlerFor, you can create a custom HTTP handler for
+// custom registries.
+//
+// The functions Push and PushAdd push the metrics from the default registry via
+// HTTP to a Pushgateway. With PushFrom and PushAddFrom, you can push the
+// metrics from custom registries. However, often you just want to push a
+// handfull of Collectors only. For that case, there are the convenience
+// functions PushCollectors and PushAddCollectors.
 package prometheus

prometheus/expvar.go → prometheus/expvar_collector.go

@@ -18,21 +18,21 @@ import (
 	"expvar"
 )
 
-// ExpvarCollector collects metrics from the expvar interface. It provides a
-// quick way to expose numeric values that are already exported via expvar as
-// Prometheus metrics. Note that the data models of expvar and Prometheus are
-// fundamentally different, and that the ExpvarCollector is inherently
-// slow. Thus, the ExpvarCollector is probably great for experiments and
-// prototying, but you should seriously consider a more direct implementation of
-// Prometheus metrics for monitoring production systems.
-//
-// Use NewExpvarCollector to create new instances.
-type ExpvarCollector struct {
+type expvarCollector struct {
 	exports map[string]*Desc
 }
 
-// NewExpvarCollector returns a newly allocated ExpvarCollector that still has
-// to be registered with the Prometheus registry.
+// NewExpvarCollector returns a newly allocated expvar Collector that still has
+// to be registered with a Prometheus registry.
+//
+// An expvar Collector collects metrics from the expvar interface. It provides a
+// quick way to expose numeric values that are already exported via expvar as
+// Prometheus metrics. Note that the data models of expvar and Prometheus are
+// fundamentally different, and that the expvar Collector is inherently slower
+// than native Prometheus metrics. Thus, the expvar Collector is probably great
+// for experiments and prototying, but you should seriously consider a more
+// direct implementation of Prometheus metrics for monitoring production
+// systems.
 //
 // The exports map has the following meaning:
 //
@@ -59,21 +59,21 @@ type ExpvarCollector struct {
 // sample values.
 //
 // Anything that does not fit into the scheme above is silently ignored.
-func NewExpvarCollector(exports map[string]*Desc) *ExpvarCollector {
-	return &ExpvarCollector{
+func NewExpvarCollector(exports map[string]*Desc) Collector {
+	return &expvarCollector{
 		exports: exports,
 	}
 }
 
 // Describe implements Collector.
-func (e *ExpvarCollector) Describe(ch chan<- *Desc) {
+func (e *expvarCollector) Describe(ch chan<- *Desc) {
 	for _, desc := range e.exports {
 		ch <- desc
 	}
 }
 
 // Collect implements Collector.
-func (e *ExpvarCollector) Collect(ch chan<- Metric) {
+func (e *expvarCollector) Collect(ch chan<- Metric) {
 	for name, desc := range e.exports {
 		var m Metric
 		expVar := expvar.Get(name)

prometheus/go_collector.go

@@ -17,7 +17,7 @@ type goCollector struct {
 
 // NewGoCollector returns a collector which exports metrics about the current
 // go process.
-func NewGoCollector() *goCollector {
+func NewGoCollector() Collector {
 	return &goCollector{
 		goroutines: NewGauge(GaugeOpts{
 			Namespace: "go",

prometheus/http.go

@@ -23,6 +23,37 @@ import (
 	"time"
 )
 
+// Handler returns the HTTP handler for the global Prometheus registry. It is
+// already instrumented with InstrumentHandler (using "prometheus" as handler
+// name). Usually the handler is used to handle the "/metrics" endpoint.
+//
+// Please note the issues described in the doc comment of InstrumentHandler. You
+// might want to consider using UninstrumentedHandler instead.
+func Handler() http.Handler {
+	// TODO
+	// return InstrumentHandler("prometheus", defRegistry)
+	return nil
+}
+
+// UninstrumentedHandler works in the same way as Handler, but the returned HTTP
+// handler is not instrumented. This is useful if no instrumentation is desired
+// (for whatever reason) or if the instrumentation has to happen with a
+// different handler name (or with a different instrumentation approach
+// altogether). See the InstrumentHandler example.
+func UninstrumentedHandler() http.Handler {
+	// TODO
+	// return defRegistry
+	return nil
+}
+
+func HandlerFor(r Registry, opts HandlerOpts) http.Handler {
+	return nil // TODO
+}
+
+type HandlerOpts struct {
+	// TODO check how http stdlib is done, error handling, logging
+}
+
 var instLabels = []string{"method", "code"}
 
 type nower interface {

prometheus/process_collector.go

@@ -28,7 +28,7 @@ type processCollector struct {
 // NewProcessCollector returns a collector which exports the current state of
 // process metrics including cpu, memory and file descriptor usage as well as
 // the process start time for the given process id under the given namespace.
-func NewProcessCollector(pid int, namespace string) *processCollector {
+func NewProcessCollector(pid int, namespace string) Collector {
 	return NewProcessCollectorPIDFn(
 		func() (int, error) { return pid, nil },
 		namespace,
@@ -43,7 +43,7 @@ func NewProcessCollector(pid int, namespace string) *processCollector {
 func NewProcessCollectorPIDFn(
 	pidFn func() (int, error),
 	namespace string,
-) *processCollector {
+) Collector {
 	c := processCollector{
 		pidFn:     pidFn,
 		collectFn: func(chan<- Metric) {},

prometheus/push.go

@@ -29,17 +29,25 @@ package prometheus
 // Note that all previously pushed metrics with the same job and instance will
 // be replaced with the metrics pushed by this call. (It uses HTTP method 'PUT'
 // to push to the Pushgateway.)
-func Push(job, instance, url string) error {
+func Push(job, instance, url string) error { // TODO grouping
 	return defRegistry.Push(job, instance, url, "PUT")
 }
 
 // PushAdd works like Push, but only previously pushed metrics with the same
 // name (and the same job and instance) will be replaced. (It uses HTTP method
 // 'POST' to push to the Pushgateway.)
-func PushAdd(job, instance, url string) error {
+func PushAdd(job, instance, url string) error { // TODO grouping
 	return defRegistry.Push(job, instance, url, "POST")
 }
 
+func PushFrom(r Registry, grouping ) error {
+	return nil // TODO
+}
+
+func PushAddFrom(r Registry, grouping ) error {
+	return nil // TODO
+}
+
 // PushCollectors works like Push, but it does not collect from the default
 // registry. Instead, it collects from the provided collectors. It is a
 // convenient way to push only a few metrics.