apiserver: add /introspection/... endpoints

[axw]

Description of change

Expose the introspection socket endpoints under
the "/introspection/" prefix in the API server.
Only users with controller superuser access, or
read access to the controller model, may access
these endpoints.

QA steps

1. juju bootstrap localhost
2. open browser to https://<controller-address>:17070/introspection/metrics
3. enter "user-admin" as username, admin password as password (obtain using juju gui --show-credentials)
4. juju add-user prometheus
5. juju change-user-password prometheus
6. juju grant prometheus read controller
7. set up prometheus:

$ sudo snap install prometheus
$ sudo bash -c "juju controller-config ca-cert | tail -n +2 | sed 's/^\s*//' > /var/snap/prometheus/current/juju.ca"
$ sudo vim /var/snap/prometheus/current/prometheus.yml, add the following scrape config:

scrape_configs:
  - job_name: 'juju'
    metrics_path: '/introspection/metrics'    
    scheme: 'https'                           
    static_configs:                           
      - targets: ['<controller-address>:17070']      
    basic_auth:                               
      username: 'user-prometheus'
      password: '<prometheus-password>'
    tls_config:
      ca_file: /var/snap/prometheus/current/juju.ca

8. Reload prometheus (e.g. sudo snap disable/enable prometheus)
9. Open up prometheus GUI and verify the target is being scraped OK (http://localhost:9090/targets)

Documentation changes

We should add web documentation on how to set up Prometheus, as described above.

[wallyworld]

Niiiiice

[axw]

$$merge$$

[jujubot]

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju