Enables Series API in loki

[owen-d]

What

This PR implements a series api equivalent in loki, which enables resolving log streams from a list of matchers.

Examples

$ curl -s "http://localhost:3100/loki/api/v1/series" --data-urlencode 'match={container_name=~"prometheus.*", component="server"}' --data-urlencode 'match={app="loki"}' | jq '.'
{
  "status": "success",
  "data": [
    {
      "container_name": "loki",
      "app": "loki",
      "stream": "stderr",
      "filename": "/var/log/pods/default_loki-stack-0_50835643-1df0-11ea-ba79-025000000001/loki/0.log",
      "name": "loki",
      "job": "default/loki",
      "controller_revision_hash": "loki-stack-757479754d",
      "statefulset_kubernetes_io_pod_name": "loki-stack-0",
      "release": "loki-stack",
      "namespace": "default",
      "instance": "loki-stack-0"
    },
    {
      "chart": "prometheus-9.3.3",
      "container_name": "prometheus-server-configmap-reload",
      "filename": "/var/log/pods/default_loki-stack-prometheus-server-696cc9ddff-87lmq_507b1db4-1df0-11ea-ba79-025000000001/prometheus-server-configmap-reload/0.log",
      "instance": "loki-stack-prometheus-server-696cc9ddff-87lmq",
      "pod_template_hash": "696cc9ddff",
      "app": "prometheus",
      "component": "server",
      "heritage": "Tiller",
      "job": "default/prometheus",
      "namespace": "default",
      "release": "loki-stack",
      "stream": "stderr"
    },
    {
      "app": "prometheus",
      "component": "server",
      "filename": "/var/log/pods/default_loki-stack-prometheus-server-696cc9ddff-87lmq_507b1db4-1df0-11ea-ba79-025000000001/prometheus-server/0.log",
      "release": "loki-stack",
      "namespace": "default",
      "pod_template_hash": "696cc9ddff",
      "stream": "stderr",
      "chart": "prometheus-9.3.3",
      "container_name": "prometheus-server",
      "heritage": "Tiller",
      "instance": "loki-stack-prometheus-server-696cc9ddff-87lmq",
      "job": "default/prometheus"
    }
  ]
}

Problems/Prior art

Chunk involvement

We currently need to iterate through chunks for this feature because labels are stored inside chunks and not in indices.

How Prometheus does it

Prometheus does this in a very similar way, by querying for all matching series within a time range and plucking the labels.

Possible improvements

SeriesIDs as uniqueness

Starting in the v9 schema, SeriesIDs are encoded in the index. There's even unexported functions lookupSeriesByMetricNameMatchers and lookupChunksBySeries to support it. Assuming that a sha256 of the labels is reasonable as a measure of uniqueness (seems so to me), we could make the upstream change and implement it this way. Then, we'd only need to pull one chunk per series instead of every chunk in the time range.

I avoided parsing the seriesID out of the chunk's key. This may be feasible, if a bit hacky, but would allow us to fetch only one chunk per stream.

Storing labels in indices

Labels could also be stored in the indices, removing the need to pull any chunks from storage.

Related

closes #113

[cyriltovena]

/cc @davkal

[davkal]

Cool! Let me know when this is merged.

[davkal]

From a consumer point of view, I need a structure that ends up like this:

[
  {"label": "label1", "values": ["value1-1", "value1-2", ...]},
  {"label": "label2", "values":  ["value2-1"]},
]

to suggest the right label sub tree at this caret entry point: {foo="bar",|}.

This is something I can create from the result example above, but perhaps it's worth adding a second API endpoint that gives me straight up what I need :)

[cyriltovena]

We won't change this path, for compatibility with Prometheus but we can have another path for what you need. Any suggestions of a good path for this?

[davkal]

We won't change this path, for compatibility with Prometheus but we can have another path for what you need. Any suggestions of a good path for this?

Effectively I'm querying for labels, so it should be some sub path of the labels api?

Edit: But I don't want to hold up this PR :P

[cyriltovena]

LGTM, with one comment.

[rfratto]

Great work @owen-d! This is looking good. Aside from my comments, I'd also like to see the API doc updated to reference the new endpoint.

[rfratto]

LGTM, but I'd still like to see docs/api.md updated.

[pracucci]

Very good job @owen-d! Unfortunately I didn't have time to review it before it was merged, anyway LGTM. I just noticed that the series deduplication suffers hash collisions (which is a thing we're trying to move away from in Cortex and I guess is desiderable to not have in Loki too, but I agree it's not easy to trigger an xxhash collision).

[pracucci]

What does expecting su mean?