internal/store/pod.go: Only create last_terminated containers series if containers are terminated state

[lilic]

See previous PRs for backgrounds. #1381 and #1378

kube_pod_init_container_status_last_terminated_reason and kube_pod_container_status_last_terminated_reason metrics are changed.

cc @brancz as you reviewed the others, this is the last one 🎉

[brancz]

/lgtm

[brancz]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: brancz, lilic

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [brancz,lilic]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[jutley]

@lilic I added the kube_pod_container_status_last_terminated_reason metric in #535 because there was no good way to alert when a container was running out of memory. Since the container restarts automatically, it is usually not scraped when it is in the terminated state. Using the last terminated reason fixed this, because the metric continues to show that an OOM happened even when the container starts back up.

If this metric is only going to be reported when the container is in a terminated state, it seems like this use case will no longer be supported. Is there another way that we will be able to reliable alert on OOMs on kube-state-metrics:v2.0?