Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Start tracking k8s liveness probes #1320

Merged
merged 2 commits into from
Jun 14, 2019
Merged

Start tracking k8s liveness probes #1320

merged 2 commits into from
Jun 14, 2019

Commits on Jun 11, 2019

  1. Remove unneeded class forward declaration 

    Not used in this header file, so shouldn't be needed.
    @mstemm
    mstemm committed Jun 11, 2019
    Configuration menu
    Copy the full SHA
    f58be37 View commit details
    Browse the repository at this point in the history

  2. Start tracking k8s liveness/readiness probes 

    K8s has a similar but not identical method as docker for container
health checks. They are called liveness/readiness probes and are a part
of the pod specification, and not a part of the image.

Luckily, the pod configuration *is* a part of the container metadata as
stringified json, with a label
"annotation.kubectl.kubernetes.io/last-applied-configuration", so we can
use that label to identify liveness/readiness probes.

New methods in the docker container resolver handle parsing the pod
specification (and healthcheck info) out of the container json and
creating health probes from them.

A new class sinsp_container_info::container_health_probe represents one
of these health probes. It has a probe
type (healthcheck/liveness/readiness), the executable and arguments, and
methods to serialize/unserialize from json. The serialization doesn't
preserve the original container json--they only keep the exe + args.

The container info now has a list of possible health probe objects and
iterates over them when dumping the container to json.

For threads, switch everything to use a threadinfo category instead of a
simple bool for has healthcheck. The possible values for the category
are:

  - CAT_NONE: no specific category
  - CAT_CONTAINER: a process run in a container and *not* any
    of the following more specific categories.
  - CAT_HEALTHCHECK: part of a container healthcheck
  - CAT_LIVENESS_PROBE: part of a k8s liveness probe
  - CAT_READINESS_PROBE: part of a k8s readiness probe

Identify_healthcheck becomes identify_category() but
otherwise behaves the same (passing categories down and checking the
args list otherwise).

The logic in indentify_healthcheck tries to handle the common cases
first:

 - not running in a container or container info not present: CAT_NONE
 - vpid=1: CAT_CONTAINER
 - inherit categories other than CAT_NONE directly from parent

If those fail, the more expensive steps of matching against the health
check args and possibly traversing the parent state are done.

The filterchecks aren't quite as generic as the threadinfo categories to
keep the filtering simple. A new field
proc.is_container_{liveness,readiness}_probe checks for k8s
liveness/readiness probes, and container.{liveness,readiness}_probe
prints the exe + args.
    @mstemm
    mstemm committed Jun 11, 2019
    Configuration menu
    Copy the full SHA
    21f1139 View commit details
    Browse the repository at this point in the history