Add CRI container log format support back for elastic search.

[Random-Liu]

The CRI container log format support was removed accidentally in #58525. This PR adds that back.

I've tested it, and it works:

SSSSS
------------------------------
[sig-instrumentation] Cluster level logging using Elasticsearch [Feature:Elasticsearch] 
  should check that logs from containers are ingested into Elasticsearch
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/instrumentation/logging/elasticsearch/basic.go:39
[BeforeEach] [sig-instrumentation] Cluster level logging using Elasticsearch [Feature:Elasticsearch]
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:141
STEP: Creating a kubernetes client
Mar 28 08:09:01.724: INFO: >>> kubeConfig: /home/lantaol/.kube/config
STEP: Building a namespace api object
Mar 28 08:09:02.952: INFO: No PodSecurityPolicies found; assuming PodSecurityPolicy is disabled.
STEP: Waiting for a default service account to be provisioned in namespace
[BeforeEach] [sig-instrumentation] Cluster level logging using Elasticsearch [Feature:Elasticsearch]
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/instrumentation/logging/elasticsearch/basic.go:32
[It] should check that logs from containers are ingested into Elasticsearch
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/instrumentation/logging/elasticsearch/basic.go:39
Mar 28 08:09:02.988: INFO: Checking the Elasticsearch service exists.
Mar 28 08:09:03.025: INFO: Checking to make sure the Elasticsearch pods are running
Mar 28 08:09:03.066: INFO: Checking to make sure we are talking to an Elasticsearch service.
Mar 28 08:09:03.176: INFO: Checking health of Elasticsearch service.
Mar 28 08:09:03.299: INFO: Starting repeating logging pod synthlogger
STEP: Waiting for logs to ingest
Mar 28 08:09:17.420: INFO: Sending a search request to Elasticsearch with the following query: kubernetes.pod_name:synthlogger AND kubernetes.namespace_name:e2e-tests-es-logging-pqlx7
Mar 28 08:09:27.420: INFO: Sending a search request to Elasticsearch with the following query: kubernetes.pod_name:synthlogger AND kubernetes.namespace_name:e2e-tests-es-logging-pqlx7
Mar 28 08:09:37.420: INFO: Sending a search request to Elasticsearch with the following query: kubernetes.pod_name:synthlogger AND kubernetes.namespace_name:e2e-tests-es-logging-pqlx7
Mar 28 08:09:47.420: INFO: Sending a search request to Elasticsearch with the following query: kubernetes.pod_name:synthlogger AND kubernetes.namespace_name:e2e-tests-es-logging-pqlx7
Mar 28 08:09:57.420: INFO: Sending a search request to Elasticsearch with the following query: kubernetes.pod_name:synthlogger AND kubernetes.namespace_name:e2e-tests-es-logging-pqlx7
Mar 28 08:10:07.420: INFO: Sending a search request to Elasticsearch with the following query: kubernetes.pod_name:synthlogger AND kubernetes.namespace_name:e2e-tests-es-logging-pqlx7
[AfterEach] [sig-instrumentation] Cluster level logging using Elasticsearch [Feature:Elasticsearch]
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:142
Mar 28 08:10:07.607: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "e2e-tests-es-logging-pqlx7" for this suite.
Mar 28 08:10:57.758: INFO: Waiting up to 30s for server preferred namespaced resources to be successfully discovered
Mar 28 08:11:00.046: INFO: namespace: e2e-tests-es-logging-pqlx7, resource: bindings, ignored listing per whitelist
Mar 28 08:11:00.338: INFO: namespace e2e-tests-es-logging-pqlx7 deletion completed in 52.693713026s

• [SLOW TEST:118.614 seconds]
[sig-instrumentation] Cluster level logging using Elasticsearch [Feature:Elasticsearch]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/instrumentation/common/framework.go:23
  should check that logs from containers are ingested into Elasticsearch
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/instrumentation/logging/elasticsearch/basic.go:39
------------------------------
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSMar 28 08:11:00.346: INFO: Running AfterSuite actions on all node
Mar 28 08:11:00.346: INFO: Running AfterSuite actions on node 1

Ran 1 of 845 Specs in 123.981 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 844 Skipped PASS

Ginkgo ran 1 suite in 2m4.323020647s
Test Suite Passed
2018/03/28 08:11:00 process.go:152: Step './hack/ginkgo-e2e.sh --ginkgo.focus=Cluster\slevel\slogging\susing\sElasticsearch' finished in 2m5.943972428s
2018/03/28 08:11:00 e2e.go:83: Done

Mark 1.10, because this is a regression for CRI container runtimes in 1.10.

The original support was added in 1.9. #54777

Release note:

Fix a bug that fluentd doesn't inject container logs for CRI container runtimes (containerd, cri-o etc.) into elasticsearch on GCE.

[crassirostris]

/lgtm
/approve

[crassirostris]

/lgtm cancel

@Random-Liu Was is intended for master branch or 1.10 branch? I see the base currently is master, but the milestone is for 1.10

Please clarify

[Random-Liu]

@crassirostris We need to do cherrypick later. :)

[Random-Liu]

@crassirostris Ping! :)

[crassirostris]

/lgtm
/retest

@crassirostris We need to do cherrypick later. :)

This doesn't look good. I think it's better to merge this PR in the current milestone and then create a cherry-pick with 1.10 milestone. Making the original PR with 1.10 milestone doesn't make sense to me

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: crassirostris, Random-Liu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cluster/addons/fluentd-elasticsearch/OWNERS [crassirostris]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-github-robot]

[MILESTONENOTIFIER] Milestone Pull Request Needs Approval

@Random-Liu @coffeepac @crassirostris @kubernetes/sig-instrumentation-misc

Action required: This pull request must have the status/approved-for-milestone label applied by a SIG maintainer.

Pull Request Labels

• sig/instrumentation: Pull Request will be escalated to these SIGs if needed.
• priority/critical-urgent: Never automatically move pull request out of a release milestone; continually escalate to contributor and SIG through all available channels.
• kind/bug: Fixes a bug discovered during the current release.

Help

• Additional instructions
• Commands for setting labels

[k8s-github-robot]

/test all

Tests are more than 96 hours old. Re-running tests.

[k8s-ci-robot]

@Random-Liu: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-kubernetes-e2e-gce	db110f1	link	/test pull-kubernetes-e2e-gce

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[k8s-github-robot]

Automatic merge from submit-queue (batch tested with PRs 61818, 61800). If you want to cherry-pick this change to another branch, please follow the instructions here.

[Random-Liu]

@piosz @danielmk @loburm Can you approve the cherrypick and milestone?
/cc @kubernetes/sig-instrumentation-pr-reviews

[piosz]

/status approved-for-milestone