Add kubectl log --previous support to view last terminated container log #7973

dchen1107 · 2015-05-08T17:53:07Z

View current running container:

$ cluster/kubectl.sh log kube-dns-u6iwr etcd
Starting cluster using os distro: debian
current-context: "golden-system-455_kubernetes"
Running: cluster/../cluster/gce/../../cluster/../_output/dockerized/bin/linux/amd64/kubectl log kube-dns-u6iwr etcd
2015-05-08T17:19:52.082453389Z 2015/05/08 17:19:52 etcd: no data-dir provided, using default data-dir ./default.etcd
2015-05-08T17:19:52.082507932Z 2015/05/08 17:19:52 etcd: listening for peers on http://localhost:2380
2015-05-08T17:19:52.082513303Z 2015/05/08 17:19:52 etcd: listening for peers on http://localhost:7001
2015-05-08T17:19:52.082516618Z 2015/05/08 17:19:52 etcd: listening for client requests on http://127.0.0.1:2379
2015-05-08T17:19:52.082519759Z 2015/05/08 17:19:52 etcd: listening for client requests on http://127.0.0.1:4001
2015-05-08T17:19:52.082523120Z 2015/05/08 17:19:52 etcdserver: datadir is valid for the 2.0.1 format
2015-05-08T17:19:52.082526742Z 2015/05/08 17:19:52 etcdserver: name = default
2015-05-08T17:19:52.082530076Z 2015/05/08 17:19:52 etcdserver: data dir = default.etcd
2015-05-08T17:19:52.082533163Z 2015/05/08 17:19:52 etcdserver: member dir = default.etcd/member
2015-05-08T17:19:52.082536440Z 2015/05/08 17:19:52 etcdserver: heartbeat = 100ms
2015-05-08T17:19:52.082539361Z 2015/05/08 17:19:52 etcdserver: election = 1000ms
2015-05-08T17:19:52.082542194Z 2015/05/08 17:19:52 etcdserver: snapshot count = 10000
2015-05-08T17:19:52.082545096Z 2015/05/08 17:19:52 etcdserver: advertise client URLs = http://127.0.0.1:2379,http://127.0.0.1:4001
2015-05-08T17:19:52.082553872Z 2015/05/08 17:19:52 etcdserver: initial advertise peer URLs = http://localhost:2380,http://localhost:7001
2015-05-08T17:19:52.082557335Z 2015/05/08 17:19:52 etcdserver: initial cluster = default=http://localhost:2380,default=http://localhost:7001
2015-05-08T17:19:52.082561674Z 2015/05/08 17:19:52 etcdserver: start member 6a5871dbdd12c17c in cluster f68652439e3f8f2a
2015-05-08T17:19:52.082565037Z 2015/05/08 17:19:52 raft: 6a5871dbdd12c17c became follower at term 0
2015-05-08T17:19:52.082568494Z 2015/05/08 17:19:52 raft: newRaft 6a5871dbdd12c17c [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2015-05-08T17:19:52.082571870Z 2015/05/08 17:19:52 raft: 6a5871dbdd12c17c became follower at term 1
2015-05-08T17:19:52.135010625Z 2015/05/08 17:19:52 etcdserver: added local member 6a5871dbdd12c17c [http://localhost:2380 http://localhost:7001] to cluster f68652439e3f8f2a
2015-05-08T17:19:53.778680605Z 2015/05/08 17:19:53 raft: 6a5871dbdd12c17c is starting a new election at term 1
2015-05-08T17:19:53.782947933Z 2015/05/08 17:19:53 raft: 6a5871dbdd12c17c became candidate at term 2
2015-05-08T17:19:53.782958531Z 2015/05/08 17:19:53 raft: 6a5871dbdd12c17c received vote from 6a5871dbdd12c17c at term 2
2015-05-08T17:19:53.782962231Z 2015/05/08 17:19:53 raft: 6a5871dbdd12c17c became leader at term 2
2015-05-08T17:19:53.782965857Z 2015/05/08 17:19:53 raft.node: 6a5871dbdd12c17c elected leader 6a5871dbdd12c17c at term 2
2015-05-08T17:19:53.784006311Z 2015/05/08 17:19:53 etcdserver: published {Name:default ClientURLs:[http://127.0.0.1:2379 http://127.0.0.1:4001]} to cluster f68652439e3f8f2a

View last terminated container:

$ cluster/kubectl.sh log --previous kube-dns-u6iwr etcd
Starting cluster using os distro: debian
current-context: "golden-system-455_kubernetes"
Running: cluster/../cluster/gce/../../cluster/../_output/dockerized/bin/linux/amd64/kubectl log --previous kube-dns-u6iwr etcd
2015-05-08T01:04:04.531822712Z 2015/05/08 01:04:04 etcdserver: start to snapshot (applied: 30003, lastsnap: 20002)
2015-05-08T01:04:04.539799516Z 2015/05/08 01:04:04 etcdserver: compacted log at index 30003
2015-05-08T01:04:04.567473474Z 2015/05/08 01:04:04 etcdserver: saved snapshot at index 30003
2015-05-08T02:26:45.031556653Z 2015/05/08 02:26:45 etcdserver: start to snapshot (applied: 40004, lastsnap: 30003)
2015-05-08T02:26:45.034289317Z 2015/05/08 02:26:45 etcdserver: compacted log at index 40004
2015-05-08T02:26:45.063427077Z 2015/05/08 02:26:45 etcdserver: saved snapshot at index 40004
2015-05-08T03:49:23.031419059Z 2015/05/08 03:49:23 etcdserver: start to snapshot (applied: 50005, lastsnap: 40004)
2015-05-08T03:49:23.034633854Z 2015/05/08 03:49:23 etcdserver: compacted log at index 50005
2015-05-08T03:49:23.063567111Z 2015/05/08 03:49:23 etcdserver: saved snapshot at index 50005
2015-05-08T03:49:41.240453029Z 2015/05/08 03:49:41 filePurge: successfully removed file default.etcd/member/wal/0000000000000000-0000000000000000.wal
2015-05-08T05:12:01.031479334Z 2015/05/08 05:12:01 etcdserver: start to snapshot (applied: 60006, lastsnap: 50005)
2015-05-08T05:12:01.035340614Z 2015/05/08 05:12:01 etcdserver: compacted log at index 60006
2015-05-08T05:12:01.063295479Z 2015/05/08 05:12:01 etcdserver: saved snapshot at index 60006
2015-05-08T05:12:11.283689186Z 2015/05/08 05:12:11 filePurge: successfully removed file default.etcd/member/snap/0000000000000002-0000000000002711.snap
2015-05-08T05:12:11.283939737Z 2015/05/08 05:12:11 filePurge: successfully removed file default.etcd/member/wal/0000000000000001-0000000000002712.wal
2015-05-08T06:34:41.531536560Z 2015/05/08 06:34:41 etcdserver: start to snapshot (applied: 70007, lastsnap: 60006)
2015-05-08T06:34:41.535609973Z 2015/05/08 06:34:41 etcdserver: compacted log at index 70007
2015-05-08T06:34:41.563516650Z 2015/05/08 06:34:41 etcdserver: saved snapshot at index 70007
2015-05-08T06:35:11.327789880Z 2015/05/08 06:35:11 filePurge: successfully removed file default.etcd/member/snap/0000000000000002-0000000000004e22.snap
2015-05-08T06:35:11.328059550Z 2015/05/08 06:35:11 filePurge: successfully removed file default.etcd/member/wal/0000000000000002-0000000000004e23.wal
2015-05-08T07:57:19.531724879Z 2015/05/08 07:57:19 etcdserver: start to snapshot (applied: 80008, lastsnap: 70007)
2015-05-08T07:57:19.536553856Z 2015/05/08 07:57:19 etcdserver: compacted log at index 80008
2015-05-08T07:57:19.563253209Z 2015/05/08 07:57:19 etcdserver: saved snapshot at index 80008
2015-05-08T07:57:41.373356734Z 2015/05/08 07:57:41 filePurge: successfully removed file default.etcd/member/snap/0000000000000002-0000000000007533.snap
2015-05-08T07:57:41.373606942Z 2015/05/08 07:57:41 filePurge: successfully removed file default.etcd/member/wal/0000000000000003-0000000000007534.wal
2015-05-08T09:20:00.031505826Z 2015/05/08 09:20:00 etcdserver: start to snapshot (applied: 90009, lastsnap: 80008)
2015-05-08T09:20:00.036393165Z 2015/05/08 09:20:00 etcdserver: compacted log at index 90009
2015-05-08T09:20:00.063395366Z 2015/05/08 09:20:00 etcdserver: saved snapshot at index 90009
2015-05-08T09:20:11.428310827Z 2015/05/08 09:20:11 filePurge: successfully removed file default.etcd/member/snap/0000000000000002-0000000000009c44.snap
2015-05-08T09:20:11.428828475Z 2015/05/08 09:20:11 filePurge: successfully removed file default.etcd/member/wal/0000000000000004-0000000000009c45.wal
2015-05-08T10:42:38.031654974Z 2015/05/08 10:42:38 etcdserver: start to snapshot (applied: 100010, lastsnap: 90009)

If the last terminated container is not exist:

$ cluster/kubectl.sh log --previous etcd-server-kubernetes-master
current-context: "golden-system-455_kubernetes"
Running: cluster/../cluster/gce/../../cluster/../_output/dockerized/bin/linux/amd64/kubectl log --previous etcd-server-kubernetes-master
Internal Error: previous terminated container "etcd-container" not found in pod

cc/ @vishh

vishh · 2015-05-09T00:20:19Z

contrib/completions/bash/kubectl

@@ -404,6 +404,8 @@ _kubectl_log()
    flags+=("--help")
    flags+=("-h")
    flags+=("--interactive")
+    flags+=("--previous")


Question: Can this be last instead of previous? And do we need the -p short flag?

-p is supported already.

dchen1107 · 2015-05-09T00:22:38Z

--previous was chosen by our UX lead: @bgrant0607 over --last :-)

vishh · 2015-05-09T00:23:03Z

pkg/kubelet/dockertools/manager.go

@@ -298,7 +298,7 @@ func (dm *DockerManager) inspectContainer(dockerID, containerName, tPath string)
 			if found {
 				data, err := ioutil.ReadFile(path)
 				if err != nil {
-					glog.Errorf("Error on reading termination-log %s: %v", path, err)
+					//glog.Errorf("Error on reading termination-log %s: %v", path, err)


I assume this change is unintentional.

Ahh, someone breaks that feature which generates tons of logging messages. Fixed it. Thanks!

vishh · 2015-05-09T00:26:46Z

Just a few minor nits. Overall LGTM. Thanks for this feature @dchen1107. This will be very useful.

dchen1107 · 2015-05-11T17:15:39Z

Addressed your comments, PTAL?

vishh · 2015-05-11T17:18:23Z

LGTM. WIll merge once tests pass.

Add kubectl log --previous support to view last terminated container log

derekwaynecarr · 2015-05-12T00:55:04Z

thanks for adding this and sorry I was not able to review it earlier. It lgtm and and is a big help when actually building apps on Kube

googlebot added the cla: yes label May 8, 2015

dchen1107 force-pushed the podstatus branch 2 times, most recently from b249d58 to 7b02238 Compare May 8, 2015 18:33

bgrant0607 assigned derekwaynecarr May 8, 2015

dchen1107 force-pushed the podstatus branch from 7b02238 to 22a9c92 Compare May 8, 2015 19:35

vishh reviewed May 9, 2015
View reviewed changes

dchen1107 force-pushed the podstatus branch from 22a9c92 to e6c2c88 Compare May 9, 2015 00:27

dchen1107 added 2 commits May 11, 2015 10:14

Introduce --previous option to kubectl log

ecaf087

Introduce Previous as PodLogOptions

ffa5947

dchen1107 force-pushed the podstatus branch from e6c2c88 to 75115db Compare May 11, 2015 17:14

Add support to pull log for last terminated container

86479cc

dchen1107 force-pushed the podstatus branch from 75115db to 86479cc Compare May 11, 2015 19:54

vmarmol added a commit that referenced this pull request May 12, 2015

Merge pull request #7973 from dchen1107/podstatus

4302934

Add kubectl log --previous support to view last terminated container log

vmarmol merged commit 4302934 into kubernetes:master May 12, 2015

bgrant0607 mentioned this pull request Jun 1, 2015

Feature request: make it easier to fetch the logs of a crashed container #8526

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add kubectl log --previous support to view last terminated container log #7973

Add kubectl log --previous support to view last terminated container log #7973

dchen1107 commented May 8, 2015

vishh May 9, 2015

dchen1107 May 9, 2015

dchen1107 commented May 9, 2015

vishh May 9, 2015

dchen1107 May 9, 2015

vishh commented May 9, 2015

dchen1107 commented May 11, 2015

vishh commented May 11, 2015

derekwaynecarr commented May 12, 2015

Add kubectl log --previous support to view last terminated container log #7973

Add kubectl log --previous support to view last terminated container log #7973

Conversation

dchen1107 commented May 8, 2015

vishh May 9, 2015

Choose a reason for hiding this comment

dchen1107 May 9, 2015

Choose a reason for hiding this comment

dchen1107 commented May 9, 2015

vishh May 9, 2015

Choose a reason for hiding this comment

dchen1107 May 9, 2015

Choose a reason for hiding this comment

vishh commented May 9, 2015

dchen1107 commented May 11, 2015

vishh commented May 11, 2015

derekwaynecarr commented May 12, 2015