Preserve log data for destroyed pod

[mnaughto]

Is there any way that things could be tweaked so that I can go into Browse->Pods->(frequently crashing pod)->Logs and view the latest log or a history of logs between restarts? This would make it easier to determine the cause of a problem.

Right now, the only way to see the log of a crashing pod is to scale it to zero, scale it up, click on the pod count, click on the pod, and click on logs, all hopefully before it restarts again.

Ideally, I'd be able to view either a history of failure logs or the most recent failure log, rather than getting the message saying "An error occurred loading the log."

Does this all make sense?

[deads2k]

Take a look at oc logs -p. I think it does exactly what you want, "view logs for the previous instance of the container".

[mnaughto]

Got it. It'd be nice if that could make its way into the web interface at some point (I guess that's what I'm requesting here).

[deads2k]

Got it. It'd be nice if that could make its way into the web interface at some point (I guess that's what I'm requesting here).

/cc @jwforres

[jwforres]

There is a k8s issue open related to this, the API needs to add a flag that will always return the logs for the latest container that was run / is running, the web console will update to always send that flag.

[benjaminapetersen]

@jwforres do you remember which issue? I took a stab at finding it to add a reference here but was not able to land on it.

[jwforres]

@benjaminapetersen i dont, @smarterclayton mentioned there was one, but not what it was

[smarterclayton]

Look under my issues in Kube with label usability

[mnaughto]

This is all I could find that seemed related: kubernetes/kubernetes#13780

There's also this (linked to in that pull request): #3943

[jwforres]

Maybe this? kubernetes/kubernetes#15826

[jwforres]

according to @smarterclayton the fix is that the API server is going to return the latest available container log, whether that container has already died or is currently running. so we'll need that fix rebased from kube

[spadgett]

Looks like this is fixed after the last rebase.