Load logs in reverse order or only load the last N lines

[passcod]

Describe the problem/challenge you have

When opening the log view, all the logs from the start of time are streamed and displayed. This is slow, so if a pod has been running for a few days it can take half an hour for the latest logs, i.e. what I'm interested in, to appear.

Describe the solution you'd like

Two solutions I can see:

• loading the last N lines first, and then backfilling the rest
• a switch to load only the last N lines (plus any future lines, à la tail -fn N)

Environment:

• Octant version (use octant version):
  Version: 0.14.1
  Git commit: 51ce7f6
  Built: 2020-07-20T23:52:37Z

• Kubernetes version (use kubectl version):
  Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"e6503f8d8f769ace2f338794c914a96fc335df0f", GitTreeState:"archive", BuildDate:"2020-07-01T16:28:46Z", GoVersion:"go1.14.4", Compiler:"gc", Platform:"linux/amd64"}
  Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"e6503f8d8f769ace2f338794c914a96fc335df0f", GitTreeState:"clean", BuildDate:"2020-06-26T03:39:24Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

• OS (macOS 10.15, Windows 10, Ubuntu 19.10 etc):
  Archlinux

[wwitzel3]

Thank you for reporting this.

My thoughts:

Add since support to our current log streaming.
Default that to the last 15 or 30 minutes.
Provide a drop down in the log viewer with some pre-defined options, but also allow it to take custom input.

• 15 minutes
• 30 minutes
• 1 hour
• 3 hours
• 5 hours

input would be something like: 5m, 12h, 1d, all

[passcod]

Oh, time is an interesting concept if it can be done, yeah

[wwitzel3]

Yeah, I was thinking time mainly because that is usually a better measure of what I want to see when I'm dealing with random logs. When I'm debugging, I don't know if the last 20 or even 100 lines will have what I want, depending on verbosity, but I know that the error happened in the last 5 minutes.

But happy to entertain other approaches.

[mklanjsek]

Another request for this from our slack channel:
https://kubernetes.slack.com/archives/CM37M9FCG/p1599204923030200

[mklanjsek]

Add since support to our current log streaming.

Even better, we should implement more general filtering by time by defining the startTime and endTime to our logging interface. Quite often you want to see what exactly happened around specific time when an incident occurred in the past, e.g see logs for 5 minute interval from a week ago.

The default should probably be last 30 minutes.