Support resuming from the last log when retrying

[tksm]

Fixes #229

This PR is to support resuming from the last log when retrying.

Here is the summary of the changes.

1. Always set PodLogOptions's Timestamps to true
   1. Print timestamps only when "--timestamps" is specified
2. Remember the last timestamp and line counts during this timestamp when printing log lines
   1. 📝 The timestamp in logs is RFC3339Nano, but PodLogOptions's SinceTime is RFC3339.
3. When resuming, set the last timestamp to PodLogOptions's SinceTime
   1. Skip lines we've already seen during the timestamp to avoid duplication

I found the rate limiter of retrying was not appropriate while I tested this feature, so I also tweaked it.

How to test

The below steps are the way to test the resuming feature by restarting the kube-apiserver.

1. Deploy a pod that prints line numbers

apiVersion: v1
kind: Pod
metadata:
  name: counter
spec:
  terminationGracePeriodSeconds: 0
  containers:
  - name: c
    command:
    - /bin/sh
    - -c
    - |
      N=1
      while true; do
        for _ in $(seq 5); do
          echo "line $N"
          N=$((N+1))
        done
        sleep 10
      done
    image: busybox:1.36.0

2. Run stern to watch the pod

dist/stern pod/counter

+ counter › c
counter c line 1
counter c line 2
counter c line 3
counter c line 4
counter c line 5

3. Restart the kube-apiserver in another terminal

If you use kind, you can restart the kube-apiserver by the command below.

docker exec -it kind-control-plane /bin/bash -c 'kill -9 $(pgrep kube-apiserver)'

The connection will be disconnected, but it will retry soon and resume logs from the last line.

counter c line 15
counter c line 16
counter c line 17
counter c line 18
counter c line 19
counter c line 20
- counter › c
failed to tail: unexpected EOF, will retry
+ counter › c
- counter › c
failed to tail: Get "https://127.0.0.1:64321/api/v1/namespaces/default/pods/counter/log?container=c&follow=true&sinceTime=2023-02-17T04%3A50%3A38Z&timestamps=true": EOF, will retry
+ counter › c
- counter › c
failed to tail: Get "https://127.0.0.1:64321/api/v1/namespaces/default/pods/counter/log?container=c&follow=true&sinceTime=2023-02-17T04%3A50%3A38Z&timestamps=true": EOF, will retry
+ counter › c
counter c line 21
counter c line 22
counter c line 23
counter c line 24
counter c line 25

[superbrothers]

I'll check this PR this weekend.

[superbrothers]

I prefer to use k8s.io/utils/pointer here.

Suggested change

	SinceSeconds: &sinceSeconds,
	SinceSeconds: pointer.Int64(int64(config.Since.Seconds())),

[tksm]

I fixed it in 51f9032 👍

[tksm]

Thank you for the review!