Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"kubectl logs --selector" ignores "--tail=-1" #74900

Closed
adkerr opened this issue Mar 4, 2019 · 8 comments

Comments

Projects
None yet
6 participants
@adkerr
Copy link

commented Mar 4, 2019

What happened:

When attempting to get all log lines using kubectl logs -l app=my.app --tail=-1 only 10 lines of logs are returned

What you expected to happen:

All log lines should be returned

How to reproduce it (as minimally and precisely as possible):

kubectl logs -l app=my.app --tail=-1

Anything else we need to know?:

This was introduced in ea7065d in an attempt to fix another issue where selectors would ignore tail values !=-1. I think the problem here is the logic cannot tell if the -1 was set explicitly by the command line or is simply the initial default value.

Environment:

  • Kubernetes version (use kubectl version): 1.13.3
  • Cloud provider or hardware configuration: Kubeadm
  • OS (e.g: cat /etc/os-release): 18.04.2 LTS (Bionic Beaver)
  • Kernel (e.g. uname -a): Linux ubuntu 4.15.0-45-generic #48-Ubuntu SMP Tue Jan 29 16:28:13 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools:
  • Others:

@adkerr adkerr added the kind/bug label Mar 4, 2019

@adkerr

This comment has been minimized.

Copy link
Author

commented Mar 4, 2019

/sig cli

@liucimin

This comment has been minimized.

Copy link
Contributor

commented Mar 5, 2019

Maybe is not a bug.
See the kubectl help.

--tail=-1: Lines of recent log file to display. Defaults to -1 with no selector, showing all log lines otherwise
10, if a selector is provided.

I think i mean a selector is provided the --tail should not use -1 ?

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Mar 5, 2019

@JishanXing

This comment has been minimized.

Copy link
Contributor

commented Mar 5, 2019

Maybe is not a bug.
See the kubectl help.

--tail=-1: Lines of recent log file to display. Defaults to -1 with no selector, showing all log lines otherwise
10, if a selector is provided.

I think i mean a selector is provided the --tail should not use -1 ?

Thanks for you reply.

There are some background in below tickets, --tail=-1 is one of the option. Please check.

#51027
#35335

@adkerr

This comment has been minimized.

Copy link
Author

commented Mar 7, 2019

@liucimin The help indicates that the default is 10 if the selector is used. But if someone explicitly
asks for -1 it should still work, at least in my opinion.

@mikespokefire

This comment has been minimized.

Copy link

commented Mar 13, 2019

Encountering similar behaviour to this myself. Is there any chance whatever fixes get applied can also be applied to the 1.12.x branch? Since that also has strange behaviour around logs with label selectors and setting --tail. See https://github.com/kubernetes/kubernetes/blob/v1.12.6/pkg/kubectl/cmd/logs.go#L180-L184

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Mar 26, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 1, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 8, 2019

@JishanXing

This comment has been minimized.

Copy link
Contributor

commented Apr 10, 2019

This ticket has two issues:
First, It's said "TailLines must be greater than or equal to 0", however we can specify -1.
kubectl logs -l app=test --tail=-3
error: TailLines must be greater than or equal to 0

kubectl logs -l app=test --tail=-1
12462: Wed Apr 10 06:51:34 UTC 2019
12463: Wed Apr 10 06:51:35 UTC 2019
12464: Wed Apr 10 06:51:36 UTC 2019
12465: Wed Apr 10 06:51:37 UTC 2019
12466: Wed Apr 10 06:51:38 UTC 2019
12467: Wed Apr 10 06:51:39 UTC 2019
12468: Wed Apr 10 06:51:40 UTC 2019
12469: Wed Apr 10 06:51:41 UTC 2019
12470: Wed Apr 10 06:51:42 UTC 2019
12471: Wed Apr 10 06:51:43 UTC 2019

Second, --tail=-1 is the legacy usage which show all lines, here just return 10 lines.
kubectl logs -l app=test --tail=-1

The behavior after fix:

Tail value   Show lines
Default      All lines
-1           All lines
<-1          error: TailLines must be greater than or equal to -1
=0           No logs, always used with -f
>0           Show N lines

Test cases
./kubectl logs -l app=test --tail=-3
./kubectl logs -l app=test --tail=-1
./kubectl logs -l app=test --tail=2
./kubectl logs -l app=test --tail=0

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 10, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 12, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 12, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 16, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 16, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 16, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 17, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 17, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 18, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 18, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 19, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 19, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 25, 2019

@wdoekes

This comment has been minimized.

Copy link

commented Apr 25, 2019

Interesting. I think this also causes --since-time to fail:

$ kubectl -n NS --timestamps --tail=100000 POD-79c6f59559-slrst \
    --since-time='2019-04-25T10:22:26+02:00' | sort -n | head -n1 | sed -e 's/].*/]/g'
2019-04-25T08:22:23.547400000Z [pid: 13|app: 0|req: 135/325]

^-- expected record

$ kubectl -n NS logs --timestamps --selector=app=APPNAME \
    --since-time='2019-04-25T10:22:26+02:00' | sort -n | head -n1 | sed -e 's/].*/]/g'
2019-04-25T08:44:34.048756000Z [pid: 12|app: 0|req: 137/1153]

^-- some later record (there are only 2 pods running, so --max-log-requests is not relevant)

$ kubectl -n NS logs --timestamps --tail=10000 --selector=app=APPNAME \
    --since-time='2019-04-25T10:22:26+02:00' | sort -n | head -n1 | sed -e 's/].*/]/g'
2019-04-25T08:22:23.547400000Z [pid: 13|app: 0|req: 135/325]

^-- adding --tail=N makes it work

@wdoekes

This comment has been minimized.

Copy link

commented Apr 25, 2019

A bit strange that it finds records a few seconds older as well.. but that's not a big concern, and not relevant for this issue.

$ kubectl -n NS logs --timestamps POD-79c6f59559-p5mg5 \
    --since-time='2019-04-25T10:15:00+02:00' | sort -n | head -n2 | sed -e 's/].*/]/g'
2019-04-25T08:14:59.911926000Z [pid: 13|app: 0|req: 25/83]
2019-04-25T08:15:00.058213000Z [pid: 14|app: 0|req: 45/84]  <-- first record from 10:15+

vs.

# docker logs k8s_web_POD-79c6f59559-p5mg5_NS_e427eade-6731-11e9-a87b-002590e425d2_0 \
    --timestamps --since='2019-04-25T10:15:00+02:00' 2>&1 | sort -n | head -n2 | sed -e 's/].*/]/g'
2019-04-25T08:15:00.058213000Z [pid: 14|app: 0|req: 45/84]  <-- first record from 10:15+
2019-04-25T08:15:00.205318000Z [pid: 13|app: 0|req: 26/85]

Although that only happens when the --tail workaround is used:

$ kubectl -n NS logs --timestamps POD-79c6f59559-p5mg5 \
    --since-time='2019-04-25T10:15:00+02:00' | sort -n | head -n2 | sed -e 's/].*/]/g'
2019-04-25T08:15:00.058213000Z [pid: 14|app: 0|req: 45/84]  <-- first record from 10:15+
2019-04-25T08:15:00.205318000Z [pid: 13|app: 0|req: 26/85]

🤔

P.S.

Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.1", GitCommit:"b7394102d6ef778017f2ca4046abbaa23b88c290", GitTreeState:"clean", BuildDate:"2019-04-08T17:11:31Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.2", GitCommit:"17c77c7898218073f14c8d573582e8d2313dc740", GitTreeState:"clean", BuildDate:"2018-10-26T15:21:24Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 28, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Apr 29, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue May 10, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue May 17, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue May 28, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue May 29, 2019

JishanXing added a commit to JishanXing/kubernetes that referenced this issue Jun 2, 2019

pull bot pushed a commit to sadlil/kubernetes that referenced this issue Jul 1, 2019

Merge pull request kubernetes#74943 from JishanXing/bug-fix
Fix kubernetes#74900 - kubectl logs --selector supports --tail=-1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.