Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set klog's logtostderr flag #2529

Merged
merged 3 commits into from
Feb 7, 2019
Merged

Set klog's logtostderr flag #2529

merged 3 commits into from
Feb 7, 2019

Conversation

gjtempleton
Copy link
Contributor

@gjtempleton gjtempleton commented Feb 5, 2019

1. Why is this pull request needed and what does it do?

The update of client-go to v10.0.0 has moved K8s logging from glog to klog.

This change removes the previous non-obvious flag setting behaviour used for glog and then ensures that klog, the replacement for glog has the relevant flag set. See klog code for the reason why this has to happen: https://github.com/coredns/coredns/blob/92836cc6f9aaf47a349993271ddae273d8e7d25d/vendor/k8s.io/klog/klog.go

Note that simply updating to the version of klog updating this flag to default to true rather than false was tested and still caused crashes.

Tested locally:

kubectl logs -f -n kube-system core-dns-8ff86fc8c-s2hng
.:53
2019-02-05T20:52:29.173Z [INFO] CoreDNS-1.3.1
2019-02-05T20:52:29.173Z [INFO] linux/amd64, go1.11.4, 66623192-dirty
CoreDNS-1.3.1
linux/amd64, go1.11.4, 66623192-dirty
2019-02-05T20:52:29.173Z [INFO] plugin/reload: Running configuration MD5 = a03e157fe14ec2d191bddec59dc496fc
E0205 20:53:15.981439       1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=15, ErrCode=NO_ERROR, debug=""
E0205 20:53:15.981458       1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=15, ErrCode=NO_ERROR, debug=""
E0205 20:53:15.981458       1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=15, ErrCode=NO_ERROR, debug=""
E0205 20:53:15.981470       1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=15, ErrCode=NO_ERROR, debug=""
E0205 20:53:15.981849       1 reflector.go:251] github.com/coredns/coredns/plugin/kubernetes/controller.go:322: Failed to watch *v1.Namespace: Get https://10.254.0.1:443/api/v1/namespaces?resourceVersion=343852&timeout=5m46s&timeoutSeconds=346&watch=true: dial tcp 10.254.0.1:443: connect: connection refused
E0205 20:53:15.981851       1 reflector.go:251] github.com/coredns/coredns/plugin/kubernetes/controller.go:320: Failed to watch *v1.Pod: Get https://10.254.0.1:443/api/v1/pods?resourceVersion=347745&timeout=5m19s&timeoutSeconds=319&watch=true: dial tcp 10.254.0.1:443: connect: connection refused
E0205 20:53:15.981851       1 reflector.go:251] github.com/coredns/coredns/plugin/kubernetes/controller.go:317: Failed to watch *v1.Endpoints: Get https://10.254.0.1:443/api/v1/endpoints?resourceVersion=347877&timeout=5m29s&timeoutSeconds=329&watch=true: dial tcp 10.254.0.1:443: connect: connection refused
E0205 20:53:15.981876       1 reflector.go:251] github.com/coredns/coredns/plugin/kubernetes/controller.go:315: Failed to watch *v1.Service: Get https://10.254.0.1:443/api/v1/services?resourceVersion=343852&timeout=8m26s&timeoutSeconds=506&watch=true: dial tcp 10.254.0.1:443: connect: connection refused

No restarts of pods observed.

2. Which issues (if any) are related?

#2464 , supersedes the sticking plaster attempted by #2525

3. Which documentation changes (if any) need to be made?

Noted as a bug fix for a regression in v1.3.1 only.

@corbot
Copy link

corbot bot commented Feb 5, 2019

Thank you for your contribution. I've just checked the OWNERS files to find a suitable reviewer. This search was successful and I've asked fturib (via plugin/kubernetes/OWNERS) for a review.

If you have questions or suggestions for this bot, please file an issue against the miekg/dreck repository.

The bot understands the commands that are listed here.

@corbot corbot bot requested a review from fturib February 5, 2019 21:25
@codecov-io
Copy link

codecov-io commented Feb 5, 2019

Codecov Report

Merging #2529 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2529      +/-   ##
==========================================
+ Coverage   55.37%   55.38%   +0.01%     
==========================================
  Files         204      204              
  Lines       10346    10349       +3     
==========================================
+ Hits         5729     5732       +3     
- Misses       4196     4197       +1     
+ Partials      421      420       -1
Impacted Files Coverage Δ
plugin/kubernetes/setup.go 70.46% <100%> (+0.46%) ⬆️
plugin/forward/connect.go 80.59% <0%> (-4.48%) ⬇️
plugin/route53/route53.go 85.18% <0%> (+2.77%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 92836cc...5dd3935. Read the comment docs.

@gjtempleton gjtempleton changed the title Parse as well as setlogtostderr flag Set klog's logtostderr flag Feb 5, 2019
@@ -33,12 +36,15 @@ import (
var log = clog.NewWithPlugin("kubernetes")

func init() {
// Kubernetes plugin uses the kubernetes library, which uses glog (ugh), we must set this *flag*,
// Kubernetes plugin uses the kubernetes library, which now uses klog (ugh), we must set and parse this *flag*,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I though klog was better than glog? Remove the 'ugh' at least (glog is almost evil on how it interacts with its environment)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it is a significant step forward from glog and looks to be making progress on continuing to clean up a lot of the minor pain points, have cleared up the comment.

Think it's worth still keeping there given the current behaviour, at least as used by the k8s libraries, is still to log to disk.

@miekg miekg merged commit 39cf482 into coredns:master Feb 7, 2019
Jason-ZW pushed a commit to rancher/coredns that referenced this pull request Apr 17, 2019
* Parse as well as  setlogtostderr flag

* Enforce setting of logtostderr for klog

* Clearup comment on klog
@gjtempleton gjtempleton deleted the Klog-Stderr branch May 28, 2019 13:51
dna2github pushed a commit to dna2fork/coredns that referenced this pull request Jul 19, 2019
* Parse as well as  setlogtostderr flag

* Enforce setting of logtostderr for klog

* Clearup comment on klog
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants