unexpected error in namespace watcher thread

[mszabo]

I get this error after the plugin is running for a while:

td-agent-v3-t2rm7 td-agent 2020-03-24 11:52:50 +0000 [error]: #0 unexpected error error_class=NoMethodError error="undefined method `<' for nil:NilClass"
td-agent-v3-t2rm7 td-agent   2020-03-24 11:52:50 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-kubernetes_metadata_filter-2.4.5/lib/fluent/plugin/kubernetes_metadata_watch_namespaces.rb:43:in `rescue in set_up_namespace_thread'
td-agent-v3-t2rm7 td-agent   2020-03-24 11:52:50 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-kubernetes_metadata_filter-2.4.5/lib/fluent/plugin/kubernetes_metadata_watch_namespaces.rb:38:in `set_up_namespace_thread'
td-agent-v3-t2rm7 td-agent   2020-03-24 11:52:50 +0000 [error]: #0 /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-kubernetes_metadata_filter-2.4.5/lib/fluent/plugin/filter_kubernetes_metadata.rb:277:in `block in configure'
td-agent-v3-t2rm7 td-agent 2020-03-24 11:52:50 +0000 [error]: #0 unexpected error error_class=NoMethodError error="undefined method `<' for nil:NilClass"
td-agent-v3-t2rm7 td-agent   2020-03-24 11:52:50 +0000 [error]: #0 suppressed same stacktrace
td-agent-v3-t2rm7 td-agent 2020-03-24 11:52:51 +0000 [info]: Worker 0 finished unexpectedly with status 1

I believe Thread.current[:namespace_watch_retry_count] is never initialised if there are no namespace related events before the API server disconnects. So basically only the ERROR case is executed in the watcher block.

The same problem could happen with the pod watcher as well, but I didn't observe it there as we have many pod related events, while our namespaces do not really change.

@qingling128 what would be the good place to initialise the variable to handle this case?

[qingling128]

Sent a PR.