Controller stops updating BIG-IP configuration after an exception occurs in the python driver

[russokj]

Description

If the bigipdriver.py process encounters an exception in the _do_reset thread, then the thread dies and no longer processes updates to the config file. The front-end controller is unaware of this and keeps updating the config file. Many of these exceptions are transient in nature and the python driver should handle them gracefully.

Diagnostic Information

Typical log file has an error similar to the following:

21:55:44 2018/01/23 04:53:31 [INFO] Traceback (most recent call last):
21:55:44 2018/01/23 04:53:31 [INFO]   File "/opt/rh/python27/root/usr/lib64/python2.7/threading.py", line 804, in __bootstrap_inner
21:55:44 2018/01/23 04:53:31 [INFO]     self.run()
21:55:44 2018/01/23 04:53:31 [INFO]   File "/opt/rh/python27/root/usr/lib64/python2.7/threading.py", line 757, in run
21:55:44 2018/01/23 04:53:31 [INFO]     self.__target(*self.__args, **self.__kwargs)
21:55:44 2018/01/23 04:53:31 [INFO]   File "/app/python/bigipconfigdriver.py", line 320, in _do_reset
21:55:44 2018/01/23 04:53:31 [INFO]     config = _parse_config(self._config_file)
21:55:44 2018/01/23 04:53:31 [INFO]   File "/app/python/bigipconfigdriver.py", line 584, in _parse_config
21:55:44 2018/01/23 04:53:31 [INFO]     config_json = json.load(config)
21:55:44 2018/01/23 04:53:31 [INFO]   File "/opt/rh/python27/root/usr/lib64/python2.7/json/__init__.py", line 291, in load
21:55:44 2018/01/23 04:53:31 [INFO]     **kw)
21:55:44 2018/01/23 04:53:31 [INFO]   File "/opt/rh/python27/root/usr/lib64/python2.7/json/__init__.py", line 339, in loads
21:55:44 2018/01/23 04:53:31 [INFO]     return _default_decoder.decode(s)
21:55:44 2018/01/23 04:53:31 [INFO]   File "/opt/rh/python27/root/usr/lib64/python2.7/json/decoder.py", line 364, in decode
21:55:44 2018/01/23 04:53:31 [INFO]     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
21:55:44 2018/01/23 04:53:31 [INFO]   File "/opt/rh/python27/root/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode
21:55:44 2018/01/23 04:53:31 [INFO]     raise ValueError("No JSON object could be decoded")
21:55:44 2018/01/23 04:53:31 [INFO] ValueError: No JSON object could be decoded
21:55:44 2018/01/23 04:53:31 [INFO] [2018-01-23 04:53:31,065 __main__ INFO] entering inotify loop to watch /tmp/k8s-bigip-ctlr.config553021570/config.json
21:55:44 2018/01/23 04:53:31 [INFO] 

[russokj]

Need to cherry-pick the fix for the bigipconfigdriverlpy from the k8s repo (see F5Networks/k8s-bigip-ctlr#505).