Feed server can fail due to a race condition.

[tshead2]

// feed server stderr log
[root@slycat-srn slycat]# ls -l feed-server-stderr.txt 
-rw-r--r-- 1 slycat slycat 1806 Jun  9 13:52 feed-server-stderr.txt
[root@slycat-srn slycat]# cat feed-server-stderr.txt 
ERROR:tornado.application:Uncaught exception in /changes-feed
Traceback (most recent call last):
  File "/home/slycat/install/conda/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/websocket.py", line 402, in _run_callback
    callback(*args, **kwargs)
  File "/home/slycat/src/slycat/feed-server/slycat-feed-server.py", line 222, in open
    raw_feed.add_client(self)
  File "/home/slycat/src/slycat/feed-server/slycat-feed-server.py", line 177, in add_client
    if is_project_reader(self._projects[model["project"]], client.user):
KeyError: u'71f47b44e3abe0fd8f4e2d7796e3b355'
ERROR:tornado.application:Uncaught exception in /changes-feed
Traceback (most recent call last):
  File "/home/slycat/install/conda/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/websocket.py", line 402, in _run_callback
    callback(*args, **kwargs)
  File "/home/slycat/src/slycat/feed-server/slycat-feed-server.py", line 222, in open
    raw_feed.add_client(self)
  File "/home/slycat/src/slycat/feed-server/slycat-feed-server.py", line 177, in add_client
    if is_project_reader(self._projects[model["project"]], client.user):
KeyError: u'71f47b44e3abe0fd8f4e2d7796e3b355'
ERROR:tornado.application:Uncaught exception in /changes-feed
Traceback (most recent call last):
  File "/home/slycat/install/conda/lib/python2.7/site-packages/tornado-4.1-py2.7-linux-x86_64.egg/tornado/websocket.py", line 402, in _run_callback
    callback(*args, **kwargs)
  File "/home/slycat/src/slycat/feed-server/slycat-feed-server.py", line 222, in open
    raw_feed.add_client(self)
  File "/home/slycat/src/slycat/feed-server/slycat-feed-server.py", line 177, in add_client
    if is_project_reader(self._projects[model["project"]], client.user):
KeyError: u'71f47b44e3abe0fd8f4e2d7796e3b355'

[tshead2]

It still isn't clear to me how this happens. However, I've added some code to catch the exception, log some useful information, and keep the server going.