New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FileWatcher watching a file containing server list. #53

Closed
wants to merge 3 commits into
base: master
from

Conversation

Projects
None yet
4 participants
@ryotarai
Contributor

ryotarai commented Mar 26, 2014

I added a new watcher, FileWatcher, which watches a server-list file. Because this watcher uses Listen gem, changes of the file will be detected immediately with something like inotify.

Why I implemented this

I wrote this watcher for using with Serf. Serf can get join event and leave event. If a event handler of Serf adds (or removes) a server from the file, synapse will adds (or removes) the backend server to haproxy.cfg.
Of course, this works well without Serf too.

@mirakui

This comment has been minimized.

Show comment
Hide comment
@mirakui

mirakui commented Mar 27, 2014

cool

@patrickviet

This comment has been minimized.

Show comment
Hide comment
@patrickviet

patrickviet Apr 3, 2014

@ryotarai there is a bug here if you aren't explicitly using inotify with the listen gem (and even if you do, it might overflow - I once implemented a filesystem mirror with inotify and in case of very many file changes it would miss some). Here is how it goes:

  • there is a netsplit, lots of servers leave
  • there is a rejoin, they all come back

This means that you might have the file be created by serf at the beginning of the second with 10 servers, and then at the end of the second with 11 servers (for example). If you are polling with stat(2) (default behavior) and not using inotify, then 'listen' will only see one change.

This is how I solved it:
https://github.com/patrickviet/synapse/blob/serf/lib/synapse/service_watcher/serf.rb#L38

Patrick

patrickviet commented Apr 3, 2014

@ryotarai there is a bug here if you aren't explicitly using inotify with the listen gem (and even if you do, it might overflow - I once implemented a filesystem mirror with inotify and in case of very many file changes it would miss some). Here is how it goes:

  • there is a netsplit, lots of servers leave
  • there is a rejoin, they all come back

This means that you might have the file be created by serf at the beginning of the second with 10 servers, and then at the end of the second with 11 servers (for example). If you are polling with stat(2) (default behavior) and not using inotify, then 'listen' will only see one change.

This is how I solved it:
https://github.com/patrickviet/synapse/blob/serf/lib/synapse/service_watcher/serf.rb#L38

Patrick

@jolynch

This comment has been minimized.

Show comment
Hide comment
@jolynch

jolynch Sep 7, 2015

Contributor

@ryotarai Can you rebase this against latest master and I can review if you're still interested in this watcher.

Contributor

jolynch commented Sep 7, 2015

@ryotarai Can you rebase this against latest master and I can review if you're still interested in this watcher.

@ryotarai

This comment has been minimized.

Show comment
Hide comment
@ryotarai

ryotarai Sep 8, 2015

Contributor

Thank you for mention. Since I'm no longer interested in this feature, I close this PR

Contributor

ryotarai commented Sep 8, 2015

Thank you for mention. Since I'm no longer interested in this feature, I close this PR

@ryotarai ryotarai closed this Sep 8, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment