Golang package for gossip based membership and failure detection
Go Other
Permalink
Failed to load latest commit information.
test Only setup subnet on Mac OS X Sep 29, 2013
.gitignore Adding a gitignore file Sep 9, 2013
LICENSE Initial commit Sep 9, 2013
Makefile Added make target to fetch dependencies, and updated README Jun 28, 2017
README.md Merge pull request #129 from hashicorp/f-reduce-lan-min-suspicion-mul… Jul 17, 2017
alive_delegate.go Adding Alive delegate to support custom filtering logic May 5, 2015
awareness.go Adds a raw health score accessor. Jun 3, 2016
awareness_test.go Adds a raw health score accessor. Jun 3, 2016
broadcast.go Ensure log messages prefixed with 'memberlist' Jan 10, 2014
broadcast_test.go broadcast layer can now notify when a message is fully gossiped out Oct 2, 2013
config.go Merge pull request #129 from hashicorp/f-reduce-lan-min-suspicion-mul… Jul 17, 2017
conflict_delegate.go Adding support for a ConflictDelegate Feb 3, 2014
delegate.go Cleans up some formatting. Feb 23, 2017
event_delegate.go First pass at supporting meta data updates Jan 27, 2014
integ_test.go Reduce integ test timing Nov 14, 2014
keyring.go Expose a method to validate a gossip encryption key before use. Jun 29, 2016
keyring_test.go Standardize 'primary key' in error messages Apr 9, 2014
logging.go Adds logging helper functions. Dec 16, 2015
logging_test.go Gets rid of testing mocks. Dec 16, 2015
memberlist.go Merge pull request #123 from nathanielc/nc-issue#122 Aug 15, 2017
memberlist_test.go Fix failing test due to comparing IP address in 12 byte format to 4 b… Aug 15, 2017
merge_delegate.go Allow returning custom error for merge delegate Feb 22, 2015
mock_transport.go Adds a basic network-free transport and some tests. Feb 23, 2017
net.go Return a better error when join fails (#124) Jun 22, 2017
net_test.go Return a better error when join fails (#124) Jun 22, 2017
net_transport.go Tweaks the advertise address signature. Feb 23, 2017
ping_delegate.go Adds a direct TCP fallback ping piplined with the UDP indirect pings. Jun 2, 2015
queue.go Adding Prune to TransmitLimitedQueue Dec 1, 2013
queue_test.go Adding Prune to TransmitLimitedQueue Dec 1, 2013
security.go Formatting adjustments Apr 3, 2014
security_test.go Rounded out keyring tests Apr 4, 2014
state.go fix deadlock in shutdown process of memberlist May 26, 2017
state_test.go Fixes unit test nack check. May 25, 2017
suspicion.go Simplifies state logic by allowing zero values for k. May 31, 2016
suspicion_test.go Simplifies state logic by allowing zero values for k. May 31, 2016
tag.sh Adds a simple script to a tag releases. Apr 13, 2017
todo.md Adding long-term todo list Oct 8, 2013
transport.go Tweaks the advertise address signature. Feb 23, 2017
transport_test.go Adds a basic network-free transport and some tests. Feb 23, 2017
util.go Cleans up addresses in TCP methods. Feb 23, 2017
util_test.go Removes some debug code from a unit test. Feb 22, 2017

README.md

memberlist GoDoc

memberlist is a Go library that manages cluster membership and member failure detection using a gossip based protocol.

The use cases for such a library are far-reaching: all distributed systems require membership, and memberlist is a re-usable solution to managing cluster membership and node failure detection.

memberlist is eventually consistent but converges quickly on average. The speed at which it converges can be heavily tuned via various knobs on the protocol. Node failures are detected and network partitions are partially tolerated by attempting to communicate to potentially dead nodes through multiple routes.

Building

If you wish to build memberlist you'll need Go version 1.2+ installed.

Please check your installation with:

go version

Run make deps to fetch dependencies before building

Usage

Memberlist is surprisingly simple to use. An example is shown below:

/* Create the initial memberlist from a safe configuration.
   Please reference the godoc for other default config types.
   http://godoc.org/github.com/hashicorp/memberlist#Config
*/
list, err := memberlist.Create(memberlist.DefaultLocalConfig())
if err != nil {
	panic("Failed to create memberlist: " + err.Error())
}

// Join an existing cluster by specifying at least one known member.
n, err := list.Join([]string{"1.2.3.4"})
if err != nil {
	panic("Failed to join cluster: " + err.Error())
}

// Ask for members of the cluster
for _, member := range list.Members() {
	fmt.Printf("Member: %s %s\n", member.Name, member.Addr)
}

// Continue doing whatever you need, memberlist will maintain membership
// information in the background. Delegates can be used for receiving
// events when members join or leave.

The most difficult part of memberlist is configuring it since it has many available knobs in order to tune state propagation delay and convergence times. Memberlist provides a default configuration that offers a good starting point, but errs on the side of caution, choosing values that are optimized for higher convergence at the cost of higher bandwidth usage.

For complete documentation, see the associated Godoc.

Protocol

memberlist is based on "SWIM: Scalable Weakly-consistent Infection-style Process Group Membership Protocol". However, we extend the protocol in a number of ways:

  • Several extensions are made to increase propagation speed and convergence rate.
  • Another set of extensions, that we call Lifeguard, are made to make memberlist more robust in the presence of slow message processing (due to factors such as CPU starvation, and network delay or loss).

For details on all of these extensions, please read our paper "Lifeguard : SWIM-ing with Situational Awareness", along with the memberlist source. We welcome any questions related to the protocol on our issue tracker.