Golang package for gossip based membership and failure detection
Switch branches/tags
Clone or download
rboyer Unflake a bunch of tests.
* TestMemberlist_EncryptedGossipTransition was rewritten to actually
  test a recommended upgrade path, instead of testing a crazy
  split-brain situation that would fail the test roughly half of the
  time.

* Forced tests to defer a call to Memberlist.Shutdown to avoid bleedover.

* Forced each test to pick a random port for its cluster to avoid bleedover.

* Got tests to pass the race detector.
Latest commit 2072f3a Oct 23, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
test Only setup subnet on Mac OS X Sep 29, 2013
.gitignore Adding a gitignore file Sep 9, 2013
.travis.yml switch to VMs Oct 25, 2018
LICENSE Initial commit Sep 9, 2013
Makefile Only evaluate DEPS variable once in makefile. Oct 25, 2018
README.md Update README.md Oct 25, 2018
alive_delegate.go Adding Alive delegate to support custom filtering logic May 5, 2015
awareness.go Adds a raw health score accessor. Jun 3, 2016
awareness_test.go Adds a raw health score accessor. Jun 3, 2016
broadcast.go Ensure log messages prefixed with 'memberlist' Jan 10, 2014
broadcast_test.go broadcast layer can now notify when a message is fully gossiped out Oct 2, 2013
config.go Merge pull request #129 from hashicorp/f-reduce-lan-min-suspicion-mul… Jul 17, 2017
conflict_delegate.go Adding support for a ConflictDelegate Feb 3, 2014
delegate.go Cleans up some formatting. Feb 23, 2017
event_delegate.go First pass at supporting meta data updates Jan 27, 2014
integ_test.go Unflake a bunch of tests. Oct 25, 2018
keyring.go Expose a method to validate a gossip encryption key before use. Jun 29, 2016
keyring_test.go Standardize 'primary key' in error messages Apr 9, 2014
logging.go Adds logging helper functions. Dec 16, 2015
logging_test.go Gets rid of testing mocks. Dec 16, 2015
memberlist.go Unflake a bunch of tests. Oct 25, 2018
memberlist_test.go Unflake a bunch of tests. Oct 25, 2018
merge_delegate.go Allow returning custom error for merge delegate Feb 22, 2015
mock_transport.go Adds a basic network-free transport and some tests. Feb 23, 2017
net.go Make some tests pass with modern go Aug 9, 2018
net_test.go Unflake a bunch of tests. Oct 25, 2018
net_transport.go Tweaks the advertise address signature. Feb 23, 2017
ping_delegate.go Adds a direct TCP fallback ping piplined with the UDP indirect pings. Jun 2, 2015
queue.go Unflake a bunch of tests. Oct 25, 2018
queue_test.go Adding Prune to TransmitLimitedQueue Dec 1, 2013
security.go Formatting adjustments Apr 3, 2014
security_test.go Rounded out keyring tests Apr 4, 2014
state.go Unflake a bunch of tests. Oct 25, 2018
state_test.go Unflake a bunch of tests. Oct 25, 2018
suspicion.go Use time.Since instead of time.Now().Sub Nov 9, 2017
suspicion_test.go Simplifies state logic by allowing zero values for k. May 31, 2016
tag.sh Adds a simple script to a tag releases. Apr 13, 2017
todo.md Adding long-term todo list Oct 8, 2013
transport.go Update transport.go Nov 16, 2017
transport_test.go Unflake a bunch of tests. Oct 25, 2018
util.go Slightly speed up the node list shuffle using rand.Shuffle directly. Oct 25, 2018
util_test.go Adds support for joins to raw IPv6 addresses. Nov 10, 2017
z_test.go Unflake a bunch of tests. Oct 25, 2018

README.md

memberlist GoDoc Build Status

memberlist is a Go library that manages cluster membership and member failure detection using a gossip based protocol.

The use cases for such a library are far-reaching: all distributed systems require membership, and memberlist is a re-usable solution to managing cluster membership and node failure detection.

memberlist is eventually consistent but converges quickly on average. The speed at which it converges can be heavily tuned via various knobs on the protocol. Node failures are detected and network partitions are partially tolerated by attempting to communicate to potentially dead nodes through multiple routes.

Building

If you wish to build memberlist you'll need Go version 1.2+ installed.

Please check your installation with:

go version

Run make deps to fetch dependencies before building

Usage

Memberlist is surprisingly simple to use. An example is shown below:

/* Create the initial memberlist from a safe configuration.
   Please reference the godoc for other default config types.
   http://godoc.org/github.com/hashicorp/memberlist#Config
*/
list, err := memberlist.Create(memberlist.DefaultLocalConfig())
if err != nil {
	panic("Failed to create memberlist: " + err.Error())
}

// Join an existing cluster by specifying at least one known member.
n, err := list.Join([]string{"1.2.3.4"})
if err != nil {
	panic("Failed to join cluster: " + err.Error())
}

// Ask for members of the cluster
for _, member := range list.Members() {
	fmt.Printf("Member: %s %s\n", member.Name, member.Addr)
}

// Continue doing whatever you need, memberlist will maintain membership
// information in the background. Delegates can be used for receiving
// events when members join or leave.

The most difficult part of memberlist is configuring it since it has many available knobs in order to tune state propagation delay and convergence times. Memberlist provides a default configuration that offers a good starting point, but errs on the side of caution, choosing values that are optimized for higher convergence at the cost of higher bandwidth usage.

For complete documentation, see the associated Godoc.

Protocol

memberlist is based on "SWIM: Scalable Weakly-consistent Infection-style Process Group Membership Protocol". However, we extend the protocol in a number of ways:

  • Several extensions are made to increase propagation speed and convergence rate.
  • Another set of extensions, that we call Lifeguard, are made to make memberlist more robust in the presence of slow message processing (due to factors such as CPU starvation, and network delay or loss).

For details on all of these extensions, please read our paper "Lifeguard : SWIM-ing with Situational Awareness", along with the memberlist source. We welcome any questions related to the protocol on our issue tracker.