New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOSP'2015 Poster Submission #69

Closed
Hoverbear opened this Issue Jul 24, 2015 · 11 comments

Comments

Projects
None yet
4 participants
@Hoverbear
Copy link
Owner

Hoverbear commented Jul 24, 2015

@ycoady suggested we consider submitting a poster to SOSP'2015!

Location: Monterey, California, USA
Submission deadline (HARD, NO EXTENSIONS): July 31, 2015 (23:59PM PDT)
Acceptance notification: August 7, 2015
Conference: October 4-7, 2015

Details:
http://www.ssrc.ucsc.edu/sosp15/callForPosters.html

Abstract

A Rust-Hulled Raft

Andrew Hobden (University of Victoria, undergraduate student, will present)
Dan Burkert, James McGlashan, Yvonne Coady (University of Victoria)

Demonstration: Possible and prepared, executes on GEE, but not necessary.

Though the design of Ongaro and Osterhout’s Raft algorithm dramatically improves
understandability of consensus relative to Paxos, an implementation can still be relatively
challenging to validate and verify. Part of the problem stems from infrastructure bugs
Engler identified in low-level systems. By leveraging higher-level language mechanisms,
such as those in Go, Java, C++, C#, Scala, and Erlang, many implementations are
bypassing these bugs, remaining true to the original Raft design, but ultimately
compromising on performance or understandability. Our work evaluates the potential of
Rust to build a fast, low-level implementation that is less susceptible to several
classifications of Engler’s bugs and corresponds directly to the original design.

Specifically, we evaluate a prototype Raft library that interfaces with custom or preexisting
Log and State Machine implementations, providing a Client, Server, and Consensus
Module within its core. We focus on an asynchronous single threaded event loop model in
the Rust language, allowing our implementation to have strong safety and performance
characteristics with low demands. Communication, a primary performance concern, is kept
as lightweight and efficient through Renshaw's Cap'n Proto implementation.

Our implementation features over 85% code coverage, and initial benchmarking on the
Consensus module (no network traffic) shows that it scales well. Specifically, a proposal
to a single node cluster takes approximately 1 microsecond, and a 7 node cluster
approximately 115 microseconds. Additionally, we show how lost cost language features in
Rust prevents many of the chronic bugs typically found in C based systems. We are
currently exploring opportunities to further enhance trust and security, testing our
implementation for failure conditions, and evaluating key features of the implementation
relative to a number of other contemporary implementations of Raft in various high-level
languages. (Note to reviewers: Our Rust-based Raft consensus has been a community
based collaboration, and is currently scheduled as the topic for a Rust Meetup at Mozila
Headquarters in late August. We will be including the feedback from this community in
our poster if it is accepted, as well as expanding and improving benchmarking
measurements).

Poster

raft poster 5

@ycoady

This comment has been minimized.

Copy link
Collaborator

ycoady commented Jul 25, 2015

love it...
making a mess with a few questions and edits, feel free to ignore!

Today our systems operate in extreme conditions, functioning across containers, virtual machines, infrastructure, networks, embedded systems, in our pockets, and even inside of us. Many of these systems depend on one another for operation, while others are able to operate in failing connectivity without disaster. In many cases it is preferable to have the latter, especially if the continued operation of the system is at stake. Distributed consensus represents one small part of a larger system, and offers the ability to maintain a replicated persistent log containing actions that are applied globally into a state machine. This allows n clients to communicate to a cluster of m servers in a stable and predictable manner, even in failing network conditions. Could be worth mentioning something about how the complexity of Paxos was uninviting, but now many more researchers are diving in with many more languages... or something?!

Using Ongaro and Osterhouts's Raft algorithm we are developing a fast, low level, low demands implementation of the system in an opinionated, minimal way. Might be worth clarifying exactly what you mean by "low demands" and "opinionated"? The Raft library interfaces with custom or preexisting Log and State Machine implementations, providing a Client, Server, and Consensus Module within its core. We have chosen an asynchronous single threaded event loop model in the Rust language, allowing our implementation to have strong safety and performance characteristics with low demands. Communication, a primary performance concern, is kept as lightweight and fast as possible by using Renshaw's Cap'n Proto implementation. TODO: Talk about performance tests and testing. Absolutely---if it is possible to show yours relative to any of the others on the Raft page, that would ROCK! Updating your row there before submitting would be great as well! We are currently exploring opportunities in trust and security as well as testing our implementation for further failure conditions.

b e a u t i f u l !

@dwrensha

This comment has been minimized.

Copy link

dwrensha commented Jul 25, 2015

s/Rensha/David Renshaw/

@Hoverbear

This comment has been minimized.

Copy link
Owner

Hoverbear commented Jul 25, 2015

@dwrensha Gosh darnit I do that all the time, I'm so sorry!

From your twitter:

@dwrensha

The 'w' comes earlier than you'd think!

@Hoverbear

This comment has been minimized.

Copy link
Owner

Hoverbear commented Jul 25, 2015

@ycoady Once we get more features working and have more battle testing doing some performance measurements would be very useful, yeah. I remember Min was working on a consensus tester!

Moreso, though, I'm interested in testing reliablility and stablility.

homu added a commit that referenced this issue Jul 25, 2015

Auto merge of #72 - Hoverbear:cargo, r=danburkert
Improve the Cargo.toml

Just expand on the `Cargo.toml` a bit in preparation for release. Taken from #69. r+?
@Hoverbear

This comment has been minimized.

Copy link
Owner

Hoverbear commented Jul 30, 2015

@james-darkfox want to be included in on this? No obligations from you other than keep contributing! :)

@james-darkfox

This comment has been minimized.

Copy link
Contributor

james-darkfox commented Jul 31, 2015

@Hoverbear Sure!

@Hoverbear

This comment has been minimized.

Copy link
Owner

Hoverbear commented Jul 31, 2015

Updated abstract with @ycoady's feedback.

@Hoverbear

This comment has been minimized.

Copy link
Owner

Hoverbear commented Jul 31, 2015

Updated poster and abstract with some initial benchmarks. The abstract is too darn long!

@Hoverbear

This comment has been minimized.

Copy link
Owner

Hoverbear commented Aug 1, 2015

We submitted this! Closing for now. If we get accepted I'll open a new issue!

@Hoverbear Hoverbear closed this Aug 1, 2015

@Hoverbear

This comment has been minimized.

Copy link
Owner

Hoverbear commented Aug 12, 2015

The poster was rejected.

@james-darkfox

This comment has been minimized.

Copy link
Contributor

james-darkfox commented Aug 12, 2015

Rejected due to any reason specified or undisclosed why?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment