Foundationdb

[jenschelkopf]

Added support for testing FoundationDB with Jepsen.

Includes Salticid support for:

• Installing FoundationDB and configuring for triple replication
• Completely removing FoundationDB (nuking), including data and log files
• Collecting the log files
• Removing the log files

Includes three tests for Jepsen:

• foundationdb: Create a new key/value pair for a list of elements, retries failed transactions using the FoundationDB API's automatic transaction retry loop
• foundationdb-append: Append a list of element to a single key/value pair, retries failed transactions using the FoundationDB API's automatic transaction retry loop
• foundationdb-append-noretry: Append a list of element to a single key/value pair, does not retry failed transactions

To run these tests yourself, you'll need to:

• Install the FoundationDB client on the machine running Jepsen (you don't need the server)
• Copy /etc/foundationdb/fdb.cluster from one of the database servers to the machine running Jepsen. You can put in either the default cluster location, or in the directory from which you'll run Jepsen

[aphyr]

Jenn, you're the best. Thanks so much for writing this. :)

[aphyr]

Incidentally, I'm restructuring Jepsen to make linearizability checking and database setup easier; salticid is going away and Jepsen itself will take care of all the automation. May make things easier if you keep using Jepsen going forward. :)

[jenschelkopf]

Kyle - I've been thinking of some other things I could do with Jepsen, like the multi-data center idea that we talked about. So easier is great!

[jenschelkopf]

@colin-scott My first thought is to make sure you have the FoundationDB client installed, which you can get here. If that doesn't work for you, send me an email at jennifer.rullmann@foundationdb.com or post a question to our community site, and I'll take a look!

[colin-scott]

Hey Jennifer, that worked for me. I'm mostly wondering whether we should
download it automatically though, since all Jepsen apps (not just
foundationdb) don't work until that client is installed, i.e. it has now
become an undocumented installation step for getting Jepsen up and running.

Perhaps another solution: is it possible to tell Clojure / lein that the
foundationdb app should be lazily compiled?
On Mar 27, 2014 6:09 AM, "Jennifer Rullmann" notifications@github.com
wrote:

@colin-scott https://github.com/colin-scott My first thought is to make
sure you have the FoundationDB client installed, which you can get herehttps://foundationdb.com/get.
If that doesn't work for you, send me an email at
jennifer.rullmann@foundationdb.com or post a question to our community
site http://community.foundationdb.com/, and I'll take a look!

Reply to this email directly or view it on GitHubhttps://github.com//pull/10#issuecomment-38799949
.

[aphyr]

Yeah, I agree that we should not require users to go through a separate FDB install process to use Jepsen. I suggest one of:

1. The FDB Maven packages provide all dependencies required,
2. Pull the FDB tests out into a separate project that depends on Jepsen
3. We leave this work on a separate FDB branch

Thoughts?

[jenschelkopf]

Yeah, not ideal! I'd like to keep foundationdb part of the project and main branch if possible - I think that's easier to maintain and for people to find.

I like @colin-scott's idea of loading the app dynamically. This isn't pretty but it seems to work well:

  (when (or (contains? (into #{} app-names) "foundationdb") (contains? (into #{} app-names) "foundationdb-append") (contains? (into #{} app-names) "foundationdb-append-noretry"))
    (load-file "src/jepsen/foundationdb.clj")
    (def app-map (merge app-map {
      "foundationdb" (eval (read-string "jepsen.foundationdb/foundationdb-app"))
      "foundationdb-append" (eval (read-string "jepsen.foundationdb/foundationdb-append-app"))
      "foundationdb-append-noretry" (eval (read-string "jepsen.foundationdb/foundationdb-append-noretry-app"))
      })))

I'm happy to submit another pull request if you like this idea.

[aphyr]

(or (contains? (into #{} app-names) "foundationdb") 
    (contains? (into #{} app-names) "foundationdb-append")
    (contains? (into #{} app-names) "foundationdb-append-noretry")`.

can be

(some #{"foundationdb"
        "foundationdb-append"
        "foundationdb-append-noretry"} app-names)`

and I think (eval 'jepsen.foundationdb/foundationdb-app) will do in place of (eval (read-string "jepsen.foundationdb/foundationdb-app")). You may also be able to get away with (require 'jepsen.foundationdb) in place of load-file.

Works for me!

[jenschelkopf]

I made the changes you suggested (much cleaner!) and submitted another pull request.