Basic usage instructions

[jasonkolb]

Is there a page somewhere explaining basic usage? I'd like to run the elasticsearch tests but when I use the syntax in the readme I get a FileNotFoundException:

Caused by: java.io.FileNotFoundException: Could not locate jepsen/tests__init.class or jepsen/tests.clj on classpath:

I'm not too familiar with Clojure though, so if this is a really basic error feel free to tell me to go pound sand.

[abailly]

Hi Jason,
Can you please provide a detailed log of how you are running the tests?

[jasonkolb]

Sure... just copying and pasting the command from the readme:

lein with-profile +elasticsearch test jepsen.system.elasticsearch-test

I tried running this both from the root code directory and the directory containing the test class, no result.

Thanks for your help!

[abailly]

I am a moron. In the latest PR that was merged by @aphyr I forgot to include the jepsen.tests file that now contains all that is needed by test cases. Apologies for the inconvenience, I am sending another PR...

[abailly]

#40 fixes the issue.

[jasonkolb]

Cool, that did indeed fix that issue. Now I'm getting an unkown host exception which I've copied below. I assume there's some setup required before running this test (setting up virtual hosts of some kind I suspect), but I can't find any documentation describing what that setup is.

ERROR in (create-test) (Util.java:344)
Uncaught exception, not in assertion.
expected: nil
actual: com.jcraft.jsch.JSchException: java.net.UnknownHostException: n1
at com.jcraft.jsch.Util.createSocket (Util.java:344)
com.jcraft.jsch.Session.connect (Session.java:215)
com.jcraft.jsch.Session.connect (Session.java:183)
clj_ssh.ssh$connect.invoke (ssh.clj:327)
jepsen.control$session.invoke (control.clj:177)
clojure.lang.AFn.applyToHelper (AFn.java:154)
clojure.lang.AFn.applyTo (AFn.java:144)
clojure.core$apply.invoke (core.clj:624)
jepsen.core$fcatch$wrapper__7367.doInvoke (core.clj:39)
clojure.lang.RestFn.invoke (RestFn.java:408)
clojure.core$pmap$fn__6328$fn__6329.invoke (core.clj:6463)
clojure.core$binding_conveyor_fn$fn__4145.invoke (core.clj:1910)
clojure.lang.AFn.call (AFn.java:18)
java.util.concurrent.FutureTask.run (FutureTask.java:262)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615)
java.lang.Thread.run (Thread.java:745)

[abailly]

you can find instructions on setting up lxc-based hosts in the file lxc.md. I also started a project to fire up a Vagrant-based VM with everything configured to run jepsen tests in it, based on the aforementioned lxc configuration. See https://github.com/abailly/jepsen-vagrant</shameless plug>

[papisz]

I've got some issues running jepsen too. I'm using ubuntu 14.04, I've followed the instructions from lxc.md, got 5 nodes up & running, ssh works.
When I try to invoke the elasticsearch example, I get:

$ lein with-profile +elasticsearch test jepsen.system.elasticsearch-test

lein test jepsen.system.elasticsearch-test
SLF4J: The following loggers will not work becasue they were created
SLF4J: during the default configuration phase of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#substituteLogger
SLF4J: clj-ssh.ssh
SLF4J: clj-ssh.ssh
SLF4J: clj-ssh.ssh
SLF4J: clj-ssh.ssh

lein test :only jepsen.system.elasticsearch-test/create-test

ERROR in (create-test) (Session.java:770)
Uncaught exception, not in assertion.
expected: nil
  actual: com.jcraft.jsch.JSchException: reject HostKey: n1
 at com.jcraft.jsch.Session.checkHost (Session.java:770)
    com.jcraft.jsch.Session.connect (Session.java:342)
    com.jcraft.jsch.Session.connect (Session.java:183)
    clj_ssh.ssh$connect.invoke (ssh.clj:327)
    jepsen.control$session.invoke (control.clj:177)
    clojure.lang.AFn.applyToHelper (AFn.java:154)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invoke (core.clj:624)
    jepsen.core$fcatch$wrapper__7365.doInvoke (core.clj:39)
    clojure.lang.RestFn.invoke (RestFn.java:408)
    clojure.core$pmap$fn__6328$fn__6329.invoke (core.clj:6463)
    clojure.core$binding_conveyor_fn$fn__4145.invoke (core.clj:1910)
    clojure.lang.AFn.call (AFn.java:18)
    java.util.concurrent.FutureTask.run (FutureTask.java:262)
    java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145)
    java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615)
    java.lang.Thread.run (Thread.java:745)

Ran 1 tests containing 1 assertions.
0 failures, 1 errors.
Tests failed.
Error encountered performing task 'test' with profile(s): 'base,system,user,provided,dev,elasticsearch'
Tests failed.

[abailly]

Is this error transient or constant? This sometimes happen haphazardly because of random network errors as noted in #38 The error you get seems to indicate that the host key has changed, so it might also be necessary to clean up known_hosts file of user running tests.

[papisz]

Unfortunately it is constant.
I've cleared up the known_hosts file - the problem remains.

The ssh server log from n1:

    Aug 27 16:24:31 n1 sshd[2549]: Received disconnect from 10.0.3.1: 3: com.jcraft.jsch.JSchException: reject HostKey: n1 [preauth]

Tried also this one (http://anahorny.blogspot.com/2013/05/solution-for-comjcraftjschjschexception.html), but now I get another error:

  $ lein with-profile +elasticsearch test jepsen.system.elasticsearch-test

lein test jepsen.system.elasticsearch-test

lein test :only jepsen.system.elasticsearch-test/create-test

ERROR in (create-test) (Session.java:512)
Uncaught exception, not in assertion.
expected: nil
  actual: com.jcraft.jsch.JSchException: Auth fail
 at com.jcraft.jsch.Session.connect (Session.java:512)
    com.jcraft.jsch.Session.connect (Session.java:183)
    clj_ssh.ssh$connect.invoke (ssh.clj:327)
    jepsen.control$session.invoke (control.clj:177)
    clojure.lang.AFn.applyToHelper (AFn.java:154)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invoke (core.clj:624)
    jepsen.core$fcatch$wrapper__7365.doInvoke (core.clj:39)
    clojure.lang.RestFn.invoke (RestFn.java:408)
    clojure.core$pmap$fn__6328$fn__6329.invoke (core.clj:6463)
    clojure.core$binding_conveyor_fn$fn__4145.invoke (core.clj:1910)
    clojure.lang.AFn.call (AFn.java:18)
    java.util.concurrent.FutureTask.run (FutureTask.java:262)
    java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145)
    java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615)
    java.lang.Thread.run (Thread.java:745)

Ran 1 tests containing 1 assertions.
0 failures, 1 errors.
Tests failed.
Error encountered performing task 'test' with profile(s): 'base,system,user,provided,dev,elasticsearch'
Tests failed.

[papisz]

OK, I've figured it out.
First of all, on my local ubuntu known_hosts is hashed and jsch doesn't like it:
http://anahorny.blogspot.com/2013/05/solution-for-comjcraftjschjschexception.html

Secondly, the jepsen docs in lxc.md recommend to install debian on nodes, but it gives much pain (the default user on debian is root and not ubuntu as is hardcoded in the jepsen library, there's no sudo, when the test tries to install the libraries I get the errors about --force-yes). And the errors keep coming.

I think that it would be nice because of the jepsen-newcomers to make a note in the docs that current version may not go well with debian nodes ;-)

[aphyr]

Good call, haha--I run debian testing on my host and LXC nodes, but adduser ubuntu and installed a few basic packages ages ago, and forgot to add em to the docs.

The ubuntu user is a holdover from some old scripts that ran on ubuntu; we really should swap to a different user. Root may be the best way to go, really, for VMs like this.