missing CodeGenModule.h #18

kaarrot · 2014-10-06T05:29:08Z

Hi,
I sucessufly build Julia with the following changes to Make.user but getting following error when building the package:

julia> Pkg.build("Cxx")
INFO: Building Cxx
Tuning for julia installation at: /home/kuba/PRJ/JULIA/0.4/julia/usr/bin
CC /home/kuba/.julia/v0.4/Cxx/deps/build/bootstrap.o
../src/bootstrap.cpp:49:35: fatal error: CodeGen/CodeGenModule.h: No such file or directory
#include "CodeGen/CodeGenModule.h"
^
compilation terminated.
make: *** [build/bootstrap.o] Error 1

Julia:
Version 0.4.0-dev+949 (2014-10-05 02:28 UTC)
Commit 7dd704b (1 day old master)
x86_64-redhat-linux

Thanks for looking into this,
kuba

Keno · 2014-10-06T08:47:43Z

That file definitely still exists: https://github.com/llvm-mirror/clang/blob/master/lib/CodeGen/CodeGenModule.h can you check that clang is properly checked out in deps/llvm-svn/tools/clang?

kaarrot · 2014-10-06T21:52:42Z

Looks like setting BUILD_LLVM_CLANG=1 in Make.user does not check out and build Clang automatically. Manual build seems to push things a little bit further.

Right now though I'm facing a link error :

$ make -f BuildBootstrap.Makefile JULIA_HOME=/data/job/lab/kuba/build/julia_all/julia/usr/bin/
CC /data/home/rothk/.julia/v0.4/Cxx/deps/build/bootstrap.o
LINK /data/home/rothk/.julia/v0.4/Cxx/deps/usr/lib/libcxxffi.so
/usr/bin/ld: cannot find -llldbAPI

Is this a part of libcxx (which I haven't build btw?)
Btw. Would it help to build Julia and packages with Clang instead of GCC?

Thanks
kuba

Keno · 2014-10-06T21:59:05Z

Usually everything is checked out correctly. However it seems that in your case something got messed up. That error message indicates that lldb wasn't built correctly (should by in deps/tools/lldb). I don't actually use LLDB in the C++ FFI, so you could edit the Makefile to take out the link lines, but I use it in one of the example scripts and in the upcoming debugger that is based on this package, which is why it is there. Can you try making sure LLDB builds properly for you and report any errors (I've been fixing build problems in LLDB, but I think I got most of them now).

kaarrot · 2014-10-06T22:05:32Z

Sure, I just noticed overlooked LLDB - building it now.

kaarrot · 2014-10-06T23:00:20Z

I was stopped by a few missing dependencies: Swig, libedit.
In the meantime I removed LLDB parts from the Makefile and get libcxxffi.so generated.
Although in Julia I get:
julia> using Cxx
Warning: redefining constant libcxxffi
ERROR: error compiling init: could not load module libcxxffi: libcxxffi: cannot open shared object file: No such file or directory
in include at ./boot.jl:245
in include_from_node1 at ./loading.jl:128
in reload_path at ./loading.jl:152
in _require at ./loading.jl:67
in require at ./loading.jl:52
in require3911 at /PRJ/julia/usr/bin/../lib/julia/sys.so
while loading /home/kuba/.julia/v0.4/Cxx/src/Cxx.jl, in expression starting on line 37

Keno · 2014-10-06T23:01:16Z

Can you try dlopen of the absolute path and see if that gives you a better error?

kaarrot · 2014-10-06T23:02:52Z

undefined symbol: _ZN4llvm18format_object_base4homeEv

Keno · 2014-10-06T23:04:19Z

Did you build LLVM as a shared library by setting USE_LLVM_SHLIB=1 in Make.user?

kaarrot · 2014-10-06T23:16:17Z

Yes I have it set up in my Make.user.
But later on when I built Clang manually do I need to configure Makefile and pass that flag? I use cmake to build all of llvm btw.

Keno · 2014-10-06T23:16:54Z

You may have to rebuild libjulia.

kaarrot · 2014-10-06T23:48:07Z

Hmmmm that didn't help. Do I think this may be related with missing libcxxabi, libcxx or compiler-rt?

Keno · 2014-10-06T23:51:34Z

No, i'm not using any of these projects. Can you do ldd libjulia.so.

Keno · 2014-10-06T23:54:21Z

sorry, didn't mean to close :). Very unfortunate button placement.

kaarrot · 2014-10-06T23:57:21Z

$ ldd libjulia.so
linux-vdso.so.1 => (0x00007fffac759000)
librt.so.1 => /lib64/librt.so.1 (0x00007f6656015000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f6655e10000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007f6655bef000)
libatomic.so.1 => /home/kuba/toolchains/lib64/libatomic.so.1 (0x00007f66559e8000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f66557ca000)
libz.so.1 => /lib64/libz.so.1 (0x00007f66555b4000)
libstdc++.so.6 => /home/kuba/toolchains/lib64/libstdc++.so.6 (0x00007f66552ab000)
libm.so.6 => /lib64/libm.so.6 (0x00007f6655026000)
libgcc_s.so.1 => /home/kuba/toolchains/lib64/libgcc_s.so.1 (0x00007f6654e10000)
libc.so.6 => /lib64/libc.so.6 (0x00007f6654a7d000)
/lib64/ld-linux-x86-64.so.2 (0x0000003419400000)

Keno · 2014-10-06T23:58:11Z

Ok, so it's not linking the LLVM shared library. Try make clean && make in the julia directory?

kaarrot · 2014-10-06T23:59:48Z

Keno, which linux distro did you test this on? I may try this evening from scratch with more modern setup.

Keno · 2014-10-07T00:00:30Z

I tested this on Ubuntu (14.04 I believe).

kaarrot · 2014-10-07T00:16:12Z

Where USE_LLVM_SHLIB is used? After it is defined in Makefile.inc I can't find it is used anywhere else?

Keno · 2014-10-07T00:17:24Z

Here: https://github.com/JuliaLang/julia/blob/master/src/Makefile#L24
and here: https://github.com/Keno/Cxx.jl/blob/master/deps/BuildBootstrap.Makefile#L51

kaarrot · 2014-10-07T00:41:41Z

Somethings tells me the USE_LLVM_SHLIB does not make into .src/Makefile

As a test, I commented out the if lines jus to see what happens:

#ifeq ($(USE_LLVM_SHLIB),1)
LLVMLINK = $(call exec,$(LLVM_CONFIG) --ldflags) -lLLVM-$(call exec,$(LLVM_CONFIG) --version)
#endif

...but I don't understand the error :)
libjulia.so: undefined reference to llvm::isCurrentDebugType(char const*)' libjulia.so: undefined reference tollvm::DebugFlag'

Does that mean the llvm libraries to link with are not in the LD_LIBRARY_PATH?

kaarrot · 2014-10-07T17:51:50Z

Ok, I think I have Julia linked with LLVM lib now. I had to hack $LLVMLINK in Julia Makefile to make it work.
At this point I'm sure in my setup, for some reason USE_LLVM_SHLIB is evaluated always to 0

Here is the output:

$ ldd libjulia.so
linux-vdso.so.1 => (0x00007fff955ff000)
libLLVM-3.6svn.so => /PRJ/julia_all/julia/usr/lib/./libLLVM-3.6svn.so (0x00007fc4ba088000)
librt.so.1 => /lib64/librt.so.1 (0x00007fc4b9e54000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fc4b9c50000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007fc4b9a2f000)
libatomic.so.1 => /home/kuba/toolchains/lib64/libatomic.so.1 (0x00007fc4b9827000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fc4b960a000)
libz.so.1 => /lib64/libz.so.1 (0x00007fc4b93f4000)
libstdc++.so.6 => /home/kuba/toolchains/lib64/libstdc++.so.6 (0x00007fc4b90ea000)
libm.so.6 => /lib64/libm.so.6 (0x00007fc4b8e66000)
libgcc_s.so.1 => /home/kuba/toolchains/lib64/libgcc_s.so.1 (0x00007fc4b8c50000)
libc.so.6 => /lib64/libc.so.6 (0x00007fc4b88bc000)
/lib64/ld-linux-x86-64.so.2 (0x0000003419400000)

After rebuilding Julia and Pkg.build("Cxx") I get:

ERROR: error compiling init: could not load module libcxxffi: libcxxffi: cannot open shared object file: No such file or directory
in include at ./boot.jl:245
in include_from_node1 at ./loading.jl:128
in reload_path at ./loading.jl:152
in _require at ./loading.jl:67
in require at ./loading.jl:52
in require3935 at /PRJ/julia_all/julia/usr/bin/../lib/julia/sys.so
while loading /home/kuba/.julia/v0.4/Cxx/src/Cxx.jl, in expression starting on line 37

I double checked and the libcxxffi.so is in my $LD_LIBRARY_PATH.

Keno · 2014-10-07T17:52:59Z

The error message isn't great. Where did you set USE_LLVM_SHLIB? The libcxxffi makefile uses it too, so if there's a problem you'll have to change it there too.

kaarrot · 2014-10-07T18:00:07Z

Yes, I also did change these lines in BuildBootstrap.Makefile
#ifeq ($(USE_LLVM_SHLIB),1)
LDFLAGS += -lLLVM-$(call exec,$(LLVM_CONFIG) --version)
#endif

Keno · 2014-10-07T18:01:52Z

Did you rebuild libcxxffi afterwards (you can verify using ldd). Afterwards try dlopening the path directly again. The error message is pretty bad otherwise.

kaarrot · 2014-10-07T18:07:25Z

$ ldd libcxxffi.so
linux-vdso.so.1 => (0x00007fff355be000)
libjulia.so => /PRJ/julia_all/julia/usr/lib/libjulia.so (0x00007f8f332df000)
libLLVM-3.6svn.so => /PRJ/julia_all/julia/usr/lib/libLLVM-3.6svn.so (0x00007f8f31e3b000)
libstdc++.so.6 => /home/kuba/toolchains/lib64/libstdc++.so.6 (0x00007f8f31b32000)
libm.so.6 => /lib64/libm.so.6 (0x00007f8f31883000)
libgcc_s.so.1 => /home/kuba/toolchains/lib64/libgcc_s.so.1 (0x00007f8f3166c000)
libc.so.6 => /lib64/libc.so.6 (0x00007f8f312d9000)
librt.so.1 => /lib64/librt.so.1 (0x00007f8f310d1000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f8f30ecc000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007f8f30cab000)
libatomic.so.1 => /home/kuba/toolchains/lib64/libatomic.so.1 (0x00007f8f30aa4000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f8f30886000)
libz.so.1 => /lib64/libz.so.1 (0x00007f8f30670000)
/lib64/ld-linux-x86-64.so.2 (0x0000003419400000)

julia> dlopen("/home/kuba/.julia/v0.4/Cxx/deps/usr/lib")
ERROR: could not load module /home/kuba/.julia/v0.4/Cxx/deps/usr/lib: /home/kuba/.julia/v0.4/Cxx/deps/usr/lib: cannot read file data: Is a directory
in dlopen at ./c.jl:28

Keno · 2014-10-07T18:11:05Z

By the path I meant the full path to the file (libcxxffi.so).

kaarrot · 2014-10-07T18:27:17Z

Sorry, my bad.
julia> dlopen("/home/kuba/.julia/v0.4/Cxx/deps/usr/lib/libcxxffi.so")
Ptr{Void} @0x00000000042dbf40

Keno · 2014-10-07T18:29:11Z

Ok, that's good. I don't know why it can't find it, but I think if you dlopen it manually before doing using Cxx, you should be fine.

kaarrot · 2014-10-07T18:34:47Z

Then, If I type "using Cxx" Julia crashes.

signal (11): Segmentation fault
unknown function (ip: 8747958)
init at /home/kuba/.julia/v0.4/Cxx/src/Cxx.jl:35
unknown function (ip: -1346084784)
jl_apply_generic at /PRJ/julia_all/julia/src/gf.c:1641
do_call at /PRJ/julia_all/julia/src/interpreter.c:66
eval at /PRJ/julia_all/julia/src/interpreter.c:207
...
...

Keno · 2014-10-07T18:38:01Z

That's progress :). Are you running the debug or release version on julia?

kaarrot · 2014-10-07T18:44:56Z

Heh, I was about to add the following:

FYI, I still haven't built LLDB.
Also, when building Cxx completes I get the message:

julia> Pkg.build("Cxx")
INFO: Building Cxx
Tuning for julia installation at: /PRJ/julia_all/julia/usr/bin
Not building debug library because corresponding julia DEBUG library does not exist.
To build, simply run the build again once the library at
/PRJ/julia_all/julia/usr/bin/../../usr/lib/libjulia-debug.so
has been built.

kaarrot · 2014-10-07T18:55:38Z

How do I build debug julia?

jakebolewski · 2014-10-07T19:06:11Z

make debug

kaarrot · 2014-10-07T19:26:09Z

Ok, pasting the output from the debug version:
julia> Pkg.build("Cxx")
INFO: Building Cxx
Tuning for julia installation at: /PRJ/julia_all/julia/usr/bin
Not building release library because corresponding julia RELEASE library does not exist.
To build, simply run the build again once the library at
/PRJ/julia_all/julia/usr/bin/../../usr/lib/libjulia.so
has been built.
LINK /home/kuba/.julia/v0.4/Cxx/deps/usr/lib/libcxxffi-debug.so

julia> dlopen("/home/kuba/.julia/v0.4/Cxx/deps/usr/lib/libcxxffi-debug.so")
Ptr{Void} @0x0000000003927540

julia> using Cxx

signal (11): Segmentation fault
unknown function (ip: 8747966)
init at /home/kuba/.julia/v0.4/Cxx/src/Cxx.jl:35
unknown function (ip: -156741520)
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
jl_trampoline at /PRJ/julia_all/julia/src/builtins.c:822
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
jl_apply_generic at /PRJ/julia_all/julia/src/gf.c:1640
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
do_call at /PRJ/julia_all/julia/src/interpreter.c:65
eval at /PRJ/julia_all/julia/src/interpreter.c:207
jl_interpret_toplevel_expr at /PRJ/julia_all/julia/src/interpreter.c:26
jl_toplevel_eval_flex at /PRJ/julia_all/julia/src/toplevel.c:496
jl_parse_eval_all at /PRJ/julia_all/julia/src/toplevel.c:544
jl_load at /PRJ/julia_all/julia/src/toplevel.c:583
jl_load_ at /PRJ/julia_all/julia/src/toplevel.c:592
include at ./boot.jl:245
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
jl_apply_generic at /PRJ/julia_all/julia/src/gf.c:1621
include_from_node1 at ./loading.jl:128
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
jl_apply_generic at /PRJ/julia_all/julia/src/gf.c:1621
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
do_call at /PRJ/julia_all/julia/src/interpreter.c:65
eval at /PRJ/julia_all/julia/src/interpreter.c:207
jl_interpret_toplevel_expr at /PRJ/julia_all/julia/src/interpreter.c:26
jl_toplevel_eval_flex at /PRJ/julia_all/julia/src/toplevel.c:496
jl_toplevel_eval at /PRJ/julia_all/julia/src/toplevel.c:520
jl_toplevel_eval_in at /PRJ/julia_all/julia/src/builtins.c:417
jl_f_top_eval at /PRJ/julia_all/julia/src/builtins.c:447
reload_path at ./loading.jl:152
require at ./loading.jl:67
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
jl_trampoline at /PRJ/julia_all/julia/src/builtins.c:822
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
jl_apply_generic at /PRJ/julia_all/julia/src/gf.c:1621
require at ./loading.jl:52
require3923 at /PRJ/julia_all/julia/usr/bin/../lib/julia/sys.so (unknown line)
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
jl_trampoline at /PRJ/julia_all/julia/src/builtins.c:822
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
jl_apply_generic at /PRJ/julia_all/julia/src/gf.c:1621
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
eval_import_path at /PRJ/julia_all/julia/src/toplevel.c:332
eval_import_path at /PRJ/julia_all/julia/src/toplevel.c:359
jl_toplevel_eval_flex at /PRJ/julia_all/julia/src/toplevel.c:406
jl_toplevel_eval at /PRJ/julia_all/julia/src/toplevel.c:520
jl_toplevel_eval_in at /PRJ/julia_all/julia/src/builtins.c:417
jl_f_top_eval at /PRJ/julia_all/julia/src/builtins.c:447
eval_user_input at ./REPL.jl:54
unknown function (ip: -55736505)
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
jl_apply_generic at /PRJ/julia_all/julia/src/gf.c:1621
anonymous at ./task.jl:89
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
jl_trampoline at /PRJ/julia_all/julia/src/builtins.c:822
jl_apply at /PRJ/julia_all/julia/src/julia.h:991
start_task at /PRJ/julia_all/julia/src/task.c:426
switch_stack at /PRJ/julia_all/julia/src/task.c:207
jl_switch_stack at /PRJ/julia_all/julia/src/task.c:218
jl_handle_stack_switch at /PRJ/julia_all/julia/src/task.c:435
julia_trampoline at /PRJ/julia_all/julia/src/init.c:1016
unknown function (ip: 4204057)
__libc_start_main at /lib64/libc.so.6 (unknown line)
unknown function (ip: 4199881)
Segmentation fault

kaarrot · 2014-10-09T01:01:41Z

If I decide to build LLVM separately what flags should I provide to configure script?
At the moment I use:
./configure --enable-shared
Do also include: --enable-libffi?

kaarrot · 2014-10-09T01:55:57Z

Another update.... So I rebuilt LLVM/Clang/compiler-rt from scratch, rebuild Julia, Cxx.... and no crash so far.... Not sure about these warnings. Can I assume all works? :0

julia> dlopen("/home/kuba/.julia/v0.4/Cxx/deps/usr/lib/libcxxffi.so")
Ptr{Void} @0x0000000002b904d0

julia> using Cxx
WARNING: Could not add directory/usr/include/c++/4.8to clang search path!
WARNING: Could not add directory/usr/include/x86_64-linux-gnu/c++/4.8/to clang search path!
WARNING: Could not add directory/usr/include/x86_64-linux-gnuto clang search path!
Warning: requiring "Cxx" did not define a corresponding module.

julia>

kaarrot · 2014-10-09T01:56:57Z

oh, forgot to mention. I configured llvm with: ../configure --enable-shared --enable-pic --enable-libffi

Keno · 2014-10-09T01:58:06Z

Yes, that looks good. The warnings should be harmless, but you may have problems with headers not found, in which case you need to add the proper paths for your system to the search path.

kaarrot · 2014-10-10T15:30:37Z

Building and loading Cxx seems to be working at the moment. I will open a separate issue regarding new runtime problems I bumped into.

Keno · 2014-10-10T17:15:06Z

Awesome, thanks for being so patient.

timholy · 2014-12-07T14:18:01Z

I'm giving this a whirl and experiencing the same error. Let's get this to the point where it's easier, and described in the README---so if possible I'd like to minimize the number of manual build steps.

I added this Make.user file:

BUILD_LLVM_CLANG=1
USE_LLVM_SHLIB=1

then did

make -C deps distclean-openblas distclean-arpack distclean-suitesparse && make cleanall && make

However, I got the exact same missing file error shown at the top of this issue.

timholy · 2014-12-07T14:20:39Z

Should have added: Kubuntu 14.04, and

$ find . -iname "CodeGenModule.h"
./deps/llvm-3.2/tools/clang/lib/CodeGen/CodeGenModule.h

timholy · 2014-12-07T14:23:39Z

However, I don't have a deps/llvm-svn/tools/clang directory. I do have a deps/llvm-3.2/tools/clang directory, obviously.

jtravs · 2014-12-07T15:16:16Z

For me it works on OSX with this Make.user:

USE_SYSTEM_LIBM=1
LLDB_VER=master
LLVM_VER=svn
LLVM_ASSERTIONS=1
#LLVM_DEBUG=1
BUILD_LLVM_CLANG=1
BUILD_LLDB=1
USE_LLVM_SHLIB=1
LLDB_DISABLE_PYTHON=1

and a clean install, using this branch of julia: JuliaLang/julia#9263

timholy · 2014-12-07T17:04:14Z

Thanks. I commented out the USE_SYSTEM_LIBM. The BUILD_LLDB triggers the following error:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404 Not Found
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404 Not Found
make[2]: *** [lldb-svn.src.tar.gz] Error 22
make[1]: *** [julia-release] Error 2
make: *** [release] Error 2

If I comment it out, after a long build process I still get that same CodeGen/CodeGenModule.h: No such file or directory error.

timholy · 2014-12-07T17:30:29Z

Basically it looks like those links are broken in deps/Makefile.

If I use version 3.5.0 instead of master and svn, it seems that the download part succeeds. Not sure whether 3.5.0 is an acceptable version of these packages, though. I'm giving it a try.

However, I wonder if we should be downloading via svn co or git clone from the mirrors. Those URLs might be more stable.

jtravs · 2014-12-07T19:07:03Z

That's why you need the fix Keno added in JuliaLang/julia#9263
I had the exact same issue in #4

timholy · 2014-12-08T01:20:05Z

Got it. Thanks to you both. (I see @Keno merged that commit.)

Keno closed this as completed Oct 6, 2014

Keno reopened this Oct 6, 2014

kaarrot closed this as completed Oct 10, 2014

protogeezer mentioned this issue Feb 18, 2016

Crash during qttest.jl... with llvm 3.7.1 and recent v0.5 julia sources #229

Open

missing CodeGenModule.h #18

missing CodeGenModule.h #18

Comments

kaarrot commented Oct 6, 2014

Keno commented Oct 6, 2014

kaarrot commented Oct 6, 2014

Keno commented Oct 6, 2014

kaarrot commented Oct 6, 2014

kaarrot commented Oct 6, 2014

Keno commented Oct 6, 2014

kaarrot commented Oct 6, 2014

Keno commented Oct 6, 2014

kaarrot commented Oct 6, 2014

Keno commented Oct 6, 2014

kaarrot commented Oct 6, 2014

Keno commented Oct 6, 2014

Keno commented Oct 6, 2014

kaarrot commented Oct 6, 2014

Keno commented Oct 6, 2014

kaarrot commented Oct 6, 2014

Keno commented Oct 7, 2014

kaarrot commented Oct 7, 2014

Keno commented Oct 7, 2014

kaarrot commented Oct 7, 2014

kaarrot commented Oct 7, 2014

Keno commented Oct 7, 2014

kaarrot commented Oct 7, 2014

Keno commented Oct 7, 2014

kaarrot commented Oct 7, 2014

Keno commented Oct 7, 2014

kaarrot commented Oct 7, 2014

Keno commented Oct 7, 2014

kaarrot commented Oct 7, 2014

Keno commented Oct 7, 2014

kaarrot commented Oct 7, 2014

kaarrot commented Oct 7, 2014

jakebolewski commented Oct 7, 2014

kaarrot commented Oct 7, 2014

kaarrot commented Oct 9, 2014

kaarrot commented Oct 9, 2014

kaarrot commented Oct 9, 2014

Keno commented Oct 9, 2014

kaarrot commented Oct 10, 2014

Keno commented Oct 10, 2014

timholy commented Dec 7, 2014

timholy commented Dec 7, 2014

timholy commented Dec 7, 2014

jtravs commented Dec 7, 2014

timholy commented Dec 7, 2014

timholy commented Dec 7, 2014

jtravs commented Dec 7, 2014

timholy commented Dec 8, 2014