Support IBM AIX on POWER #6677

NattyNarwhal · 2018-01-25T21:09:32Z

Patch mostly complete

This is a patch to add support for AIX, the IBM Unix. The patch has been tested on AIX 6.1 TL9 on a POWER6 system, with GCC 6.3 sourced from Bull.

The port is capable of compiling itself and being installed. There may be minor issues, but these can be shaken out.

Tip: When using GCC, put -gxcoff in your CFLAGS to generate COFF (the actual AIX binary format) debugging info. Unfortunately, dbx still snarfs on gcc compiled programs and gdb can't load cores either way; but it does let you run under gdb fine.

PASE is most similar to AIX, and perhaps I assume other commercial Unices, as well as PowerPC Linux, so grab variables from them as needed. With this, we get glib failing to build due to missing mkdtemp.

From comment: /* HACK: the preprocessor will not give us mkdtemp no matter what and Mono (for good reason) does "-Werror-implicit-function-declaration" so we error out; instead declare mkdtemp here; the linker will find mkdtemp anwyays. libuv has had similar issues, but they just ignore the compiler warning instead of failing on it. See: github.com/libuv/libuv/pull/740 */

* uncertain about NIP; is IAR the right register? need review from PPC asm people * needs calling convention work; AIX does NOT use ELF

* AIX doesn't distinguish between ENOTEMPTY and EEXIST without a define. Handle this case with a conditional cpp directive.. * When ctx is a void pointer, dereferencing as a struct will fail. Handle this case with a cast.

* vol info should be like other Unix, just needs to bridge ifdef * highly uncertain for the bounds checking, needs further review

It's not available on IBM i, and even if available, toolchains for AIX may "helpfully" swap out system headers with variants that remove functions like pthread_getattr_np.

IBM ld considers "-e" to be a flag for setting the entry point. Detect non-GNU ld, and offer equivalents when possible. The makefile in mini had to be changed as a result due to hardcoded flags.

Remove an ifdef that gated this off to AIX. It seems the old AIX specific files shouldn't be used, and that the changelog for libgc says pthread should work fine on AIX instead; yet it was blocked. Restore this, as it compiles and seems to let mono-boehm link fine.

* fstab on AIX is there, but defines a struct more like other platforms' checklist struct. * AIX, like macOS and some BSDs, doesn't define some serial types. * psignal is another victim of GNU; even then it isn't in PASE headers anyways

AIX lacks vsyslog; detect in autoconf and act appropriately.

this gets us past ifdefs, after much handwringing * get the right end of the stack; r1/stack pointer is on the wrong end and ustk is just plain broken * AIX doesn't seem to like the semantics of the mmap call valloc does here; either we're allocating into an area we shouldn't be or we're not passing the right flags. Until then, learn to live without the stack guard?

lewurm · 2018-01-25T21:16:10Z

mono/utils/mono-mmap.c

@@ -351,7 +351,8 @@ mono_mprotect (void *addr, size_t length, int flags)
 			memset (addr, 0, length);
 #else
 		memset (addr, 0, length);
-#ifdef HAVE_MADVISE
+/* AIX doesn't have MADV_FREE */
+#if defined(HAVE_MADVISE) && !defined(_AIX)


why isn't HAVE_MADVISE detected properly by autotools?

We do have that function, but AIX doesn't support the MADV_FREE flag.

is it a define (or enum or const int) on all/most systems? i.e. can you ifdef the flag?

lewurm · 2018-01-25T21:19:07Z

mono/utils/mono-proclib.c

+   defined, so undef */
+#if defined(_AIX) && defined(_ALL_SOURCE)
+#undef hz
+#endif


I would recommend to rename occurrences of hz in this file instead

lewurm · 2018-01-25T21:26:49Z

configure.ac

-LDFLAGS="${LDFLAGS} -Wl,-export-dynamic"
+# GNU specific option, this confuses IBM ld, but do offer alternatives when possible
+if test $lt_cv_prog_gnu_ld = yes; then
+	LDFLAGS="${LDFLAGS} -Wl,-export-dynamic"


not sure if that works with the linker on macOS too.

I wasn't sure if LLVM's linker supports it; has Apple moved to that yet? AFAIK, they're still using GNU binutils even with clang. In any case, this could be extended to check for LLVM's linker and use the proper flag (maybe the same as GNU's) or any other non-GNU linker, really.

I believe Apple has a very old GNU binutils fork, forked in Next days. There is lld but I don't think they use it.

Apple doesn’t currently use either lld or GNU binutils for its linker. Apple’s linker is its own Open Source ld64.

lewurm · 2018-01-25T21:30:55Z

mono/mini/mini-exceptions.c

@@ -2396,12 +2396,15 @@ mono_setup_altstack (MonoJitTlsData *tls)

 	g_assert ((guint8*)&sa >= (guint8*)tls->stack_ovf_guard_base + tls->stack_ovf_guard_size);

+/* the valloc call here seems problematic on AIX */
+#if !defined(_AIX)


ifdefing this part out, makes this code broken and useless. instead we should rather not set MONO_ARCH_SIGSEGV_ON_ALTSTACK in mini-ppc.h.

AIX libraries are very, very strange compared to other Unices. I'm not entirely sure if this is correct or optimal; it may need to special case further. For now, it's enough to get gacutil up and running. Some resources I found useful for this charade: <http://stromberg.dnsalias.org/~dstromberg/AIX-shared-libs.html> Also set libc and libintl to their proper values.

NattyNarwhal · 2018-02-02T04:13:01Z

I'd say this PR is ready to merge. To recap: dynamic linking works, we can call into AIX's weird library style with dlopen, which gets gacutil working, which in turn, makes it possible to make install.

NattyNarwhal · 2018-02-02T04:16:22Z

That being said, I still have some minor things to investigate, but those can be put aside and effort spent on getting this merged.

PASE doesn't accept anything other than SCHED_OTHER, and the funcs for getting min/max don't work either. AIX suffers, but only a little bit.

NattyNarwhal · 2018-02-02T13:54:55Z

Still mergeable, just leaving this up for future reference to debug a DNS issue in Mono:

$ cat testdns.cs
using System;
using System.Net;

class ResolveDNS {
        public static void Main(string[] args) {
                Console.WriteLine(Dns.GetHostByName(args[0]));
        }
}
$ /opt/mono/bin/mono --trace=T:System.Net.Dns testdns.exe github.com
mono_thread_internal_set_priority: pthread_setschedparam failed, error: "Operation not permitted." (1)
[1: 0.00000 0] ENTER: System.Net.Dns:.cctor ()() ip: 7000000000848f4
[1: 0.00202 0] LEAVE: System.Net.Dns:.cctor ()
[1: 0.00207 0] ENTER: System.Net.Dns:GetHostByName (string)() ip: 700000000084088
[1:] EXCEPTION handling: System.EntryPointNotFoundException: getifaddrs
[1:] EXCEPTION handling: System.EntryPointNotFoundException: getifaddrs
[1: 0.04000 1] ENTER: (wrapper managed-to-native) System.Net.Dns:GetHostByName_internal (string,string&,string[]&,string[]&,int)() ip: 700000000084450
[1: 0.05283 1] LEAVE: (wrapper managed-to-native) System.Net.Dns:GetHostByName_internal (string,string&,string[]&,string[]&,int)result=1
[1: 0.05404 1] ENTER: System.Net.Dns:hostent_to_IPHostEntry (string,string,string[],string[])() ip: 7000000000844c0
[1: 0.05879 2] ENTER: System.Net.Dns:Error_11001 (string)() ip: 7000000003d4a50
[1:] EXCEPTION handling: System.Net.Sockets.SocketException: Could not resolve host 'github.com'

Unhandled Exception:
System.Net.Sockets.SocketException (0x80004005): Could not resolve host 'github.com'
  at System.Net.Dns.Error_11001 (System.String hostName) [0x00015] in <d46682f7e6dd42a89e5b7c900df1adad>:0
  at System.Net.Dns.hostent_to_IPHostEntry (System.String originalHostName, System.String h_name, System.String[] h_aliases, System.String[] h_addrlist) [0x00082] in <d46682f7e6dd42a89e5b7c900df1adad>:0
  at System.Net.Dns.GetHostByName (System.String hostName) [0x0002f] in <d46682f7e6dd42a89e5b7c900df1adad>:0
  at ResolveDNS.Main (System.String[] args) [0x00013] in <a0fb5c27bef04bc09d46e68d134f0f47>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.Net.Sockets.SocketException (0x80004005): Could not resolve host 'github.com'
  at System.Net.Dns.Error_11001 (System.String hostName) [0x00015] in <d46682f7e6dd42a89e5b7c900df1adad>:0
  at System.Net.Dns.hostent_to_IPHostEntry (System.String originalHostName, System.String h_name, System.String[] h_aliases, System.String[] h_addrlist) [0x00082] in <d46682f7e6dd42a89e5b7c900df1adad>:0
  at System.Net.Dns.GetHostByName (System.String hostName) [0x0002f] in <d46682f7e6dd42a89e5b7c900df1adad>:0
  at ResolveDNS.Main (System.String[] args) [0x00013] in <a0fb5c27bef04bc09d46e68d134f0f47>:0

(updated test case)

edelsohn · 2018-02-02T15:38:51Z

Congratulations! Glad that Mono now is working on AIX. That's a big accomplishment!

-Wl,-b64 should not be necessary in CPPFLAGS and LDFLAGS because GCC -maix64 implies that already when it invokes the linker.

export OBJECT_MODE=64 should not be necessary with -X64. And hidden, magic, global variables are bad.

libintl.a(libintl.so.9) probably will come back to bit you because different library versionings are floaing around. libintl.so.1, which is 1.0, and libintl.so.9, which is 0.9 and libintl.so.8, which is 0.8.

-Wl,-bexpall should be useless because the explicit export list created by libtool overrides it.

NattyNarwhal · 2018-02-02T17:18:23Z

Next time I try fiddling with configure.ac (maybe see if I can get sigaltstack working, for instance) I'll consider cleaning up the defines there a little.

DNS investigation: could be mcs/class/System/System.Net.NetworkInformation/NetworkInterface.cs hardcoding an assumption on getifaddrs (which doesn't exist on AIX) or some bug in the icall part of GetHostByName_internal.

lewurm · 2018-02-02T18:15:06Z

configure.ac

+		LDFLAGS="${LDFLAGS} -Wl,-bexpall"
+		;;
+	*)
+		;;


I think we should set the same in the default case as in the GNU ld case

lewurm · 2018-02-02T18:15:58Z

libgc/configure.ac

+     *-*-os400*)
+	AC_DEFINE(GC_AIX_THREADS)
+	AC_DEFINE(_REENTRANT)
+	;;
     *-*-haiku*)


undo this change in libgc, we'll likely have a submodule for boehm GC soon, so this would be lost anyway (/cc @joncham )

lewurm · 2018-02-02T18:16:03Z

libgc/pthread_stop_world.c

@@ -2,7 +2,7 @@

 #if defined(GC_PTHREADS) && !defined(GC_SOLARIS_THREADS) \
     && !defined(GC_IRIX_THREADS) && !defined(GC_WIN32_THREADS) \
-     && !defined(GC_DARWIN_THREADS) && !defined(GC_AIX_THREADS)
+     && !defined(GC_DARWIN_THREADS)


undo this change in libgc, we'll likely have a submodule for boehm GC soon, so this would be lost anyway (/cc @joncham )

lewurm · 2018-02-02T18:16:55Z

mono/metadata/boehm-gc.c

+         * ship a "fixed" header file without "non-standard" declarations like it, so don't
+         * use it, even if it's there
+         */
+#if defined(HAVE_PTHREAD_GETATTR_NP) && defined(HAVE_PTHREAD_ATTR_GETSTACK) && !defined(_AIX)


since you don't use boehm, please undo this change

lewurm · 2018-02-02T18:37:36Z

pretty good job, congrats 🙂

looks good overall! do you think you can add a header that collects all those AIX specific extern hacks in support/ and mono/eglib? So it would be nicely in one file and not spread in the codebase.

lewurm · 2018-02-02T18:37:41Z

build

lewurm · 2018-02-02T18:38:02Z

@monojenkins build Linux PPC64 little endian

NattyNarwhal · 2018-02-02T18:43:26Z

I think a lot of the AIX extern hacks can be removed because newer GCC packaged by Bull has better includes; as David said early in the thread, GCC 4.x on AIX was a "bad vintage."

Working on a commit to do this cleanup suggested by you two.

Remove Boehm changes if we don't use Boehm, tweak some defines, remove some externs, tweak autoconf file

NattyNarwhal · 2018-02-02T20:49:30Z

Suggestions from @lewurm done.

lewurm · 2018-02-02T20:53:46Z

build

lewurm · 2018-02-02T20:53:55Z

@monojenkins build Linux PPC64 little endian

NattyNarwhal · 2018-02-03T22:15:48Z

LMK about merging this patch. I have a feeling I'll need to work on the BCL to fix a sockets issue, the same one I mentioned that prevents DNS from working; but it'd be perhaps prudent to branch again in master; and discuss the problem and what'd need to be changed in a new issue/PR.

Instead of EINTR/EAGAIN, AIX uses ERESTART. Handle this, so things like xsp are less chatty when performing async IO.

lewurm · 2018-02-05T07:59:28Z

build

lewurm · 2018-02-05T07:59:36Z

@monojenkins build Linux PPC64 little endian

lewurm · 2018-02-05T07:59:44Z

@monojenkins squash

@vargaz

* Get autotools scripts for Mono and Boehm to recognize i PASE is most similar to AIX, and perhaps I assume other commercial Unices, as well as PowerPC Linux, so grab variables from them as needed. With this, we get glib failing to build due to missing mkdtemp. * Use better cpp definition to grab more of what we want * Fix random and 64-bit file offset support * Set TARGET on IBM i on POWER * Add hack to gfile-unix.c to work around flawed AIX header From comment: /* HACK: the preprocessor will not give us mkdtemp no matter what and Mono (for good reason) does "-Werror-implicit-function-declaration" so we error out; instead declare mkdtemp here; the linker will find mkdtemp anwyays. libuv has had similar issues, but they just ignore the compiler warning instead of failing on it. See: github.com/libuv/libuv/pull/740 */ * Work around missing definition on AIX/PASE * Fix conflicting definition from due too AIX headers * Initial work to get JIT working on AIX * uncertain about NIP; is IAR the right register? need review from PPC asm people * needs calling convention work; AIX does NOT use ELF * Fix lack of casting on sigcontext macros, fix errnos for AIX * AIX doesn't distinguish between ENOTEMPTY and EEXIST without a define. Handle this case with a conditional cpp directive.. * When ctx is a void pointer, dereferencing as a struct will fail. Handle this case with a cast. * Handle regular AIX as well as PASE * Implement stack boundaries check for AIX, volume info func * vol info should be like other Unix, just needs to bridge ifdef * highly uncertain for the bounds checking, needs further review * Fix typo in mono/utils/mono-threads-aix.c * In boehm-gc.c, don't use pthread_getattr_np on AIX even if available It's not available on IBM i, and even if available, toolchains for AIX may "helpfully" swap out system headers with variants that remove functions like pthread_getattr_np. * Don't use "-export-dynamic" on non-GNU ld IBM ld considers "-e" to be a flag for setting the entry point. Detect non-GNU ld, and offer equivalents when possible. The makefile in mini had to be changed as a result due to hardcoded flags. * Use pthread-stop-world.c on AIX Remove an ifdef that gated this off to AIX. It seems the old AIX specific files shouldn't be used, and that the changelog for libgc says pthread should work fine on AIX instead; yet it was blocked. Restore this, as it compiles and seems to let mono-boehm link fine. * AIX workarounds for some of support dir * fstab on AIX is there, but defines a struct more like other platforms' checklist struct. * AIX, like macOS and some BSDs, doesn't define some serial types. * psignal is another victim of GNU; even then it isn't in PASE headers anyways * More support workarounds for AIX headers * Replacement function body for syslog2 on vsyslog-less platforms AIX lacks vsyslog; detect in autoconf and act appropriately. * Final support hacks for AIX * Fix problematic stack management on AIX this gets us past ifdefs, after much handwringing * get the right end of the stack; r1/stack pointer is on the wrong end and ustk is just plain broken * AIX doesn't seem to like the semantics of the mmap call valloc does here; either we're allocating into an area we shouldn't be or we're not passing the right flags. Until then, learn to live without the stack guard? * Disable Boehm GC on AIX Focus on getting SGen working, and avoid legacy debugging nonsense * Remove AIX ifdef-undef and replace with s/hz/uhz * Don't use sigaltstack on AIX due to bugginess with guard sections The valloc call on AIX has issues after issuing mprotect; don't do do it. I believe it fails because we aren't issuing the right flags on AIX, or because we're allocating inside of an already allocated area. * Perform a 64-bit build on AIX We still crash at the same place, but probably for the best that we switch to 64-bit. (Maybe we could make it configurable?) * Fix func descriptor ifdef, add initial (likely wrong) defs for AIX * AIX doesn't define __powerpc64__ or __ppc64__ when compiling as 64-bit. Use an alternate ifdef, and shuffle around that ifdef anyways, as it could lead to wrong results on other platforms too. * AIX likely needs special definitions, especially on 32-bit. The calling convention is closest to PPC64BE Linux, but not exactly. * Fix stack size on AIX Ruby dealt with this before: https://github.com/ruby/ruby/commit/mono/mono@a2594be783c727c6034308f5294333752c3845bb * Set up initial PPC hwcap for AIX, fix ftnptr ifdefs, tweak AIX defs * Fix hwcap to use better values on PPC AIX Still needs more adjustment though * Fix alignment issue on AIX, tweak mini-ppc values just a bit more * On AIX, doubles are always aligned by 4 if they're in structs. Since we need to have them aligned by 8 on 64-bit, this is bad, so change the type on AIX. * We can use this definition on AIX Now we can bootstrap Mono! * Shuffle MonoArray ifdef per suggestion of @vargaz * Fix System.Decimal division on big endian systems Before, we got non-sensical overflow on things like `(Decimal)Int32.MaxValue / 1000`; now they get the same value, comparing this POWER6 running AIX to my server's output running OpenBSD/amd64. * Fix dynamically linked build of Mono on AIX nm wasn't seeing 64-bit symbools as it defaults to 32-bit mode, also never try to use GNU nm * Fix PPC64 define in mono-config Interestingly, there was an AIX define already present, and that definition is present in AIX's passed includes, so continue to use it. * Don't use __thread even if detected on AIX tl;dr: The assembler gets angry with __thread and minimal-toc, and changing options to replace minimal-toc causes linker issues; this TLS option isn't important anyways if we have pthread instead, like before when we were using Perzl's old GCC without __thread. * Get lib.a(lib.o) type archives working with P/Invoke for AIX AIX libraries are very, very strange compared to other Unices. I'm not entirely sure if this is correct or optimal; it may need to special case further. For now, it's enough to get gacutil up and running. Some resources I found useful for this charade: <http://stromberg.dnsalias.org/~dstromberg/AIX-shared-libs.html> Also set libc and libintl to their proper values. * Scheduling priority hack for i PASE doesn't accept anything other than SCHED_OTHER, and the funcs for getting min/max don't work either. AIX suffers, but only a little bit. * Final polish on request Remove Boehm changes if we don't use Boehm, tweak some defines, remove some externs, tweak autoconf file * Be aware of ERESTART, on OSes that use it Instead of EINTR/EAGAIN, AIX uses ERESTART. Handle this, so things like xsp are less chatty when performing async IO. Commit migrated from mono/mono@d568c0b

NattyNarwhal added 20 commits January 19, 2018 23:31

Get autotools scripts for Mono and Boehm to recognize i

f83f0fa

PASE is most similar to AIX, and perhaps I assume other commercial Unices, as well as PowerPC Linux, so grab variables from them as needed. With this, we get glib failing to build due to missing mkdtemp.

Use better cpp definition to grab more of what we want

dcd13e0

Fix random and 64-bit file offset support

99bb228

Set TARGET on IBM i on POWER

6b608a8

Work around missing definition on AIX/PASE

d07a00a

Fix conflicting definition from due too AIX headers

13c912d

Initial work to get JIT working on AIX

8594e2b

* uncertain about NIP; is IAR the right register? need review from PPC asm people * needs calling convention work; AIX does NOT use ELF

Fix lack of casting on sigcontext macros, fix errnos for AIX

09c124f

* AIX doesn't distinguish between ENOTEMPTY and EEXIST without a define. Handle this case with a conditional cpp directive.. * When ctx is a void pointer, dereferencing as a struct will fail. Handle this case with a cast.

Handle regular AIX as well as PASE

98a5a3c

Implement stack boundaries check for AIX, volume info func

64a0d53

* vol info should be like other Unix, just needs to bridge ifdef * highly uncertain for the bounds checking, needs further review

Fix typo in mono/utils/mono-threads-aix.c

11088fd

In boehm-gc.c, don't use pthread_getattr_np on AIX even if available

b1230e6

It's not available on IBM i, and even if available, toolchains for AIX may "helpfully" swap out system headers with variants that remove functions like pthread_getattr_np.

Don't use "-export-dynamic" on non-GNU ld

3de170c

IBM ld considers "-e" to be a flag for setting the entry point. Detect non-GNU ld, and offer equivalents when possible. The makefile in mini had to be changed as a result due to hardcoded flags.

AIX workarounds for some of support dir

8e0ab9d

* fstab on AIX is there, but defines a struct more like other platforms' checklist struct. * AIX, like macOS and some BSDs, doesn't define some serial types. * psignal is another victim of GNU; even then it isn't in PASE headers anyways

More support workarounds for AIX headers

7f0a9c6

Replacement function body for syslog2 on vsyslog-less platforms

d2b934a

AIX lacks vsyslog; detect in autoconf and act appropriately.

Final support hacks for AIX

95c1df5

NattyNarwhal requested review from BrzVlad, kumpera, lewurm, luhenry, marek-safar and vargaz as code owners January 25, 2018 21:09

lewurm reviewed Jan 25, 2018

View reviewed changes

Scheduling priority hack for i

215bc13

PASE doesn't accept anything other than SCHED_OTHER, and the funcs for getting min/max don't work either. AIX suffers, but only a little bit.

lewurm reviewed Feb 2, 2018

View reviewed changes

Final polish on request

fc5f787

Remove Boehm changes if we don't use Boehm, tweak some defines, remove some externs, tweak autoconf file

NattyNarwhal mentioned this pull request Feb 4, 2018

System.Net.NetworkInformation.NetworkInterface refactoring #6778

Open

Be aware of ERESTART, on OSes that use it

26f02fd

Instead of EINTR/EAGAIN, AIX uses ERESTART. Handle this, so things like xsp are less chatty when performing async IO.

lewurm approved these changes Feb 5, 2018

View reviewed changes

lewurm merged commit d568c0b into mono:master Feb 5, 2018

NattyNarwhal mentioned this pull request Oct 11, 2018

Mono fails to build on FreeBSD/PowerPC64 #11021

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support IBM AIX on POWER #6677

Support IBM AIX on POWER #6677

NattyNarwhal commented Jan 25, 2018 •

edited

Loading

lewurm Jan 25, 2018

NattyNarwhal Jan 25, 2018

jaykrell Jan 27, 2018

lewurm Jan 25, 2018

lewurm Jan 25, 2018

NattyNarwhal Jan 25, 2018

jaykrell Jan 27, 2018

eschaton Jan 27, 2018

lewurm Jan 25, 2018

NattyNarwhal commented Feb 2, 2018

NattyNarwhal commented Feb 2, 2018

NattyNarwhal commented Feb 2, 2018 •

edited

Loading

edelsohn commented Feb 2, 2018

NattyNarwhal commented Feb 2, 2018

lewurm Feb 2, 2018

lewurm Feb 2, 2018

lewurm Feb 2, 2018

lewurm Feb 2, 2018

lewurm commented Feb 2, 2018

lewurm commented Feb 2, 2018

lewurm commented Feb 2, 2018

NattyNarwhal commented Feb 2, 2018

NattyNarwhal commented Feb 2, 2018

lewurm commented Feb 2, 2018

lewurm commented Feb 2, 2018

NattyNarwhal commented Feb 3, 2018

lewurm commented Feb 5, 2018

lewurm commented Feb 5, 2018

lewurm commented Feb 5, 2018

Support IBM AIX on POWER #6677

Support IBM AIX on POWER #6677

Conversation

NattyNarwhal commented Jan 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NattyNarwhal commented Feb 2, 2018

NattyNarwhal commented Feb 2, 2018

NattyNarwhal commented Feb 2, 2018 • edited Loading

edelsohn commented Feb 2, 2018

NattyNarwhal commented Feb 2, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lewurm commented Feb 2, 2018

lewurm commented Feb 2, 2018

lewurm commented Feb 2, 2018

NattyNarwhal commented Feb 2, 2018

NattyNarwhal commented Feb 2, 2018

lewurm commented Feb 2, 2018

lewurm commented Feb 2, 2018

NattyNarwhal commented Feb 3, 2018

lewurm commented Feb 5, 2018

lewurm commented Feb 5, 2018

lewurm commented Feb 5, 2018

NattyNarwhal commented Jan 25, 2018 •

edited

Loading

NattyNarwhal commented Feb 2, 2018 •

edited

Loading