Fix races in AppInitMain and others with lock and atomic bools #11107

meshcollider · 2017-08-22T09:05:56Z

Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916

d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex

jonasschnelli · 2017-08-22T09:35:33Z

src/init.cpp

@@ -1632,8 +1632,12 @@ bool AppInitMain(boost::thread_group& threadGroup, CScheduler& scheduler)
    // ********************************************************* Step 11: start node

    //// debug print
-    LogPrintf("mapBlockIndex.size() = %u\n",   mapBlockIndex.size());
-    LogPrintf("nBestHeight = %d\n",                   chainActive.Height());
+    {


I guess it would make sense to also cover mapBlockIndex and chainActive to make this move-safe.

chainActive.Height() is also accessed on line 1652, you want to bring it in too?

If chainActive is added then update descriptions accordingly.

Did you search for other cases?

No haven't modified the pull yet, it still just fixes the issue linked

Indeed, ight as well pull chainActive.Height() into the cs_main here.

meshcollider · 2017-08-22T09:45:57Z

Travis error looks unrelated, timeout in sendheaders.py - assert(wait_until(test_function, timeout=60))

TheBlueMatt · 2017-08-23T19:22:42Z

While you're at it, want to make fReindex atomic so that the ThreadImport write doesn't "race" with net_processing reads?

meshcollider · 2017-09-05T01:36:57Z

Ping @TheBlueMatt if you have a bit of time :)

TheBlueMatt · 2017-09-06T00:31:29Z

src/init.cpp

+        LogPrintf("mapBlockIndex.size() = %u\n", mapBlockIndex.size());
+    }
+    LogPrintf("nBestHeight = %d\n", chainActive.Height());
+


nit: EOL whitespace here.

TheBlueMatt · 2017-09-06T00:35:41Z

src/validation.cpp

@@ -3523,7 +3523,7 @@ bool static LoadBlockIndexDB(const CChainParams& chainparams)
    // Check whether we need to continue reindexing
    bool fReindexing = false;
    pblocktree->ReadReindexing(fReindexing);
-    fReindex |= fReindexing;
+    fReindex = fReindex | fReindexing;


Might as well use the atomic or here since we're making it atomic, no?

|=? It is only defined on atomic integral types I believe, not bool

Err, indeed, quivalent is to do an if(fReindexing) fReindex = true

Yeah true that's cleaner, will do

TheBlueMatt · 2017-09-06T00:36:06Z

src/init.cpp

@@ -1632,8 +1632,12 @@ bool AppInitMain(boost::thread_group& threadGroup, CScheduler& scheduler)
    // ********************************************************* Step 11: start node

    //// debug print
-    LogPrintf("mapBlockIndex.size() = %u\n",   mapBlockIndex.size());
-    LogPrintf("nBestHeight = %d\n",                   chainActive.Height());
+    {


Indeed, ight as well pull chainActive.Height() into the cs_main here.

TheBlueMatt · 2017-09-07T18:47:35Z

utACK 731065b

jtimon · 2017-09-07T19:59:34Z

src/validation.cpp

@@ -3523,7 +3523,7 @@ bool static LoadBlockIndexDB(const CChainParams& chainparams)
    // Check whether we need to continue reindexing
    bool fReindexing = false;
    pblocktree->ReadReindexing(fReindexing);
-    fReindex |= fReindexing;


What's wrong with |= ?

See my reply to @TheBlueMatt's comment above, it's not implemented for atomic bool :)

jtimon · 2017-09-07T20:00:03Z

utACK 731065b

meshcollider · 2017-09-09T23:30:14Z

Added a new commit to make fUseCrypto atomic as well, see #10916 (comment) @TheBlueMatt @jonasschnelli

meshcollider · 2017-09-24T01:46:57Z

Friendly ping @jonasschnelli and/or @TheBlueMatt if you have time, the last commit is only a 2 line change so shouldn't be too time consuming to review ;)

TheBlueMatt · 2017-10-03T23:30:46Z

utACK c626dcb

jonasschnelli · 2017-10-04T05:36:34Z

utACK c626dcb
Generally "unrelated" cleanups (like 731065b) should be avoided.

@TheBlueMatt

…c bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes #11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

@TheBlueMatt

…d atomic bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes bitcoin#11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 bitcoin@d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

@TheBlueMatt

…d atomic bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes bitcoin#11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 bitcoin@d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

@TheBlueMatt

…d atomic bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes bitcoin#11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 bitcoin@d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

@TheBlueMatt

…d atomic bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes bitcoin#11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 bitcoin@d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

@TheBlueMatt

…d atomic bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes bitcoin#11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 bitcoin@d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

@TheBlueMatt

…d atomic bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes bitcoin#11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 bitcoin@d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

@TheBlueMatt

…d atomic bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes bitcoin#11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 bitcoin@d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

@TheBlueMatt

…d atomic bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes bitcoin#11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 bitcoin@d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

@TheBlueMatt

…d atomic bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes bitcoin#11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 bitcoin@d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

@TheBlueMatt

…d atomic bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes bitcoin#11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 bitcoin@d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

@TheBlueMatt

…d atomic bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes bitcoin#11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 bitcoin@d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

@TheBlueMatt

…d atomic bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes bitcoin#11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 bitcoin@d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

@TheBlueMatt

…d atomic bools c626dcb Make fUseCrypto atomic (MeshCollider) 731065b Consistent parameter names in txdb.h (MeshCollider) 35aeabe Make fReindex atomic to avoid race (MeshCollider) 58d91af Fix race for mapBlockIndex in AppInitMain (MeshCollider) Pull request description: Fixes bitcoin#11106 Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916 bitcoin@d291e76 just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42

Fix races in AppInitMain and others with lock and atomic bools (Bitcoin backport) Backport bitcoin bitcoin/bitcoin#11107 (excluding the last commit, it was fixed in Zcash before). This functionality is required for #4368 (details #4368 (comment)) as part of the issue #3813 resolution.

zcash: cherry picked from commit c626dcb zcash: bitcoin/bitcoin#11107

Bitcoin 0.16 locking PRs These are locking changes from upstream (bitcoin core) release 0.16 (Aug 14, 2017, to Feb 16, 2018), oldest to newest (when merged to the master branch). Each commit also includes a reference both to the PR and the upstream commit. - bitcoin/bitcoin#11126 - Excludes changes to wallet tests that we don't have. - bitcoin/bitcoin#10916 - first commit only; second commit already merged by d9fcc2b - bitcoin/bitcoin#11107 - Only the last commit. - bitcoin/bitcoin#11593 - bitcoin/bitcoin#11585 - bitcoin/bitcoin#11618 - bitcoin/bitcoin#10286 - Only the third and last commits. - bitcoin/bitcoin#11870 - bitcoin/bitcoin#12330 - bitcoin/bitcoin#12366 - bitcoin/bitcoin#12368 - bitcoin/bitcoin#12333 - Only the first commit.

7a5d181 Use the character based overload for std::string::find. (Alin Rus) c19401b Don't use pass by reference to const for cheaply-copied types (bool, char, etc.). (practicalswift) 4c5fe36 [Refactor] Remove unused fQuit var from checkqueue.h (donaloconnor) fda7a5f Cleanup: remove unneeded header includes (random-zebra) ac2476c Add test cases covering the relevant key length boundaries: 64 bytes +/- 1 byte for HMAC-SHA256 and 128 bytes +/- 1 byte for HMAC-SHA512 (practicalswift) a346262 Fix code constness in CBlockIndex::GetAncestor() overloads (Dan Raviv) 65e3f4e Refactor: More range-based for loops (random-zebra) dd3d3c4 Use range-based for loops (C++11) when looping over map elements (practicalswift) 5a7750a Move RPC registration out of AppInitParameterInteraction (Russell Yanofsky) 402b4c4 Use compile-time constants instead of unnamed enumerations (practicalswift) bbac2d0 [Trivial] Fix indentation in coins.cpp (random-zebra) e539c62 Small refactor of CCoinsViewCache::BatchWrite() (Dan Raviv) ec91759 Use MakeUnique<T>(...) instead of std::unique_ptr<T>(new T(...)) (random-zebra) 93487b1 Use unique_ptr for pcoinscatcher/pcoinsdbview/pcoinsTip/pblocktree (random-zebra) ff43d69 Use unique_ptr for pdbCopy (Db) and fix potential memory leak (practicalswift) b4d9641 Use unique_ptr for dbenv (DbEnv) (practicalswift) 36108b9 Use unique_ptr for pfilter (CBloomFilter) (practicalswift) ff1c454 Use unique_ptr for sem{Addnode,Outbound} (CSemaphore) (practicalswift) 93daf17 Use unique_ptr for httpRPCTimerInterface (HTTPRPCTimerInterface) (practicalswift) 020ac16 Init: Remove redundant exit(EXIT_FAILURE) instances and replace with (random-zebra) b9f5d1f Remove duplicate uriParts.size() > 0 check (practicalswift) 440d961 Remove redundant check (!ecc is always true) (practicalswift) bfd295b Remove redundant NULL checks after new (practicalswift) 97aad32 Make fUseCrypto atomic (MeshCollider) 2711f78 Consistent parameter names in txdb.h (MeshCollider) d40df3a Fix race for mapBlockIndex in AppInitMain (random-zebra) 03b7766 Cleanup: remove unused functions to Hash the concat of 4 or more objects (random-zebra) c520e0f Remove some unused functions and methods (Pieter Wuille) 508f1a1 range-based loops and const qualifications in net.cpp (Marko Bencun) 79b1e50 Refactor: implement CPubKey::data() (random-zebra) 614d26c Refactor: more &vec[0] to vec.data() (random-zebra) 02b6337 Changing &vec[0] to vec.data(), what 9804 missed (MeshCollider) c1c8b05 Ensure that data types are consistent (jjz) 732bb9d Fix potential null dereferences (MeshCollider) 80f35f9 Remove unreachable code (practicalswift) Pull request description: This is a collection of simple refactorings coming from upstream Bitcoin v0.16 (adapting/extending to PIVX-specific code where needed). Pull requests backported: - bitcoin#10845 (practicalswift) - bitcoin#11238 (MeshCollider) - bitcoin#11232 (jjz) - bitcoin#10793 (MeshCollider) - bitcoin#10888 (benma) - ~~bitcoin#11351 (danra)~~ [edit: removed - included in #2423] - bitcoin#11385 (sipa) - bitcoin#11107 (MeshCollider) - bitcoin#10898 (practicalswift) - bitcoin#11511 (donaloconnor) - bitcoin#11043 (practicalswift) - bitcoin#11353 (danra) - bitcoin#10749 (practicalswift) - bitcoin#11603 (ryanofsky) - bitcoin#10493 (practicalswift) - bitcoin#11337 (danra) - bitcoin#11516 (practicalswift) - bitcoin#10574 (practicalswift) - bitcoin#12108 (donaloconnor) - bitcoin#10839 (practicalswift) - bitcoin#12159 (kekimusmaximus) ACKs for top commit: Fuzzbawls: ACK 7a5d181 furszy: re-ACK 7a5d181 after rebase, no code changes. Merging.. Tree-SHA512: d92f5df47f443391a6531274a2efb9a4882c62d32eff628f795b3abce703f108c8b40ec80ac841cde6c5fdd5c9ff2b6056a31546ac2edda279f5f18fccc99c32

jonasschnelli reviewed Aug 22, 2017

View reviewed changes

jonasschnelli added the Refactoring label Aug 22, 2017

TheBlueMatt reviewed Sep 6, 2017

View reviewed changes

meshcollider added 3 commits September 7, 2017 12:00

Fix race for mapBlockIndex in AppInitMain

58d91af

Make fReindex atomic to avoid race

35aeabe

Consistent parameter names in txdb.h

731065b

jtimon reviewed Sep 7, 2017

View reviewed changes

Make fUseCrypto atomic

c626dcb

meshcollider changed the title ~~Fix race for mapBlockIndex in AppInitMain~~ Fix races in AppInitMain and others with lock and atomic bools Sep 9, 2017

maflcko merged commit c626dcb into bitcoin:master Oct 5, 2017

meshcollider deleted the fix_mapBlockIndex_race branch October 9, 2017 22:24

daira mentioned this pull request Mar 5, 2020

Shows reindex progress in metrics screen zcash/zcash#4368

Merged

gladcow mentioned this pull request Mar 7, 2020

Fix races in AppInitMain and others with lock and atomic bools (Bitcoin backport) zcash/zcash#4387

Merged

LarryRuane mentioned this pull request Mar 3, 2021

Bitcoin 0.16 locking PRs zcash/zcash#5017

Merged

LarryRuane pushed a commit to LarryRuane/zcash that referenced this pull request Apr 13, 2021

Make fUseCrypto atomic

ea9bcc4

zcash: cherry picked from commit c626dcb zcash: bitcoin/bitcoin#11107

random-zebra mentioned this pull request Jun 29, 2021

[Refactoring] Bitcoin 0.16 cherries PIVX-Project/PIVX#2462

Merged

bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix races in AppInitMain and others with lock and atomic bools #11107

Fix races in AppInitMain and others with lock and atomic bools #11107

meshcollider commented Aug 22, 2017 •

edited

jonasschnelli Aug 22, 2017

meshcollider Aug 22, 2017 •

edited

promag Aug 23, 2017

meshcollider Aug 23, 2017

TheBlueMatt Sep 6, 2017

meshcollider commented Aug 22, 2017 •

edited

TheBlueMatt commented Aug 23, 2017

meshcollider commented Sep 5, 2017

TheBlueMatt Sep 6, 2017

TheBlueMatt Sep 6, 2017

meshcollider Sep 6, 2017

TheBlueMatt Sep 6, 2017

meshcollider Sep 7, 2017

TheBlueMatt Sep 6, 2017

TheBlueMatt commented Sep 7, 2017

jtimon Sep 7, 2017

meshcollider Sep 7, 2017

jtimon commented Sep 7, 2017

meshcollider commented Sep 9, 2017 •

edited

meshcollider commented Sep 24, 2017

TheBlueMatt commented Oct 3, 2017

jonasschnelli commented Oct 4, 2017

Fix races in AppInitMain and others with lock and atomic bools #11107

Fix races in AppInitMain and others with lock and atomic bools #11107

Conversation

meshcollider commented Aug 22, 2017 • edited

Choose a reason for hiding this comment

meshcollider Aug 22, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

meshcollider commented Aug 22, 2017 • edited

TheBlueMatt commented Aug 23, 2017

meshcollider commented Sep 5, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheBlueMatt commented Sep 7, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jtimon commented Sep 7, 2017

meshcollider commented Sep 9, 2017 • edited

meshcollider commented Sep 24, 2017

TheBlueMatt commented Oct 3, 2017

jonasschnelli commented Oct 4, 2017

meshcollider commented Aug 22, 2017 •

edited

meshcollider Aug 22, 2017 •

edited

meshcollider commented Aug 22, 2017 •

edited

meshcollider commented Sep 9, 2017 •

edited