Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not compiling on Apple M1 system #8

Closed
AndyRPH opened this issue Jun 7, 2021 · 48 comments
Closed

Not compiling on Apple M1 system #8

AndyRPH opened this issue Jun 7, 2021 · 48 comments

Comments

@AndyRPH
Copy link

AndyRPH commented Jun 7, 2021

Getting some compiling errors, is this info helpful:

-- Configured operating system: MACOSX
-- Configured GMP: /opt/homebrew/lib/libgmp.a
-- Compiler flags:  -Wall   -O3 -funroll-loops -fomit-frame-pointer -pthread
-- Linker flags: -L/Applications/Chia-arm/maxplotter/chia-plotter/build/_deps/relic-src/src/low/gmp/  -static
-- Configured /Applications/Chia-arm/maxplotter/chia-plotter/build/_deps/relic-src/include/relic_conf.h.in
SODIUM_FOUND in src/CMakeLists.txt
-- Configuring done
-- Generating done
-- Build files have been written to: /Applications/Chia-arm/maxplotter/chia-plotter/build
[  0%] Building CXX object CMakeFiles/test_disk_sort.dir/test/test_disk_sort.cpp.o
[  0%] Built target arith_objs
[  2%] Built target blstmp
[  4%] Built target blake3
clang: error: unsupported option '-fopenmp'
[  6%] Built target fse
make[2]: *** [CMakeFiles/test_disk_sort.dir/test/test_disk_sort.cpp.o] Error 1
make[1]: *** [CMakeFiles/test_disk_sort.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
Scanning dependencies of target chia_plotter
[  6%] Building C object CMakeFiles/chia_plotter.dir/lib/chacha8.c.o
[  7%] Linking C static library libchia_plotter.a
[ 92%] Built target relic_s
[ 92%] Built target chia_plotter
make: *** [all] Error 2

@fermosan
Copy link
Contributor

fermosan commented Jun 7, 2021

try to:
brew update
brew install llvm libomp

@AndyRPH
Copy link
Author

AndyRPH commented Jun 7, 2021

Yup started there first. Same result shown. Not sure if I need to adjust the make_devel.sh or CMakelists somehow to force it to use the piece I was missing and pulled in from home brew. Or force it to use gcc from home brew instead of apple's clang?

@madMAx43v3r
Copy link
Owner

Should be fixed in master.

@drnick23
Copy link

drnick23 commented Jun 7, 2021

Getting some compiling errors, is this info helpful:

-- Configured operating system: MACOSX
-- Configured GMP: /opt/homebrew/lib/libgmp.a
-- Compiler flags:  -Wall   -O3 -funroll-loops -fomit-frame-pointer -pthread
-- Linker flags: -L/Applications/Chia-arm/maxplotter/chia-plotter/build/_deps/relic-src/src/low/gmp/  -static
-- Configured /Applications/Chia-arm/maxplotter/chia-plotter/build/_deps/relic-src/include/relic_conf.h.in
SODIUM_FOUND in src/CMakeLists.txt

Also trying to compile on M1 Mac.

How did you get SODIUM FOUND -- did you edit the make files? I've tried using:

brew install libsodium-dev

but doesn't seem to be found, so not sure how to add the correct library.

I'm getting:
/Users/nick/chia-plotter/lib/bls-signatures/src/privatekey.hpp:21:10: fatal error: 'gmp.h' file not found #include <gmp.h> ^~~~~~~

@fermosan
Copy link
Contributor

fermosan commented Jun 7, 2021

try:
brew install gmp and recompile

@drnick23
Copy link

drnick23 commented Jun 7, 2021

try:
brew install gmp and recompile

Tried installing/reinstalling/uninstall then install for gmp, cleaning and recompiling but still no dice.

I did manage to use

brew install libsodium

since they don't have a -dev version of that for brew, not sure if that's the missing piece.

@AndyRPH
Copy link
Author

AndyRPH commented Jun 7, 2021

I wasn't getting gmp errors before but now am. I did brew install gmp

/Applications/Chia-arm/maxplotter/chia-plotter/lib/bls-signatures/src/privatekey.hpp:21:10: fatal error: 'gmp.h' file not found
#include <gmp.h>
         ^~~~~~~
1 error generated.

However earlier I am seeing that CMake is finding gmp.


-- Build python bindings: false
-- Build tests: 1
-- Build benchmarks: 1
-- Found libsodium
-- Sodium include dir = /opt/homebrew/Cellar/libsodium/1.0.18_1/include
GMP_INCLUDES=/opt/homebrew/include
-- Found libgmp

@drnick23
Copy link

drnick23 commented Jun 7, 2021

I wasn't getting gmp errors before but now am. I did brew install gmp

Did you manage to get it compile on an M1 earlier, and then this error came up, or did you hit compile errors until you eventually got to this part?

Would be good to know if someone has got it compiled on MacOS and if there are any particular additional steps.

@AndyRPH
Copy link
Author

AndyRPH commented Jun 7, 2021

Haven't gotten it successfully yet, at least not properly. Well, ok, I manually copied gmp.h from homebrew's folders into where I'm compiling, and then got similar error not finding sodium (but that's installed via homebrew too. and copied it and it's folder to the right place in the build folder structure, I think.... And it compiled. But running it always gives me an error after a few minutes, and I can't be sure that it compiled right since I know the above copying wasn't the 'right' way to do it...

andy@SlatebookPro build % ./chia_plot <redacted> <redacted> /Volumes/Plotter/Chia/Temp1/ /Volumes/SecondPlot/Chia/ 2 7
Number of Threads: 2
Number of Sort Buckets: 2^7 (128)
Pool Public Key:   < redacted >
Farmer Public Key: < redacted>
Working Directory:   /Volumes/Plotter/Chia/Temp1/
Working Directory 2: /Volumes/SecondPlot/Chia/
Plot Name: plot-k32-2021-06-07-12-36-< redacted>
[P1] Table 1 took 209.274 sec
zsh: bus error  ./chia_plot   /Volumes/Plotter/Chia/Temp1/ /Volumes/SecondPlot/Chia/ 2 7
andy@SlatebookPro build %

@Hardcore-fs
Copy link

Hardcore-fs commented Jun 7, 2021

yep its a bug in the apple kernel I think
go into
/chia/DiskSort.hpp & /chia/DiskTable.h
and reduce the 65536 to 49152

@AndyRPH
Copy link
Author

AndyRPH commented Jun 7, 2021

Well how about that! It's rolling along nicely now. Haven't completed a full plot yet but got further so far than previously!

@Hardcore-fs
Copy link

Hardcore-fs commented Jun 7, 2021

only told you so i can find out the specs of the M1.....
just a tip.....
GET A temp monitor "istat menu" demo
i found the shitty little macpro books pull 99 deg....

you don't want to destroy the cpu, with apples shitty cooling

Untitled

@AndyRPH
Copy link
Author

AndyRPH commented Jun 7, 2021

Nope, same bus error a little later on

[P1] Table 1 took 58.2284 sec                                        
[P1] Table 2 took 1419.72 sec, found 4295056583 matches              
zsh: bus error  /Applications/Chia-arm/maxplotter/chia-plotter/build/chia_plot     4 7   

@Hardcore-fs
Copy link

Hardcore-fs commented Jun 7, 2021

ok reduce it down to 32768, looks like the m1 is different to intel
or take a look at the patches to get rid of this crap completely in the :

https://github.com/madMAx43v3r/chia-plotter/pulls

that was my job for this morning/
or dig around the source for any more

@AndyRPH
Copy link
Author

AndyRPH commented Jun 7, 2021

Does this write any more detailed logs anywhere to see what it's erroring out on?

Is it hitting an open file count limit? https://forum.unity.com/threads/too-many-open-files.1036252/

Trying the increase to maxfiles from above and running again.

@Hardcore-fs
Copy link

Hardcore-fs commented Jun 7, 2021

Running out of files..... I doubt it...
and as for memory.. i've take it all the way down to 1Mb free from 96gb and the osx is STILL stable!!!!

compile the debug version
run "dsymutil" againt the binary
launch lldb with the executable as in just stick "lldb" before it
load the "symbol" file generated by "dsy*" command;
"target symbols add {file generated goes here}"
"r" to run it
"l" to list source...
"gui" to get into the gui...

one i get this FS thing fixed.. ill take a look

@AndyRPH
Copy link
Author

AndyRPH commented Jun 8, 2021

only told you so i can find out the specs of the M1.....

just a tip.....

GET A temp monitor "istat menu" demo

i found the shitty little macpro books pull 99 deg....

you don't want to destroy the cpu, with apples shitty cooling

Untitled

So you have an M1, or not your screenshot and you are trying to tease out specs as I test lower values?

Have istat, don't pay much attention to the temps, since I've no concern it would "destroy the cpu" simply by taxing it heavily, plus, they generally stand behind their warranties well.

@AndyRPH
Copy link
Author

AndyRPH commented Jun 8, 2021

Worthwhile to keep lowering it or just chill until that pull request is merged?

[P1] Table 1 took 233.675 sec                                       
[P1] Table 2 took 2069.09 sec, found 4294955798 matches             
Process 12089 stopped                                               
* thread #18, stop reason = EXC_BAD_ACCESS (code=2, address=0x1706c3ff8)                                                                
    frame #0: 0x000000019594a624 libsystem_pthread.dylib`___chkstk_darwin + 64                                                          
libsystem_pthread.dylib`___chkstk_darwin:                           
->  0x19594a624 <+64>: ldur   x11, [x11, #-0x8]                     
    0x19594a628 <+68>: mov    x10, sp                               
    0x19594a62c <+72>: cmp    x9, #0x1, lsl #12         ; =0x1000   
    0x19594a630 <+76>: b.lo   0x19594a648               ; <+100>    
  thread #19, stop reason = EXC_BAD_ACCESS (code=2, address=0x17074fff8)                                                                
    frame #0: 0x000000019594a624 libsystem_pthread.dylib`___chkstk_darwin + 64                                                          
libsystem_pthread.dylib`___chkstk_darwin:                           
->  0x19594a624 <+64>: ldur   x11, [x11, #-0x8]                     
    0x19594a628 <+68>: mov    x10, sp                               
    0x19594a62c <+72>: cmp    x9, #0x1, lsl #12         ; =0x1000   
    0x19594a630 <+76>: b.lo   0x19594a648               ; <+100>    
(lldb) 

@madMAx43v3r
Copy link
Owner

it's merged now

@we-sell-bags
Copy link

yep.. i got the same.
so i'm just building the new release
it's taken hours to get it working on osx
bit shocked at your first run though, but i can guess why......
go into "spotlight" and remove the drive from indexing."add to privacy"...., osx tries to index the files as they build...

MacBook Pro (Retina, 15-inch, Mid 2015).. badly overheating so i think it could do a lot better.....
[P1] Table 1 took 153.776 sec
[P1] Table 2 took 2065.24 sec, found 4294975870 matches
[P1] Lost 8275 matches due to 32-bit overflow.
[P1] Table 3 took 2212.68 sec, found 4294865102 matches

still running but further than you got with the old copy.. so i think it will be fine.
also the disk space is NOT what is shown in the readme .. it's a lot more.

@madMAx43v3r
Copy link
Owner

madMAx43v3r commented Jun 8, 2021

it's a lot more

It shouldn't... how much more are we talking?
Are you using chia_plot binary? (not the test_phase_x binaries?)

@AndyRPH
Copy link
Author

AndyRPH commented Jun 8, 2021

it's merged now

No luck, missing a dependency?

Applications/Chia-arm/maxplotter/chia-plotter/include/chia/chia_filesystem.hpp:20:10: fatal error: 'filesystem.hpp' file not found
#include "filesystem.hpp"
         ^~~~~~~~~~~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/chia_plot.dir/src/chia_plot.cpp.o] Error 1
make[1]: *** [CMakeFiles/chia_plot.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

@madMAx43v3r
Copy link
Owner

Yes, no idea how it works in chiapos, have to figure it out later..

@AndyRPH
Copy link
Author

AndyRPH commented Jun 8, 2021

Do I just need to comment out the #include filesystem line?

Yes, no idea how it works in chiapos, have to figure it out later..

@Hardcore-fs
Copy link

Hardcore-fs commented Jun 8, 2021

yep i had exactly the same.
let me check what i did....
took me hours... that and a dependency after this...
lol... don't know what the hell i did ...

ahhhh....

I think i did this..

CMakeLists.txt
include_directories( lib include lib/bls-signatures/src/ include/ghc )

then in "/include" i added:
"ghc"
which i pulled from here:
https://github.com/gulrak/filesystem
"include/ghc"

then in "chia_filesystem.hpp"
i changed the include line to
#include "ghc/filesystem.hpp"

it might also need a "brew install" but leave that till later just incase.

but all this is caused by a couple of lines checking a directory validity..
which i stated in another thread, that there are easier ways without pulling libraries from all over the place.
just becasue a developer has a full set of libraries does not mean they have to use them....

@madMAx43v3r
Copy link
Owner

yeah I will remove that dependency again...

@AndyRPH
Copy link
Author

AndyRPH commented Jun 8, 2021

Hmm, that compiled, until it hit a new snag at the end

[100%] Linking CXX executable chia_plot
clang: warning: argument unused during compilation: '-fmax-errors=1' [-Wunused-command-line-argument]
ld: library not found for -lstdc++fs
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [chia_plot] Error 1
make[1]: *** [CMakeFiles/chia_plot.dir/all] Error 2
make: *** [all] Error 2

@madMAx43v3r
Copy link
Owner

yeah just remove that also from CMakeLists.txt

@AndyRPH
Copy link
Author

AndyRPH commented Jun 8, 2021

Cool, plotting away now.

@AndyRPH
Copy link
Author

AndyRPH commented Jun 8, 2021

Interesting. 2 hours later, still plotting. It seems slower than normal. Running Pechy's opimized/combined branch, with just 2 threads and default RAM setting, I was able to plot a K32 on my apple M1 in just over 3 hours to an SSD in a TB3 enclosure.

Here I have tmp1 as an ssd in a USB3.1 enclosure (so 5 Gbps), and tmp2 for the ssd in the TB3 enclosure (40 Gbps). (both NVME 1tb SSDs starting empty).

Number of Threads: 4
Number of Sort Buckets: 2^7 (128)
Working Directory:   /Volumes/SecondPlot/Chia/
Working Directory 2: /Volumes/Plotter/Chia/
[P1] Table 1 took 144.793 sec
[P1] Table 2 took 1442.49 sec, found 4294861799 matches
[P1] Table 3 took 1444.04 sec, found 4294696360 matches
[P1] Table 4 took 1521.13 sec, found 4294443899 matches
[P1] Table 5 took 1509.81 sec, found 4294011018 matches
[P1] Table 6 took 1481.61 sec, found 4293069472 matches

Should it be faster?

@AndyRPH
Copy link
Author

AndyRPH commented Jun 8, 2021

End result, failed: libc++abi: terminating with uncaught exception of type std::runtime_error: thread failed with: fread() failed

P1] Table 1 took 144.793 sec
[P1] Table 2 took 1442.49 sec, found 4294861799 matches
[P1] Table 3 took 1444.04 sec, found 4294696360 matches
[P1] Table 4 took 1521.13 sec, found 4294443899 matches
[P1] Table 5 took 1509.81 sec, found 4294011018 matches
[P1] Table 6 took 1481.61 sec, found 4293069472 matches
[P1] Table 7 took 1408.48 sec, found 4291117334 matches
Phase 1 took 8952.89 sec
[P2] max_table_size = 4294967296
[P2] Table 7 scan took 16.841 sec
[P2] Table 7 rewrite took 89.7504 sec, dropped 0 entries (0 %)
[P2] Table 6 scan took 44.013 sec
libc++abi: terminating with uncaught exception of type std::runtime_error: thread failed with: fread() failed

@madMAx43v3r
Copy link
Owner

Is that with latest master? Ie. with this fix 7e768ca ?

@madMAx43v3r
Copy link
Owner

Otherwise I would check kernel logs for USB errors..

@AndyRPH
Copy link
Author

AndyRPH commented Jun 8, 2021

No, let me recompile and start another.

@Hardcore-fs
Copy link

Hardcore-fs commented Jun 8, 2021

ok something else is going on...
i got a couple of good runs from it over night. on osx 10.13.6

when I say good runs, they completed & validated.
and the disk space is a little over, but meets the expectation.

what would be good is if it could utilise more RAM, without ram disks.

@AndyRPH
Copy link
Author

AndyRPH commented Jun 8, 2021

It may be something unique to OS 11, or the fact that I'm using an M1 16GB MBpro. Either way, recompiled and running another plot to see how this one pans out.

@AndyRPH
Copy link
Author

AndyRPH commented Jun 9, 2021

Same issue again. Interestingly, OS X reported that my ssd was disconnected unsafely. It's never done this after plotting 100+ plots on it using the normal chia client... OS X did log a nice crash report for these, although I've no idea how to read it.

[P1] Table 1 took 114.4 sec
[P1] Table 2 took 1405.51 sec, found 4294914671 matches
[P1] Table 3 took 1437.47 sec, found 4294848187 matches
[P1] Table 4 took 1510.19 sec, found 4294797192 matches
[P1] Table 5 took 1517.14 sec, found 4294614037 matches
[P1] Table 6 took 1462.26 sec, found 4294283127 matches
[P1] Table 7 took 1411.14 sec, found 4293587244 matches
Phase 1 took 8858.64 sec
[P2] max_table_size = 4294967296
[P2] Table 7 scan took 18.9194 sec
[P2] Table 7 rewrite took 125.305 sec, dropped 0 entries (0 %)
[P2] Table 6 scan took 44.1335 sec
libc++abi: terminating with uncaught exception of type std::runtime_error: thread failed with: fread() failed
zsh: abort      ./chia_plot   /Volumes/SecondPlot/Chia/ /Volumes/Plotter/Chia/ 4 7

Crash report:

Process:               chia_plot [10393]
Path:                  /Applications/Chia-arm/*/chia_plot
Identifier:            chia_plot
Version:               0
Code Type:             ARM-64 (Native)
Parent Process:        zsh [661]
Responsible:           Terminal [621]
User ID:               501

Date/Time:             2021-06-08 19:03:44.354 -0400
OS Version:            macOS 11.4 (20F71)
Report Version:        12
Anonymous UUID:        5A178B84-540C-061C-9B61-6E160D768795


Time Awake Since Boot: 35000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
terminating with uncaught exception of type std::runtime_error: thread failed with: fread() failed
abort() called

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x0000000183460e68 __pthread_kill + 8
1   libsystem_pthread.dylib       	0x000000018349343c pthread_kill + 292
2   libsystem_c.dylib             	0x00000001833db460 abort + 104
3   libc++abi.dylib               	0x0000000183451b44 abort_message + 132
4   libc++abi.dylib               	0x0000000183442e3c demangling_terminate_handler() + 284
5   libobjc.A.dylib               	0x000000018333bca4 _objc_terminate() + 160
6   libc++abi.dylib               	0x0000000183450f2c std::__terminate(void (*)()) + 20
7   libc++abi.dylib               	0x0000000183453c0c __cxxabiv1::failed_throw(__cxxabiv1::__cxa_exception*) + 36
8   libc++abi.dylib               	0x0000000183453bb8 __cxa_throw + 140
9   chia_plot                     	0x0000000100172284 Thread<std::__1::pair<unsigned long, unsigned long> >::wait() + 220 (Thread.h:81)
10  chia_plot                     	0x0000000100171610 ThreadPool<std::__1::pair<unsigned long, unsigned long>, std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long>, DiskTable<phase1::tmp_entry_x>::local_t>::take(std::__1::pair<unsigned long, unsigned long>&) + 68 (ThreadPool.h:57)
11  chia_plot                     	0x00000001001759b8 Processor<std::__1::pair<unsigned long, unsigned long> >::take_copy(std::__1::pair<unsigned long, unsigned long> const&) + 24 (Thread.h:32) [inlined]
12  chia_plot                     	0x00000001001759b8 DiskTable<phase1::tmp_entry_x>::read(Processor<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, int, unsigned long) const + 436 (DiskTable.h:84)
13  chia_plot                     	0x000000010016e55c void phase2::compute_table<phase1::tmp_entry_x, phase2::entry_x, DiskSort<phase2::entry_x, phase2::get_pos<phase2::entry_x> > >(int, int, DiskSort<phase2::entry_x, phase2::get_pos<phase2::entry_x> >*, DiskTable<phase2::entry_x>*, table_t const&, bitfield*, bitfield const*) + 1352 (phase2.hpp:106)
14  chia_plot                     	0x00000001001265a8 phase2::compute(phase1::output_t const&, phase2::output_t&, int, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) + 1492 (phase2.hpp:159)
15  chia_plot                     	0x0000000100122b04 create_plot(int, int, std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > const&, std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 2644 (chia_plot.cpp:111)
16  chia_plot                     	0x00000001001219b4 main + 3716 (chia_plot.cpp:184)
17  libdyld.dylib                 	0x00000001834b1450 start + 4

Thread 1:
0   libsystem_kernel.dylib        	0x000000018345c548 __psynch_cvwait + 8
1   libsystem_pthread.dylib       	0x0000000183493dac _pthread_cond_wait + 1248
2   libc++.1.dylib                	0x00000001833edefc std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
3   chia_plot                     	0x00000001001789a4 Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::loop(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 220 (Thread.h:114)
4   chia_plot                     	0x0000000100178e94 decltype(*(std::__1::forward<Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*>(fp0)).*fp(std::__1::forward<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(fp1))) std::__1::__invoke<void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, void>(void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*&&)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) + 8 (type_traits:3688) [inlined]
5   chia_plot                     	0x0000000100178e94 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, 2ul, 3ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >&, std::__1::__tuple_indices<2ul, 3ul>) + 36 (thread:280) [inlined]
6   chia_plot                     	0x0000000100178e94 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >(void*) + 72 (thread:291)
7   libsystem_pthread.dylib       	0x0000000183493878 _pthread_start + 320
8   libsystem_pthread.dylib       	0x000000018348e5e0 thread_start + 8

Thread 2:
0   libsystem_kernel.dylib        	0x000000018345c548 __psynch_cvwait + 8
1   libsystem_pthread.dylib       	0x0000000183493dac _pthread_cond_wait + 1248
2   libc++.1.dylib                	0x00000001833edefc std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
3   chia_plot                     	0x00000001001789a4 Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::loop(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 220 (Thread.h:114)
4   chia_plot                     	0x0000000100178e94 decltype(*(std::__1::forward<Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*>(fp0)).*fp(std::__1::forward<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(fp1))) std::__1::__invoke<void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, void>(void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*&&)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) + 8 (type_traits:3688) [inlined]
5   chia_plot                     	0x0000000100178e94 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, 2ul, 3ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >&, std::__1::__tuple_indices<2ul, 3ul>) + 36 (thread:280) [inlined]
6   chia_plot                     	0x0000000100178e94 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >(void*) + 72 (thread:291)
7   libsystem_pthread.dylib       	0x0000000183493878 _pthread_start + 320
8   libsystem_pthread.dylib       	0x000000018348e5e0 thread_start + 8

Thread 3:
0   libsystem_kernel.dylib        	0x000000018345c548 __psynch_cvwait + 8
1   libsystem_pthread.dylib       	0x0000000183493dac _pthread_cond_wait + 1248
2   libc++.1.dylib                	0x00000001833edefc std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
3   chia_plot                     	0x00000001001789a4 Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::loop(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 220 (Thread.h:114)
4   chia_plot                     	0x0000000100178e94 decltype(*(std::__1::forward<Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*>(fp0)).*fp(std::__1::forward<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(fp1))) std::__1::__invoke<void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, void>(void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*&&)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) + 8 (type_traits:3688) [inlined]
5   chia_plot                     	0x0000000100178e94 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, 2ul, 3ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >&, std::__1::__tuple_indices<2ul, 3ul>) + 36 (thread:280) [inlined]
6   chia_plot                     	0x0000000100178e94 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >(void*) + 72 (thread:291)
7   libsystem_pthread.dylib       	0x0000000183493878 _pthread_start + 320
8   libsystem_pthread.dylib       	0x000000018348e5e0 thread_start + 8

Thread 4:
0   libsystem_kernel.dylib        	0x000000018345c548 __psynch_cvwait + 8
1   libsystem_pthread.dylib       	0x0000000183493dac _pthread_cond_wait + 1248
2   libc++.1.dylib                	0x00000001833edefc std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
3   chia_plot                     	0x00000001001789a4 Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::loop(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 220 (Thread.h:114)
4   chia_plot                     	0x0000000100178e94 decltype(*(std::__1::forward<Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*>(fp0)).*fp(std::__1::forward<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(fp1))) std::__1::__invoke<void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, void>(void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*&&)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) + 8 (type_traits:3688) [inlined]
5   chia_plot                     	0x0000000100178e94 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, 2ul, 3ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >&, std::__1::__tuple_indices<2ul, 3ul>) + 36 (thread:280) [inlined]
6   chia_plot                     	0x0000000100178e94 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::vector<phase2::entry_x, std::__1::allocator<phase2::entry_x> > >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >(void*) + 72 (thread:291)
7   libsystem_pthread.dylib       	0x0000000183493878 _pthread_start + 320
8   libsystem_pthread.dylib       	0x000000018348e5e0 thread_start + 8

Thread 5:
0   libsystem_kernel.dylib        	0x000000018345c548 __psynch_cvwait + 8
1   libsystem_pthread.dylib       	0x0000000183493dac _pthread_cond_wait + 1248
2   libc++.1.dylib                	0x00000001833edefc std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
3   chia_plot                     	0x0000000100176d28 Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::loop(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 220 (Thread.h:114)
4   chia_plot                     	0x00000001001771f8 decltype(*(std::__1::forward<Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*>(fp0)).*fp(std::__1::forward<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(fp1))) std::__1::__invoke<void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, void>(void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*&&)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) + 8 (type_traits:3688) [inlined]
5   chia_plot                     	0x00000001001771f8 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, 2ul, 3ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >&, std::__1::__tuple_indices<2ul, 3ul>) + 36 (thread:280) [inlined]
6   chia_plot                     	0x00000001001771f8 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >(void*) + 72 (thread:291)
7   libsystem_pthread.dylib       	0x0000000183493878 _pthread_start + 320
8   libsystem_pthread.dylib       	0x000000018348e5e0 thread_start + 8

Thread 6:
0   libsystem_kernel.dylib        	0x000000018345c548 __psynch_cvwait + 8
1   libsystem_pthread.dylib       	0x0000000183493dac _pthread_cond_wait + 1248
2   libc++.1.dylib                	0x00000001833edefc std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
3   chia_plot                     	0x0000000100176d28 Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::loop(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 220 (Thread.h:114)
4   chia_plot                     	0x00000001001771f8 decltype(*(std::__1::forward<Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*>(fp0)).*fp(std::__1::forward<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(fp1))) std::__1::__invoke<void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, void>(void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*&&)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) + 8 (type_traits:3688) [inlined]
5   chia_plot                     	0x00000001001771f8 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, 2ul, 3ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >&, std::__1::__tuple_indices<2ul, 3ul>) + 36 (thread:280) [inlined]
6   chia_plot                     	0x00000001001771f8 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >(void*) + 72 (thread:291)
7   libsystem_pthread.dylib       	0x0000000183493878 _pthread_start + 320
8   libsystem_pthread.dylib       	0x000000018348e5e0 thread_start + 8

Thread 7:
0   libsystem_kernel.dylib        	0x000000018345c548 __psynch_cvwait + 8
1   libsystem_pthread.dylib       	0x0000000183493dac _pthread_cond_wait + 1248
2   libc++.1.dylib                	0x00000001833edefc std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
3   chia_plot                     	0x0000000100176d28 Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::loop(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 220 (Thread.h:114)
4   chia_plot                     	0x00000001001771f8 decltype(*(std::__1::forward<Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*>(fp0)).*fp(std::__1::forward<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(fp1))) std::__1::__invoke<void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, void>(void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*&&)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) + 8 (type_traits:3688) [inlined]
5   chia_plot                     	0x00000001001771f8 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, 2ul, 3ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >&, std::__1::__tuple_indices<2ul, 3ul>) + 36 (thread:280) [inlined]
6   chia_plot                     	0x00000001001771f8 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >(void*) + 72 (thread:291)
7   libsystem_pthread.dylib       	0x0000000183493878 _pthread_start + 320
8   libsystem_pthread.dylib       	0x000000018348e5e0 thread_start + 8

Thread 8:
0   libsystem_kernel.dylib        	0x000000018345c548 __psynch_cvwait + 8
1   libsystem_pthread.dylib       	0x0000000183493dac _pthread_cond_wait + 1248
2   libc++.1.dylib                	0x00000001833edefc std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 28
3   chia_plot                     	0x0000000100176d28 Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::loop(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 220 (Thread.h:114)
4   chia_plot                     	0x00000001001771f8 decltype(*(std::__1::forward<Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*>(fp0)).*fp(std::__1::forward<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(fp1))) std::__1::__invoke<void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, void>(void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*&&)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) + 8 (type_traits:3688) [inlined]
5   chia_plot                     	0x00000001001771f8 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, 2ul, 3ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >&, std::__1::__tuple_indices<2ul, 3ul>) + 36 (thread:280) [inlined]
6   chia_plot                     	0x00000001001771f8 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) noexcept, Thread<std::__1::pair<std::__1::vector<phase1::tmp_entry_x, std::__1::allocator<phase1::tmp_entry_x> >, unsigned long> >*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >(void*) + 72 (thread:291)
7   libsystem_pthread.dylib       	0x0000000183493878 _pthread_start + 320
8   libsystem_pthread.dylib       	0x000000018348e5e0 thread_start + 8

Thread 0 crashed with ARM Thread State (64-bit):
    x0: 0x0000000000000000   x1: 0x0000000000000000   x2: 0x0000000000000000   x3: 0x0000000000000000
    x4: 0x000000016fce19d0   x5: 0x000000016fce1f80   x6: 0x0000000000000064   x7: 0x0000000100d05500
    x8: 0xd364487e61cc6f8c   x9: 0xd364487f610d12cc  x10: 0x0000000000000002  x11: 0x0000000000000004
   x12: 0x0000000000000000  x13: 0x0000000000000008  x14: 0x0000000000000010  x15: 0x000000000000051b
   x16: 0x0000000000000148  x17: 0x00000001f1deae90  x18: 0x0000000000000000  x19: 0x0000000000000006
   x20: 0x0000000100c17d40  x21: 0x0000000000000103  x22: 0x0000000100c17e20  x23: 0x0000000121f09110
   x24: 0x000000000000fff5  x25: 0x0000000000000002  x26: 0x0000000100d051e8  x27: 0x000000016fce2698
   x28: 0x000000000000000a   fp: 0x000000016fce1ef0   lr: 0x000000018349343c
    sp: 0x000000016fce1ed0   pc: 0x0000000183460e68 cpsr: 0x40000000
   far: 0x00000001e23af9c8  esr: 0x56000080


Binary Images:
       0x10011c000 -        0x10027bfff +chia_plot (0) <6697B881-D164-3EFF-9001-117DB4A7DEDD> /Applications/Chia-arm/*/chia_plot
       0x100b5c000 -        0x100bdbfff  dyld (852) <CF624584-51D6-329D-91C5-6BEDFADFB2D2> /usr/lib/dyld
       0x1831ee000 -        0x1831effff  libsystem_blocks.dylib (79) <D1EEB8DD-4C98-3860-951D-8EC75B5C887E> /usr/lib/system/libsystem_blocks.dylib
       0x1831f0000 -        0x183227fff  libxpc.dylib (2038.120.1) <EFD268F9-B27D-36AA-86A6-8774041CDAD1> /usr/lib/system/libxpc.dylib
       0x183228000 -        0x18323ffff  libsystem_trace.dylib (1277.120.1) <43C2510E-8316-3BEF-8AA5-7FD377B8E517> /usr/lib/system/libsystem_trace.dylib
       0x183240000 -        0x1832b3fff  libcorecrypto.dylib (1000.120.2) <B3A58FED-01F5-3DEB-AB84-56A360C9E37D> /usr/lib/system/libcorecrypto.dylib
       0x1832b4000 -        0x1832dffff  libsystem_malloc.dylib (317.121.1) <009D3C3B-E487-36BE-83FC-6EAB61DAA5EC> /usr/lib/system/libsystem_malloc.dylib
       0x1832e0000 -        0x183324fff  libdispatch.dylib (1271.120.2) <F6550274-5861-3B66-BE4B-C0544C4CB795> /usr/lib/system/libdispatch.dylib
       0x183325000 -        0x18335efff  libobjc.A.dylib (824) <60D3BBDD-A4B2-353B-8BCD-57F133BB66FC> /usr/lib/libobjc.A.dylib
       0x18335f000 -        0x183361fff  libsystem_featureflags.dylib (28.60.1) <1901A100-8004-3B64-8B22-1A77D944C60E> /usr/lib/system/libsystem_featureflags.dylib
       0x183362000 -        0x1833e3fff  libsystem_c.dylib (1439.100.3) <EF81BFF2-9151-3386-9508-1BAAE249D84A> /usr/lib/system/libsystem_c.dylib
       0x1833e4000 -        0x18343ffff  libc++.1.dylib (905.6) <3937F450-366C-3790-99CB-5F8AF80C33B0> /usr/lib/libc++.1.dylib
       0x183440000 -        0x183457fff  libc++abi.dylib (905.6) <E17CBF1D-9020-31D0-820D-9AD86150966D> /usr/lib/libc++abi.dylib
       0x183458000 -        0x18348bfff  libsystem_kernel.dylib (7195.121.3) <FCC78075-2A82-34F5-AC08-B555E39DB1DE> /usr/lib/system/libsystem_kernel.dylib
       0x18348c000 -        0x183498fff  libsystem_pthread.dylib (454.120.2) <E2FAA164-BEA2-37A2-935B-BBA138DDC9B6> /usr/lib/system/libsystem_pthread.dylib
       0x183499000 -        0x1834dafff  libdyld.dylib (852) <F775B9FF-1995-3613-A55A-3046E628AD8A> /usr/lib/system/libdyld.dylib
       0x1834db000 -        0x1834e1fff  libsystem_platform.dylib (254.80.2) <9E692E15-B0B4-313D-A7A1-C713845AE9BF> /usr/lib/system/libsystem_platform.dylib
       0x1834e2000 -        0x18350dfff  libsystem_info.dylib (542.40.3) <9A2DB28F-34B6-3833-AFBB-425138321913> /usr/lib/system/libsystem_info.dylib
       0x1858a1000 -        0x1858abfff  libsystem_darwin.dylib (1439.100.3) <193EC794-972E-3A94-857E-78621A7845F7> /usr/lib/system/libsystem_darwin.dylib
       0x185ce4000 -        0x185ceffff  libsystem_notify.dylib (279.40.4) <40EB921B-26F6-381F-A09D-2C81AC4FDD2D> /usr/lib/system/libsystem_notify.dylib
       0x187cc1000 -        0x187cd0fff  libsystem_networkextension.dylib (1295.120.5) <87BBED5B-50E1-3EB9-AE9F-DC634054217C> /usr/lib/system/libsystem_networkextension.dylib
       0x187d26000 -        0x187d3dfff  libsystem_asl.dylib (385) <FDF27EA4-D035-30F4-9C8E-5F276533170F> /usr/lib/system/libsystem_asl.dylib
       0x189293000 -        0x18929afff  libsystem_symptoms.dylib (1431.120.1) <BC3928DE-7A97-3190-B06E-080A30D32463> /usr/lib/system/libsystem_symptoms.dylib
       0x18b0ed000 -        0x18b0fefff  libsystem_containermanager.dylib (318.100.4) <1FFA0A3E-EF38-314A-BE28-5348FF274778> /usr/lib/system/libsystem_containermanager.dylib
       0x18bd6a000 -        0x18bd6efff  libsystem_configuration.dylib (1109.120.1) <ECEA3DAA-ADFC-3D0D-8951-279CDDB553EF> /usr/lib/system/libsystem_configuration.dylib
       0x18bd6f000 -        0x18bd73fff  libsystem_sandbox.dylib (1441.120.5) <0D01ECDE-9F24-30DE-9F37-6A980752BE0B> /usr/lib/system/libsystem_sandbox.dylib
       0x18c730000 -        0x18c732fff  libquarantine.dylib (119.40.2) <D48BB719-6CD2-3F08-9064-0992A490E4F2> /usr/lib/system/libquarantine.dylib
       0x18cc93000 -        0x18cc97fff  libsystem_coreservices.dylib (127.1) <E281EB73-54F4-30A8-8E56-9208D17CEAEA> /usr/lib/system/libsystem_coreservices.dylib
       0x18ce8f000 -        0x18cebcfff  libsystem_m.dylib (3186.100.3) <62AE5ECE-F59B-36CC-BE4F-935C4AAA5A50> /usr/lib/system/libsystem_m.dylib
       0x18cebe000 -        0x18cec3fff  libmacho.dylib (980) <A93DD379-6612-3200-9B02-84803274B6B9> /usr/lib/system/libmacho.dylib
       0x18cedc000 -        0x18cee9fff  libcommonCrypto.dylib (60178.120.3) <4F46EA59-019E-34AA-8EFA-44534DC4321F> /usr/lib/system/libcommonCrypto.dylib
       0x18ceea000 -        0x18cef4fff  libunwind.dylib (201) <241E3D3F-802A-39EA-8F5F-2F03137BE8D3> /usr/lib/system/libunwind.dylib
       0x18cef5000 -        0x18cefcfff  liboah.dylib (203.46) <F1CBF4AA-033C-3ABA-A13E-696ADC7A9E6A> /usr/lib/liboah.dylib
       0x18cefd000 -        0x18cf05fff  libcopyfile.dylib (173.40.2) <54BE801A-ACD8-32B4-9462-215F6E28BE34> /usr/lib/system/libcopyfile.dylib
       0x18cf06000 -        0x18cf09fff  libcompiler_rt.dylib (102.2) <B71B5432-79C8-3D81-97C0-B48528BCFB28> /usr/lib/system/libcompiler_rt.dylib
       0x18cf0a000 -        0x18cf0cfff  libsystem_collections.dylib (1439.100.3) <BBCEB3ED-935C-322D-8E19-B9E1DFD28A72> /usr/lib/system/libsystem_collections.dylib
       0x18cf0d000 -        0x18cf0ffff  libsystem_secinit.dylib (87.60.1) <87E4EC68-41CF-31FF-BE03-1F281BEB2CDE> /usr/lib/system/libsystem_secinit.dylib
       0x18cf10000 -        0x18cf12fff  libremovefile.dylib (49.120.1) <890BF916-C32E-37CA-8469-155E62F7E8EC> /usr/lib/system/libremovefile.dylib
       0x18cf13000 -        0x18cf13fff  libkeymgr.dylib (31) <F477F112-549F-3C72-B6BD-0281BC046E54> /usr/lib/system/libkeymgr.dylib
       0x18cf14000 -        0x18cf1cfff  libsystem_dnssd.dylib (1310.120.71) <89358F53-4E2F-39C9-A55C-FFA55B27BA1A> /usr/lib/system/libsystem_dnssd.dylib
       0x18cf1d000 -        0x18cf22fff  libcache.dylib (83) <776133A5-A09C-310F-B0FE-4CA311433D94> /usr/lib/system/libcache.dylib
       0x18cf23000 -        0x18cf24fff  libSystem.B.dylib (1292.120.1) <1ADA647F-F493-3C49-AD69-C24F0D76F491> /usr/lib/libSystem.B.dylib
       0x190344000 -        0x190344fff  liblaunch.dylib (2038.120.1) <477FFDB4-6279-3B73-ADF7-933CD1E9491E> /usr/lib/system/liblaunch.dylib
       0x1927f2000 -        0x1927f2fff  libsystem_product_info_filter.dylib (8.40.1) <188B6361-3ABE-3320-AFF7-C4C6A0F316EC> /usr/lib/system/libsystem_product_info_filter.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=497.8M resident=0K(0%) swapped_out_or_unallocated=497.8M(100%)
Writable regions: Total=2.6G written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=2.6G(100%)
 
                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Kernel Alloc Once                   32K        1 
MALLOC                             2.3G      164 
MALLOC guard page                   96K        5 
MALLOC_MEDIUM (reserved)         280.0M        6         reserved VM address space (unallocated)
STACK GUARD                       56.1M        9 
Stack                             12.3M       11 
VM_ALLOCATE                        128K        8 
__AUTH                              45K       11 
__AUTH_CONST                        74K       38 
__DATA                            5186K       38 
__DATA_CONST                       226K       38 
__DATA_DIRTY                        73K       21 
__LINKEDIT                       491.9M        4 
__OBJC_CONST                        10K        5 
__OBJC_RO                         75.0M        1 
__OBJC_RW                         2576K        1 
__TEXT                            6056K       44 
shared memory                       32K        2 
===========                     =======  ======= 
TOTAL                              3.2G      407 
TOTAL, minus reserved VM space     3.0G      407 


@we-sell-bags
Copy link

we-sell-bags commented Jun 9, 2021

This is in the OSX kernel
for the SSD to disconnect, would be a communication issue with the SSD & mother board
which is unlikely.

what are your temps?
it might be the SSD is entering protection mode & shutting down.

try changing teh read block sizes to something stupid like 8192

chia-plotter/src/settings.cpp

Also a "stage restart" might be useful for interrupting & restarting plotting jobs....
that plotman does NOT do it. if you stop the chia daemons.

@AndyRPH
Copy link
Author

AndyRPH commented Jun 9, 2021

Yeah it didn't physically disconnect something software hiccuped. I can run 2 plots staggered 24/7 on this computer with the official client with no issues. Not sure what you mean with a stage restart since there is no plotman nor chia daemons .

I swapped the tmp 1 and tmp2 drive paths to rule out the hardware being an issue. If it quits again maybe I'll recompile with smaller read block size as you suggest. Thanks,.

@we-sell-bags
Copy link

The official client does not hammer the hardware the same way.

what i mean by restart , is that once it aborts there is no way to restart the process... which makes debugging expensive on time.

@madMAx43v3r
Copy link
Owner

You can use the test_phase_x binaries for debugging, they won't delete the files from previous phases. But also they wont create a farmable plot.

@AndyRPH
Copy link
Author

AndyRPH commented Jun 9, 2021

What's interesting is that reversing the tmps directories (both external NVME ssds 1TB, one via usb 3.1, one via TB3) resulted in a successful plot, albeit a bit slower than the standard client.

Is there any logic to why it would crash each time if I made the working directory 1 the USB 3.1 drive, and working directory 2 the TB3 drive? I can plot with standard client fine to both simultaneously without issue.

How should I run the test_phase_x tool to get more data about why it's crashing? Do they take the same parameters just output more data?


Number of Threads: 4
Number of Sort Buckets: 2^7 (128)
Pool Public Key:   <redacted>
Farmer Public Key: <redacted>
Working Directory:   /Volumes/Plotter/Chia/
Working Directory 2: /Volumes/SecondPlot/Chia/
Plot Name: plot-k32-2021-06-08-22-14-<redacted>
[P1] Table 1 took 125.086 sec
[P1] Table 2 took 1467.53 sec, found 4294943252 matches
[P1] Table 3 took 1680.41 sec, found 4294914342 matches
[P1] Table 4 took 1765.43 sec, found 4294758355 matches
[P1] Table 5 took 1601.37 sec, found 4294603451 matches
[P1] Table 6 took 1559.58 sec, found 4294212555 matches
[P1] Table 7 took 1428.26 sec, found 4293393496 matches
Phase 1 took 9627.77 sec
[P2] max_table_size = 4294967296
[P2] Table 7 scan took 113.435 sec
[P2] Table 7 rewrite took 310.08 sec, dropped 0 entries (0 %)
[P2] Table 6 scan took 42.7952 sec
[P2] Table 6 rewrite took 261.817 sec, dropped 581369789 entries (13.5384 %)
[P2] Table 5 scan took 41.872 sec
[P2] Table 5 rewrite took 226.231 sec, dropped 762050985 entries (17.7444 %)
[P2] Table 4 scan took 41.6719 sec
[P2] Table 4 rewrite took 210.29 sec, dropped 828892723 entries (19.3001 %)
[P2] Table 3 scan took 41.3834 sec
[P2] Table 3 rewrite took 206.67 sec, dropped 855118732 entries (19.91 %)
[P2] Table 2 scan took 41.0788 sec
[P2] Table 2 rewrite took 183.197 sec, dropped 865582863 entries (20.1535 %)
Phase 2 took 1721.44 sec
Wrote plot header with 268 bytes
[P3-1] Table 2 took 182.407 sec, wrote 3429360389 right entries
[P3-2] Table 2 took 256.248 sec, wrote 3429360389 left entries, 3429360389 final
[P3-1] Table 3 took 253.215 sec, wrote 3439795610 right entries
[P3-2] Table 3 took 257.147 sec, wrote 3439795610 left entries, 3439795610 final
[P3-1] Table 4 took 254.71 sec, wrote 3465865632 right entries
[P3-2] Table 4 took 259.236 sec, wrote 3465865632 left entries, 3465865632 final
[P3-1] Table 5 took 257.516 sec, wrote 3532552466 right entries
[P3-2] Table 5 took 269.292 sec, wrote 3532552466 left entries, 3532552466 final
[P3-1] Table 6 took 269.353 sec, wrote 3712842766 right entries
[P3-2] Table 6 took 282.651 sec, wrote 3712842766 left entries, 3712842766 final
[P3-1] Table 7 took 443.255 sec, wrote 4293393496 right entries
[P3-2] Table 7 took 329.206 sec, wrote 4293393496 left entries, 4293393496 final
Phase 3 took 3315.14 sec, wrote 21873810359 entries to final plot
[P4] Starting to write C1 and C3 tables
[P4] Finished writing C1 and C3 tables
[P4] Writing C2 table
[P4] Finished writing C2 table
Phase 4 took 272.309 sec, final plot size is 108814979456 bytes
Total plot creation time was 14936.7 sec

@madMAx43v3r
Copy link
Owner

The access pattern is different for tmpdir and tmpdir2, tmpdir is usually only written to or read from at a time (except phase 3 stage 2), whereas tmpdir2 usually has to do both at the same time.

@AndyRPH
Copy link
Author

AndyRPH commented Jun 9, 2021

Alright, I'm reliably getting this to work. Performance is surprising. It seems at 4 threads, no faster than the pechy/merged branch of the plotter. I'm running another now set to 5 threads.

I can close this out, or keep it around to track discoveries [no, let's use discussions for that]? I'm curious since the Apple M1 cpu has 4 performance cores, and 4 'efficiency' cores, I don't really know what it's doing when I tell it 5 or 6 threads. I know at 5 it's pegging all 4 performance cores at 90-100%, and all 4 efficiency cores have use around 40-75%. (still in phase 1) RAM doesn't seem to be bad (I've got 16gb) I glance at it now and then and the chia_plot process doesn't seem to consume more than 2.5-ish GB, although often I see it claiming less than that (for as accurate as apple's activity monitor is).

@drnick23
Copy link

drnick23 commented Jun 9, 2021

Have you tried running it once on Apple's internal disk (if you have enough space, and don't mind writing a TB or two) just to see how well it can go? I'm surprised it's not doing better, I can only guess it's causing throttling on your external SSDs. I have yet to attempt another compile.

@AndyRPH
Copy link
Author

AndyRPH commented Jun 9, 2021

Well... I've no desire to tempt fate but yeah, I'll run one plot on my internal ssd just for kicks and see what times it gives me.

@AndyRPH
Copy link
Author

AndyRPH commented Jun 10, 2021

Ran it on the internal NVME SSD of my M1 MBP. (1 TB, 16GB ram model). 4 threads, nothing else running.
Phase 1: 8733.45 sec
Phase 2: 895.871 sec
Phase 3: 1043.87 sec
Phase 4: 179.088 sec
Total: 10852.3 sec

I've hit that regularly using my TB3 NVME SSD, so, using the internal storage offered no speed gain. It's probably limited by just 4 threads on the M1 cpu.

@AndyRPH AndyRPH closed this as completed Jun 10, 2021
@we-sell-bags
Copy link

well at least it is finally running......

AmineKhaldi pushed a commit to AmineKhaldi/chia-plotter that referenced this issue Oct 26, 2021
…onsorship

Removed sponsorship notice in the Chia fork (permission granted by madMAx43v3r)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants