Feature: Docker Image #102
Conversation
|
I did a few tests here, it seems that running in Docker is 5% slower. I think that is a pretty good trade-off for anyone that cannot compile for whatever reason (Windows machines, for example) |
|
|
|
compared to WSL |
|
can you add instructions to the readme? |
|
Sure! I'll do it later today and then comment back here in the PR 🙂 Regarding performance, the 5% impact is on a Linux machine. I don't have a proper Windows to test it, it would be nice to do the same test in Windows too! |
|
@madMAx43v3r do you have a Dockerhub account to host the stable image? I don't know if you know how to use Docker, but you can build that image and them host it on your Docker account with the following commands: docker login # It will login to your Docker account
docker build -t <your-username>/chia-plotter .
docker push <your-username>/chia-plotterThe must be exactly the same as the username you used in Or should I left the image generated under my own account in the README? |
|
Yeah let's use your account & image |
|
docker does not seem to use both cpus |
Did you added the docker run \
-v <path-to-your-tmp-dir>:/mnt/harvester \
-v <path-to-your-final-dir>:/mnt/farm \
odelucca/chia-plotter \
-t /mnt/harvester/ \
-d /mnt/farm/ \
-p <pool-key> \
-f <farm-key> \
-r <number-of-cores> |
|
If you added the docker run \
-v <path-to-your-tmp-dir>:/mnt/harvester \
-v <path-to-your-final-dir>:/mnt/farm \
-m 8000 \
odelucca/chia-plotter \
-t /mnt/harvester/ \
-d /mnt/farm/ \
-p <pool-key> \
-f <farm-key>Would limit your container to 5gb RAM (with the |
|
@madMAx43v3r I've added the README section. Could you merge please? :) |
|
@delucca ok - super nitpicky haha - but there is an r missing at the end of Docker on line 179 of the Readme |
I have done this and also manually added a global wslconfig to provide 32 vcpu and 64gb ram. Still processing under 1 cpu. |
|
any plans to add this to unraid's CA? |
|
Is this image up on dockerhub yet anywhere? I'm trying to build it locally to test it (on a windows host) from the HEAD of your branch, but keep running into the following error:
|
Yes, it is: |
Awesome. I'll give up fighting cmake being cranky, and just switch to testing what you've published. Thanks! |
|
Did some digging docker wont make full use of dual cpu setups, as wsl2 only passes 1 socket over. Also the wslconfig cant overide it. So a dedicated windows version is the way to go. |
Where you able to test it? I would like to know if WSL1 is close to Linux-like Docker performance |
You could give it a try to this guide: https://gist.github.com/bkonetzny/b769afcab375dedae743ac648622a4f1 It seems that WSL1 removes that I/O bottleneck |
|
I have some plots running which finish in 3hrs, will test after that coz gotta reboot system to install. |
|
PS C:\Users\Administrator> docker run -v N::/mnt/harvester -v E::/mnt/farm -m 14000m --cpus 12 odelucca/chia-plotter -t /mnt/harvester/ -d /mnt/farm/ -p xxxxxxxxx -f xxxxxxxxxxxx -r 12 -u 6 -n 1 **Number of Buckets: 2^6 (64) -u 6 ** |
Cool :) Let me know after switching to WSL1! |
|
Ok, finally got the docker spun up on a fresh ubuntu 20.04 install. It is working good except that like I saw mentioned earlier, it is only using 1 CPU (50% usage on a dual CPU system). Did anyone figure out how to get the docker to max all cores? Since this is a blank ubntu install I went ahead and installed the regular program and can confirm that it maxes all cores. On my main unraid server though it would need to run as a docker. |
Cancel this option？ |
Resources Enable integration with my default WSL distro You don't have any WSL 2 distro. Please convert a WSL 1 distro to WSL 2, or install a new distro and it will appear here. More info |
Switched to WSL1, it seems working as expected. |
This will switch over to hyper V which also has IO bottleneck. The only possible way to mitigated the IO performance loss is using WSL1, which is not supported by docker desktop out of the box. https://gist.github.com/bkonetzny/b769afcab375dedae743ac648622a4f1 |
|
|
@NikXO6 I install Ubuntu 20.4 distro on WSL1 then run mad max plotter directly on it. |
nice |
You were able to switch Docker to use WSL1 with the guide you shared? If so, could you please add a section in the README explaining how to use WSL1 backend in Docker? (if you followed a guide, simply points to that guide) That would save me hours of responding to people explaining this hahaha |
I switch to WSL1, then install Ubuntu 20.4 distro on it. |
My quote wasn't directly to you, it was asking to @NikXO6 I know that you've used WSL directly without Docker @hung-nguyen89, but I'm interested in Docker in Windows since some users prefer using it. With Docker, it is way easier to manage large plotting clusters. It is easy to trigger a single plot process in your own machine. For those cases, Docker is overkill. But for users that are managing many nodes (like me), Docker is way easier. In my specific case, the nodes that I manage are Linux machines, so I'm not suffering from it. But some users are asking for help to me, but they manage clusters with Windows machines. |
|
@delucca |
No problem man :D I was just explaining that f my question was for the other user, just to see if he was able to run with WSL1 |
|
@delucca docker run |
|
Your v arguments are off. With one argument it allocates the space from
Docker.
Do -v hostpath:containerpath instead. Even if you want to make the paths
the same.
…On Sat, Jun 12, 2021, 5:38 AM spronty1 ***@***.***> wrote:
@delucca <https://github.com/delucca>
I'm trying to run this in unraid terminal. It keeps filling up my docker
image and crashing. The program isnt using my 2 temp dir. What am i doing
wrong?
docker run
-v /mnt/cache/ChiaPlotting2/Mad2/
-v /mnt/plottingchia/ChiaPlotting/Mad/
-v /mnt/user/201-300/
odelucca/chia-plotter
-t /mnt/cache/ChiaPlotting2/Mad2/
-2 /mnt/plottingchia/ChiaPlotting/Mad/
-d /mnt/user/201-300/
-p
af102c5ea342449d8af28ff72850606510c85f35659682a7b6051d8e7fffdc966a7d50ff38c9b849331edf613445acb1
-f
a06d85db839a173a6e79fac92a095bb35d72cbeb54903ff8cb247fe539b845d0bfb1c0f99a52405fc1f2f0e600f039e6
-r 14
-n -1
-u 128 \
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#102 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABDAJ7YQUQHWEQZA2EKP2DTSMTIRANCNFSM46NEAUDA>
.
|
|
guys, in the end what is the best solution for plotting on windows? |
Fastest: using @stotiks compiled version It depends on your needs |
|
Stotik's binary, or the docker image on the old (not WSL2) backend.
…On Sat, Jun 12, 2021 at 5:52 PM Daniel De Lucca ***@***.***> wrote:
guys, in the end what is the best solution for plotting on windows?
@hung-nguyen89 <https://github.com/hung-nguyen89>
@delucca <https://github.com/delucca>
Fastest: using @stotiks <https://github.com/stotiks> compiled version
More portable: Docker
It depends on your needs
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#102 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABDAJ3S7SKY6HK7LCA5ISDTSPJKJANCNFSM46NEAUDA>
.
|
|
on my servers the stotiks version is incredibly slow, probably not fully compatible with windows server, now i'm using wsl2 but only uses one cpu on 2! |
|
@delucca @feder-cr Steps:
Note: wsl mount disk only available on windows preview build |
Amazing! Can you open a PR adding these instructions in the Docker section of the README file please? |
Just a small disclaimer - this works great, but the disk that you want to mount cannot be the disk that you have Windows installed on: the mount command only works for an entire logical disk, meaning you can't just mount a partition, meaning you can't mount the system disk because you need to detach the disk from Windows to mount it, meaning you will have to boot Windows off of a different drive, which could be a problem for some people. |
|
instead do you have advice on how to use both cpu? because I have two cpu but with wsl I only use 1 !! |
|
Tryed to use odelucca/chia-plotter to plot chives, it seems that |
This PR adds a Docker Image to improve portability. We can use that image for both Windows and Linux machines.
How it works
You can test a preview generated image with the following command on your machine:
Of course, you can provide whatever arguments you want after the image name.
Also, you can make an alias, like:
And call it with:
I've run a few tests, and it seems that it has a small performance cost. I was not able to define the exact percentage of the performance decrease yet, but still I think that is a pretty good alternative for running the plotter in Windows machines.
Future improvements
I see two important future improvements, they are: