Large number of high CPU "/usr/bin/farmr harvester headless" instances after migrating from 0.5.9 to 0.6.5 and adding more forks

[colbyboles]

I have a large number of linux harvester machines previously harvesting chia and flax with 0.5.9 with no issues. I switched to 0.6.5 and added containers for cactus, chives, cryptodoge, flora, hddcoin, nchain, staicon, and stor (I also did this on the full-node machine of course). The harvesting of all of these forks is actually working correctly, but the harvester machines are at nearly 100% CPU on all 4c/8t and looking via htop I can see about 20 instances of "/usr/bin/farmr harvester headless" running, all at high CPU which does not seem to go away over time.

I can kill off these instances, and harvesting seems to continue running fine on all forks. I would like to give farmr a try sometime if it works properly, but it doesn't seem to be in my case.

How can I disable farmr on these harvesters? Also, I had chosen Machinaris over farmr months ago because no operational data leaves my network. It would seem that it would be better default behavior to have this feature turned off and have users opt-in instead?

[guydavis]

Good day, please see the wiki with details on disabling farmr on launch. Then please mark this question as Answered. Thanks!

[guydavis]

What did you find in the ~/.machinaris/farmr/log.txt file?

Edit: You have 24 machines all running Machinaris? Wow! As a large farmer, I'd be interested to get your overall feedback on Machinaris, particularly what could be improved at your scale. Please consider dropping into the Discord if you haven't already.

[colbyboles]

Yes I have 24 identical harvesters, 4 plotter+harvesters (plotters not using machinaris due to custom plotter + transfer code), and one farmer. I have lots of thoughts - I'll check the discord out.

[colbyboles]

BTW, setting the environment variable
-e farmr_skip_launch=true
in my docker configuration(s) seems to do nothing to prevent the installation and startup of farmr. I took a look at your code, and I don't see anything in entrypoint.sh that checks for this env var. Your comments in entrypoint.sh seem to imply that you intended to put a conditional here maybe?

  # Conditionally install farmr on harvesters and fullnodes
  /usr/bin/bash /machinaris/scripts/farmr_setup.sh > /tmp/farmr_setup.log 2>&1

Farmr is like a cancer on any of my nodes - it goes to 100% CPU, then fills the user's disk allocation with it's logfile to 100% and takes the system down. You can shut it down, delete the files / logs, but machinaris keeps reinstalling and restarting it!

[colbyboles]

I looked further into farmr_setup.sh, and it looks like you install farmr regardless of the state of farmr_skip_launch which is fine I guess, but later you have a conditional intended to not execute farmr if farmr_skip_launch exists (regardless of true/false state). In my testing, this conditional

if [[ ! -z $"farmr_skip_launch" ]]; then

doesn't work right with farmr_skip_launch=true, and still executes farmr. I would suggest something more like this, which works and is also sensitive to true or false state:

if [ "$farmr_skip_launch" == "true" ]; then

I can submit a PR if that is helpful.

[guydavis]

Good eye! Yes, typo was there. Now fixed in latest :develop images.

[LeroyINC]

Since this will be fixed in the next release.. Can someone clarify exactly where in the docker-compose.yml file "-e farmr_skip_launch=true" should be? Would that me in the main section for Machinaris: or under the environment: section?

i also see high CPU usage for farmr and do not use this service.. so i would not mind having it disabled also.

[guydavis]

Hi, this is available in :test image stream now, you can simply add to the environment section of docker-compose.yml:

        environment:
            -  farmr_skip_launch=true

[LeroyINC]

thanks

[LeroyINC]

would that be in the environment section for each fork? or juts for the main Machinaris section?

[guydavis]

Hi, in :latest and :test images, this farmr_skip_launch variable is required for each container on each harvester, In the :develop image (and soon test, then eventually latest), Farmr has been disabled by default. A month ago many Machinaris users were extremely bullish on integration with Farmr. Now that they've actually tried Farmr for a month or so, there is no longer any such enthusiam. Sorry for any troubles you have encountered!

[LeroyINC]

thanks for the update... I understand