TFarm: Getting it working

[damies13]

opentoonz/toonz/sources/toonzfarm/tfarmcontroller/tfarmcontroller.cpp
Line 60: TFilePath name(L"SOFTWARE\OpenToonz\OpenToonz\1.0\FARMROOT");

This registry key did not exist in version 1.1.2, it was instead created as:
SOFTWARE\OpenToonz\OpenToonz\1.1\FARMROOT

As a work around I was able to get tfarmcontroller to run in console mode after creating SOFTWARE\OpenToonz\OpenToonz\1.0\FARMROOT using the value from SOFTWARE\OpenToonz\OpenToonz\1.1\FARMROOT

I am not familiar enough with your code to know what variable should be used to denote the major / minor version number, but I suspect this should be an easy fix.

Hopefully I have provided enough information to get this resolved in a future version.

Dave.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[damies13]

OK I have been playing around and I can't seem to get tfarmcontroller or tfarmserver to run on any platform, I have tried:

• Windows 7 64 bit (Standard Download)
• Mac OSX (Standard Download)
• Ubuntu 16.04 (compiled using https://github.com/opentoonz/opentoonz/blob/master/doc/how_to_build_linux.md)

Has anyone got this working? if so on what platform and what hints can you give me?

Thanks,

Dave.

[damies13]

I had some success,

with help from procmon and reading the code I got it working on 1 machine on windows....

1. in the toonzfarm directory you need to create a config directory
2. in the config directory create a text file controller.txt, this just needs to contain the hostname of the machine running tfarmcontroller
3. restart tfarmcontroller (on the machine mentioned in controller.txt), it now comes up saying starting on port 1 not port 0
4. restart tfarmserver (it won't tell you its working yet)
5. select RenderFarm in OpenToonz, nothing happens (no error any more) so this is a good thing.
6. start a render task, now stuff happens:

• tfarmcontroller displays Task 0 assigned to
• tfarmserver displays the tcomposer.exe message
• tfarmserver displays Task completed at Fri Mar 3 10:45:46 2017
• tfarmcontroller displays Task 0 completed on

I'm still trying on other platforms...

[konero]

This is from the Harlequin manual, if it helps. Sorry, its long, I can delete if its no help. 👍

[damies13]

@konero thanks, this is a great help, will try it out when I get a chance. but at least I know now what the what files need to be in the config folder and what their format is. I was trying to guess how to specify the port numbers but hadn't figured it out. I had tried ":" and "," wouldn't have expected " ".

Thanks,

Dave.

[damies13]

Great news, I got a chance to test it out, using the info from my post plus the info from the Harlequin manual I got two windows 7 VM's operating as a tfarm.

Some lessons learned:

1. The entire contents of the "OpenToonz 1.1 stuff" folder (not just the toonzfarm folder) needs to be copied to a network share that all machines (OpenToonz Clients, tfarmcontroller and tfarmserver(s) ) can connect to with read write access.
2. Registry changes:
   2.1. OpenToonz 1.1 creates the registry settings in HKLM\SOFTWARE\OpenToonz\OpenToonz\1.1 these need to be copied to HKLM\SOFTWARE\OpenToonz\OpenToonz\1.0 as all OpenToonz still read from here.
   2.2. All the keys you copied (created?) in HKLM\SOFTWARE\OpenToonz\OpenToonz\1.0 need to be pointed to the "OpenToonz 1.1 stuff" network folder.
   2.3. You need to create FARMROOT and point it to the toonzfarm folder in the "OpenToonz 1.1 stuff" network folder.
   2.4. The path to the "OpenToonz 1.1 stuff" network folder needs to be identical for every machine.
3. Tasks Setup:
   3.1. When you set up the render tasks, you can only select a scene to add as a render task, so you need to use the Output Settings dialog to configure the scene render options first and save the scene before creating the task, the task doesn't update if you change these options later so if you want to change them you need to recreate the tasks.
   3.2. output of image types (JPEG, PNG, TIFF, etc) will break the scene up to groups of 10 frames and assign each group to a farm server.
   3.3. output of video types (MP4, AVI, etc) will assign assign each scene to a farm server, so if you only have 1 scene you'll get no benefit from having many servers in your farm. To get the most benefit from your farm break your move into many scenes, rather than one big scene.
4. CPU Usage: While the CPU usage was spread evenly across all cpu cores on the tfarm server, I never got over 25% CPU on 4 cores of an i7 CPU, memory also was not an issue (less than 50%) so I can only assume network disk io was the limiting factor in my farm, it was processing 5 4k frames to png per tfarm server, but my animation was very simple with only 2 columns in the Xsheet and 1 moving camera. More complex scenes might use more CPU and Memory.
5. Once I had a my tfarm working on 2 Windows 7 VM's I tried to use it with OpenToonz on a Mac. The Mac could connect to the tfarmcontroller and recognise both tfarmservers, and also received the jobs when the render task was started on the Mac. The render jobs progressed through all the states (waiting, running & completed) but did not actually do any work due to the differences in file path structure between the Mac and Windows 7 (see point 2.4 above). Below is paths shown when I tried the Mac client to Windows farm

5.1. Command Line shown on the Mac client:
"./Toonz 7.1.app/Contents/MacOS/tcomposer" "/Volumes/RedRoverNas/WD-MyBook1140-01/OpenToonz/stuff/projects/rowrow/scenes/rowrow.tnz" -tmsg 192.168.13.110 -o "/Volumes/RedRoverNas/WD-MyBook1140-01/OpenToonz/stuff/projects/rowrow/outputs/roworw.png" -range 1 50 -step 1 -shrink 1 -multimedia 1 -nthreads all -maxtilesize 2

5.2. Command Line shown on the Window 7 Server:
"tcomposer.exe "\Volumes\RedRoverNas\WD-MyBook1140-01\OpenToonz\stuff\projects\rowrow\scenes\rowrow.tnz" -tmsg 192.168.13.110 -o "\Volumes\RedRoverNas\WD-MyBook1140-01\OpenToonz\stuff\projects\rowrow\outputs\roworw.png" -range 41 50 -step 1 -shrink 1 -multimedia 1 -nthreads all -maxtilesize 2 -farm 1@win7pro-0001 -id 18.4"

5.3. "./Toonz 7.1.app/Contents/MacOS/tcomposer" didn't actually exist on my Mac, but still got translated to tcomposer.exe
5.4. /'s got translated to \'s
5.5. "\Volumes\RedRoverNas\WD-MyBook1140-01\OpenToonz\stuff" is still not a valid path on a Windows machine.

Hope this helps some get going.

Dave.

[damies13]

Next I will try with Linux, as the packaged Mac client doesn't seem to have tfarmcontroller or tfarmserver.

[damies13]

In trying to get this all working I have come across a bunch of issues, so I will raise some new issues and assign the ones I think I can fix to myself.

I'll link them here as I raise them:

• Linux Version of tfarm components uses MAC path for configfarmroot.txt #1125
• Windows Version of tfarm components Registry path hard coded version 1.0 #1126
• Mac Version of tfarm components path hard coded version 1.0 #1127
• ToonzFarm version number discrepancies #1129
• tfarmcontroller & tfarmserver on Mac OSX #1130
• toonzfarm & OpenToonz on different platforms #1131

[damies13]

Ok Mac OpenToonz connecting to tfarmcontroller & tfarmserver running on linux also doesn't work.

Command line shown on linux tfarmcontroller:
Fri Apr 7 22:59:17 2017 INF:
""./Toonz 7.1.app/Contents/MacOS/tcomposer" "/Volumes/RedRoverNas/WD-MyBook1140-01/OpenToonz/stuff/projects/rowrow/scenes/rowrow.tnz" -tmsg 192.168.13.110 -o "/Volumes/RedRoverNas/WD-MyBook1140-01/OpenToonz/stuff/projects/rowrow/outputs/roworw.png" -range 1 10 -step 1 -shrink 1 -multimedia 1 -nthreads all -maxtilesize 2"

I also have now successfully compiled OpenToonz on all 3 platforms.

In doing so I found out what happened to tfarmcontroller & tfarmserver on Mac OSX, these files are compiled into folders in the toonzfarm folder of the build folder, but the core app is compiled into a OpenToonz_1.1.app folder in the toonz folder also in the build folder.

I am now wondering what would be better, and am hoping to get some feed back on what would be best?:

• Move all the OpenToonz command line tools into OpenToonz_1.1.app/Contents/MacOS/ with the ones already there?
• Move all the OpenToonz command line tools into a seperate folder to be packaged shipped with with OpenToonz_1.1.app

#1130

[damies13]

I have submitted PR #1154, but referring back to the pages from the Harlequin manual posted above by @konero, I see there is more I need to cover, but at least we have a start that it specific to OpenToonz.

[RodneyBaker]

All of the sub issues referenced by this issue are closed.
This includes issues #1125. #1126, #1127, #1128, #1129, #1130, #1131 and #1154.
Recommend review of this issue to see what (if anything) is still open related to render farm usage.