Blender: output node and EXR

[simonebarbieri]

Changelog Description

Output node now works correctly for Multilayer EXR and keeps existing links. The output now is handled entirely by the compositor node tree.

Additional info

Output node in the compositing node tree wasn't working correctly for Multilayer EXR. Now existing links are recreated when creating an instance. Moreover, now the output is handled only by the compositor tree. Finally, when creating a render instance, a new version of the workfile is created, instead of overwriting the current one.

Testing notes:

1. Set the output to be multilayer EXR.
2. Create a new render instance.
3. Check if the output node in the compositor node tree has all the necessary links.
4. Try deleting the instance (but not the node in the tree), and recreate the instance. The node should have the same links as before.
5. Publish the render and check that the layers are correct.
6. Change to PNG or any other format, and try publish it again, and check that the render is ok, expecially the beauty render.

[ynbot]

Task linked: OP-7120 Blender: output node and EXR

[LiborBatek]

Besides basic functionality I have tried to add some Custom AOV in Ayon blender addon settings and getting this error when hitting Create for RenderMain instance in blender (3.6.5)

My Ayon Settings:

Error shown during Publish Create of the Render Instance>

Traceback (most recent call last):
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.2-nightly.2\openpype\pipeline\create\context.py", line 2059, in _create_with_unified_error
    result = creator.create(*args, **kwargs)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.2-nightly.2\openpype\hosts\blender\plugins\create\create_render.py", line 25, in create
    prepare_rendering(collection)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.2-nightly.2\openpype\hosts\blender\api\render_lib.py", line 233, in prepare_rendering
    aov_list, custom_passes = set_render_passes(settings)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.2-nightly.2\openpype\hosts\blender\api\render_lib.py", line 127, in set_render_passes
    cp_name = cp[0]
KeyError: 0

[LiborBatek]

Also because of fully automated approach speaking of output of AOVs, we need to incorporate the missing AOVs into Addon Settings too (manually created AOVs by user get swiped so the only way to create those are via predefined ones in settings)

It probably depends which render engine being used (these below in Eevee only?) ...I would just ignore those if for ie Cycles renderer being chosen by user and not these AOVs available? And vice verse if Cycles AOV not present in Eevee...just ignoring it during Node tree creation for output.

[LiborBatek]

It seems that we having issue when speaking of Combined AOV aka beauty image...

Right now when have it ON in the Settings it does not produce separate image_combined just main beauty image which is wrong...there should be both outputted.
At least happening when using format like PNG or Tiff aka separate files for passes.

This being namely an issue when using some post effect nodes on the output via Compositor...in that case we just output the processed image and theres no any raw "beauty" image without those post FXs.

If u want to test it too...

1. Set output to lets say PNG
2. Turn on Combined AOV in the Settings
   3)Do some basic post via Compositor nodes
   4)Create Render Instance and render the sequence

There will be then just the composited seq outputed not any sort of raw unprocessed image. We need to tackle this scenario for sure. Otherwise any compositing via Compositor wont be feasible to use by users...

Here I have break down for possible solution...white being as is now and green optional solution??

[LiborBatek]

Even tho it successfully creates the desired outputs (also work when "Combined" added) it fails due to wrong output file path (if version mismatching as usual) but the Repair action fails and its impossible to Publish if not corrected by manually entering file path correct to DB. See attached scrngrab and check output file node and global output path on the right (the Repair does not fix the path and stays empty). Small note, my global file path was empty before too...but should work either imho

[LiborBatek]

Small note: My assumption being that the Validator kicks in even tho the global file path shouldnt matter when using just output node in the compositor.

But I could be wrong...

[LiborBatek]

The render job renders fine on DL then, but gives me errors during Publish Job on burnins phase....I really dont know if connected or not....but I will put it here for the record rather.

 !!! AYON crashed:
2024-01-22 14:22:14:  0: STDOUT: Error:
2024-01-22 14:22:14:  0: STDOUT: Traceback (most recent call last):
2024-01-22 14:22:14:  0: STDOUT:   File "start.py", line 737, in main_cli
2024-01-22 14:22:14:  0: STDOUT:   File "C:\Program Files\Ynput\AYON 1.0.0-rc.2\dependencies\click\core.py", line 1130, in __call__
2024-01-22 14:22:14:  0: STDOUT:     return self.main(*args, **kwargs)
2024-01-22 14:22:14:  0: STDOUT:   File "C:\Program Files\Ynput\AYON 1.0.0-rc.2\dependencies\click\core.py", line 1055, in main
2024-01-22 14:22:14:  0: STDOUT:     rv = self.invoke(ctx)
2024-01-22 14:22:14:  0: STDOUT:   File "C:\Program Files\Ynput\AYON 1.0.0-rc.2\dependencies\click\core.py", line 1657, in invoke
2024-01-22 14:22:14:  0: STDOUT:     return _process_result(sub_ctx.command.invoke(sub_ctx))
2024-01-22 14:22:14:  0: STDOUT:   File "C:\Program Files\Ynput\AYON 1.0.0-rc.2\dependencies\click\core.py", line 1404, in invoke
2024-01-22 14:22:14:  0: STDOUT:     return ctx.invoke(self.callback, **ctx.params)
2024-01-22 14:22:14:  0: STDOUT:   File "C:\Program Files\Ynput\AYON 1.0.0-rc.2\dependencies\click\core.py", line 760, in invoke
2024-01-22 14:22:14:  0: STDOUT:     return __callback(*args, **kwargs)
2024-01-22 14:22:14:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\cli.py", line 260, in run
2024-01-22 14:22:14:  0: STDOUT:     runpy.run_path(script, run_name="__main__", )
2024-01-22 14:22:14:  0: STDOUT:   File "runpy.py", line 268, in run_path
2024-01-22 14:22:14:  0: STDOUT:   File "runpy.py", line 97, in _run_module_code
2024-01-22 14:22:14:  0: STDOUT:   File "runpy.py", line 87, in _run_code
2024-01-22 14:22:14:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\scripts\otio_burnin.py", line 821, in <module>
2024-01-22 14:22:14:  0: STDOUT:     burnins_from_data(
2024-01-22 14:22:14:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\scripts\otio_burnin.py", line 645, in burnins_from_data
2024-01-22 14:22:14:  0: STDOUT:     ffprobe_data = _get_ffprobe_data(full_input_path)
2024-01-22 14:22:14:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\scripts\otio_burnin.py", line 63, in _get_ffprobe_data
2024-01-22 14:22:14:  0: STDOUT:     raise RuntimeError("Failed to run: %s" % command)
2024-01-22 14:22:14:  0: STDOUT: RuntimeError: Failed to run: ['C:\\Users\\lbate\\AppData\\Local\\Ynput\\AYON\\addons\\ayon_third_party_1.0.0\\ayon_third_party\\downloads\\ffmpeg\\bin\\ffprobe.exe', '-v', 'quiet', '-print_format', 'json', '-show_format', '-show_streams', 'C:/projects/Ayon_Tests/blender/b3d_shots/sh010/work/lighting/renders/blender/ATS_sh010_lighting_v019/sh010_renderLightingMain_beauty_h264.mp4']
2024-01-22 14:22:14:  0: STDOUT: Traceback (most recent call last):
2024-01-22 14:22:14:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\dependency_packages\ayon_2310271602_windows.zip\dependencies\pyblish\plugin.py", line 527, in __explicit_process
2024-01-22 14:22:14:  0: STDOUT:     runner(*args)
2024-01-22 14:22:14:  0: STDOUT:   File "<string>", line 90, in process
2024-01-22 14:22:14:  0: STDOUT:   File "<string>", line 355, in main_process
2024-01-22 14:22:14:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\execute.py", line 267, in run_openpype_process
2024-01-22 14:22:14:  0: STDOUT:     return run_ayon_launcher_process(*args, **kwargs)
2024-01-22 14:22:14:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\execute.py", line 246, in run_ayon_launcher_process
2024-01-22 14:22:14:  0: STDOUT:     return run_subprocess(args, env=env, **kwargs)
2024-01-22 14:22:14:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\execute.py", line 162, in run_subprocess
2024-01-22 14:22:14:  0: STDOUT:     raise RuntimeError(exc_msg)
2024-01-22 14:22:14:  0: STDOUT: RuntimeError: Executing arguments was not successful: "(['C:\\Program Files\\Ynput\\AYON 1.0.0-rc.2\\ayon_console.exe', 'run', 'C:\\Users\\lbate\\AppData\\Local\\Ynput\\AYON\\addons\\openpype_3.18.5-nightly.2\\openpype\\scripts\\otio_burnin.py', 'C:/Users/lbate/AppData/Local/Temp/tmpoajh7kdm.json'],)"
2024-01-22 14:22:14:  0: STDOUT: Output:
2024-01-22 14:22:14:  0: STDOUT: >>> loading environments ...
2024-01-22 14:22:14:  0: STDOUT:   - global AYON ...
2024-01-22 14:22:14:  0: STDOUT:   - for addons ...
2024-01-22 14:22:14:  0: STDOUT: *** AYON [1.0.0-rc.2] 
2024-01-22 14:22:14:  0: STDOUT: >>> Using AYON from [ C:\Program Files\Ynput\AYON 1.0.0-rc.2 ]
2024-01-22 14:22:14:  0: STDOUT: ... AYON variant: [ production ]
2024-01-22 14:22:14:  0: STDOUT: ... AYON bundle:  [ 2024.10.0-24-01-22-B3d_FIX-03 ]
2024-01-22 14:22:14:  0: STDOUT: ... running: C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\scripts\otio_burnin.py C:/Users/lbate/AppData/Local/Temp/tmpoajh7kdm.json
2024-01-22 14:22:14:  0: STDOUT: * Burnin script started

[LiborBatek]

Speaking of function of this PR, it works when speaking about Blender itself but fails on Deadline Publish Job.

In Blender all working Fine and Validates ok. All AOVs being correctly premade in Compositor and File Output node sets correct output names and paths. It also produces all render sequences as supposed to into work render folder. So this part works fine, but the issue comes when publish job starting...

After some discussion with @antirotor it seems that the reason of failure being the JSON not being created correctly and that some render sequences being present there more than once thus causing issues when doing reviewables and burnins...

Im attaching the JSON for your inspection and also DL job publish log with the failure...
Its quite certain that this PR causing the DL issues so it needs resolution before merging this PR.
DL_PublishJob_LOG.txt

renderLightingMain_metadata.json.txt

[LiborBatek]

I have done single test and it doesnt produce beauty aov ATM ...there is a missing connection from Render Layers directly to Output node when creating new RenderMain publish Instance.

So it then fails on DL on Publish Job due to missing that files for beauty....

[mkolar]

@simonebarbieri we'll nee ayon port for this too.

[tokejepsen]

@simonebarbieri is this ready for another round of testing?

[simonebarbieri]

@tokejepsen I've got one last request from @LiborBatek and then it should be ready! Thanks :)

[LiborBatek]

Not sure what that empty means... I guess we are missing proper naming here:

Is that Composite aov or?

[LiborBatek]

After adjustment of that empty entry all works as expected!

Did try multiple cases / variety of settings for render output and all seems good!

Also turning on/off Compositing knob works well for both multilayer EXRs or single PNGs for separate output...