OP-6382 - Thumbnail Integration Problem

[tokejepsen]

Changelog Description

This ticket alerted to 3 different cases of integration issues;

• Using the Tray Publisher with the same image format (extension) for representation and review representation.
• Clash on publish file path from output definitions in ExtractOIIOTranscode.
• Clash on publish file from thumbnail in ExtractThumbnail

There might be an issue with this fix, if a studio does not use the {output} token in their render anatomy template. But thinking if they have customized it, they will be responsible to maintain these edge cases.

Testing notes:

1. Use Tray Publisher to publish Image with two different images of same extension.
   

2. Publish should succeed.

3. Setup ayon+settings://core/publish/ExtractOIIOTranscode/profiles with settings below.
   

4. Setup single frame render from Maya and publish to Deadline.

5. Publish on Deadline should succeed.

6. Setup ayon+settings://core/publish/ExtractOIIOTranscode/profiles with settings below (just adding review tag from above settings).
   

7. Setup single frame render from Maya and publish to Deadline.

8. Publish on Deadline should succeed.

[ynbot]

Task linked: OP-6382 Thumbnail integration problem

[LiborBatek]

Did test the first case using Tray Publisher

and getting this error on Integrate Asset

DEBUG: Establishing staging directory @ C:\Users\lbate\AppData\Local\Temp\traypublisher_70zblecs
DEBUG: Looking for matching profile for: hosts: "traypublisher" | families: "image" | task_names: "art" | task_types: "Art"
DEBUG: "image" not found in "families": ['review', 'render', 'prerender']
DEBUG: "traypublisher" not found in "hosts": ['standalonepublisher']
DEBUG: "traypublisher" not found in "hosts": ['maya']
DEBUG: "image" not found in "families": ['online']
DEBUG: "traypublisher" not found in "hosts": ['max']
DEBUG: Profile selected: {'hosts': [], 'task_types': [], 'task_names': [], 'template_name': 'publish', 'families': []}
DEBUG: Subset: imageReference
DEBUG: http://localhost:5000 "POST /graphql HTTP/1.1" 200 105
DEBUG: Response <RestApiResponse [200]>
INFO: Subset 'imageReference' not found, creating ...
DEBUG: Prepared subset: imageReference
DEBUG: http://localhost:5000 "POST /graphql HTTP/1.1" 200 105
DEBUG: Response <RestApiResponse [200]>
DEBUG: Source: C:\Projects\Temp_Data\volcanoFiction.png
DEBUG: Creating new version ...
DEBUG: Prepared version: v001
DEBUG: http://localhost:5000 "POST /graphql HTTP/1.1" 200 112
DEBUG: Response <RestApiResponse [200]>
DEBUG: Anatomy template name: publish
DEBUG: Anatomy template name: publish
Traceback (most recent call last):
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 183, in process
    self.register(instance, file_transactions, filtered_repres)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 283, in register
    file_transactions.add(src, dst)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\file_transaction.py", line 98, in add
    raise DuplicateDestinationError(
openpype.lib.file_transaction.DuplicateDestinationError: Transfer to destination is already in queue: C:\Projects\Temp_Data\volcanoFiction.png -> C:\projects\Ayon_Tests\2d_comps\artwork_A\publish\image\imageReference\v001\ATS_artwork_A_imageReference_v001.png. It's not allowed to be replaced by a new transfer from C:\Projects\Temp_Data\volcanoFiction_prev.png

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\pyblish\plugin.py", line 527, in __explicit_process
    runner(*args)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 188, in process
    six.reraise(KnownPublishError,
  File "six.py", line 718, in reraise
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 183, in process
    self.register(instance, file_transactions, filtered_repres)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 283, in register
    file_transactions.add(src, dst)
  File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\file_transaction.py", line 98, in add
    raise DuplicateDestinationError(
openpype.pipeline.publish.publish_plugins.KnownPublishError: Transfer to destination is already in queue: C:\Projects\Temp_Data\volcanoFiction.png -> C:\projects\Ayon_Tests\2d_comps\artwork_A\publish\image\imageReference\v001\ATS_artwork_A_imageReference_v001.png. It's not allowed to be replaced by a new transfer from C:\Projects\Temp_Data\volcanoFiction_prev.png

I was using two PNG images heres my Publisher settings

[moonyuet]

Guess the error above is related to this code:

OpenPype/openpype/plugins/publish/integrate.py

Lines 179 to 181 in c8e00d4

	file_transactions = FileTransaction(log=self.log,
	# Enforce unique transfers
	allow_queue_replacements=False)

when allow_queue_requirements set to False

[LiborBatek]

Did anotther test step (05) and Im getting error on DL for publish job:

2024-01-24 11:33:40:  0: STDOUT: DEBUG:pyblish.IntegrateAsset:Anatomy template name: publish_render
2024-01-24 11:33:40:  0: STDOUT: ERROR:pyblish.plugin:Traceback (most recent call last):
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 183, in process
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 283, in register
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\file_transaction.py", line 98, in add
2024-01-24 11:33:40:  0: STDOUT:     raise DuplicateDestinationError(
2024-01-24 11:33:40:  0: STDOUT: openpype.lib.file_transaction.DuplicateDestinationError: Transfer to destination is already in queue: C:\Users\lbate\AppData\Local\Temp\op_transcoding_cax4llrd\Main.1001.jpg -> C:\projects\Ayon_Tests\maya\props\crate\publish\render\renderGenericMain_beauty\v001\ATS_crate_renderGenericMain_beauty_v001.jpg. It's not allowed to be replaced by a new transfer from C:\Users\lbate\AppData\Local\Temp\pyblish_tmp_9hhlfndf\Main.1001_thumb.jpg
2024-01-24 11:33:40:  0: STDOUT: During handling of the above exception, another exception occurred:
2024-01-24 11:33:40:  0: STDOUT: Traceback (most recent call last):
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\pyblish\plugin.py", line 527, in __explicit_process
2024-01-24 11:33:40:  0: STDOUT:     runner(*args)
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 188, in process
2024-01-24 11:33:40:  0: STDOUT:   File "six.py", line 718, in reraise
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 183, in process
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\plugins\publish\integrate.py", line 283, in register
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\file_transaction.py", line 98, in add
2024-01-24 11:33:40:  0: STDOUT:     raise DuplicateDestinationError(
2024-01-24 11:33:40:  0: STDOUT: openpype.pipeline.publish.publish_plugins.KnownPublishError: Transfer to destination is already in queue: C:\Users\lbate\AppData\Local\Temp\op_transcoding_cax4llrd\Main.1001.jpg -> C:\projects\Ayon_Tests\maya\props\crate\publish\render\renderGenericMain_beauty\v001\ATS_crate_renderGenericMain_beauty_v001.jpg. It's not allowed to be replaced by a new transfer from C:\Users\lbate\AppData\Local\Temp\pyblish_tmp_9hhlfndf\Main.1001_thumb.jpg
2024-01-24 11:33:40:  0: STDOUT: Traceback (most recent call last):
2024-01-24 11:33:40:  0: STDOUT:   File "<string>", line 183, in process
2024-01-24 11:33:40:  0: STDOUT:   File "<string>", line 283, in register
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\file_transaction.py", line 98, in add
2024-01-24 11:33:40:  0: STDOUT:     raise DuplicateDestinationError(
2024-01-24 11:33:40:  0: STDOUT: openpype.lib.file_transaction.DuplicateDestinationError: Transfer to destination is already in queue: C:\Users\lbate\AppData\Local\Temp\op_transcoding_cax4llrd\Main.1001.jpg -> C:\projects\Ayon_Tests\maya\props\crate\publish\render\renderGenericMain_beauty\v001\ATS_crate_renderGenericMain_beauty_v001.jpg. It's not allowed to be replaced by a new transfer from C:\Users\lbate\AppData\Local\Temp\pyblish_tmp_9hhlfndf\Main.1001_thumb.jpg
2024-01-24 11:33:40:  0: STDOUT: During handling of the above exception, another exception occurred:
2024-01-24 11:33:40:  0: STDOUT: Traceback (most recent call last):
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\dependency_packages\ayon_2401161802_windows.zip\dependencies\pyblish\plugin.py", line 527, in __explicit_process
2024-01-24 11:33:40:  0: STDOUT:     runner(*args)
2024-01-24 11:33:40:  0: STDOUT:   File "<string>", line 188, in process
2024-01-24 11:33:40:  0: STDOUT:   File "six.py", line 718, in reraise
2024-01-24 11:33:40:  0: STDOUT:   File "<string>", line 183, in process
2024-01-24 11:33:40:  0: STDOUT:   File "<string>", line 283, in register
2024-01-24 11:33:40:  0: STDOUT:   File "C:\Users\lbate\AppData\Local\Ynput\AYON\addons\openpype_3.18.5-nightly.2\openpype\lib\file_transaction.py", line 98, in add
2024-01-24 11:33:40:  0: STDOUT:     raise DuplicateDestinationError(
2024-01-24 11:33:40:  0: STDOUT: openpype.pipeline.publish.publish_plugins.KnownPublishError: Transfer to destination is already in queue: C:\Users\lbate\AppData\Local\Temp\op_transcoding_cax4llrd\Main.1001.jpg -> C:\projects\Ayon_Tests\maya\props\crate\publish\render\renderGenericMain_beauty\v001\ATS_crate_renderGenericMain_beauty_v001.jpg. It's not allowed to be replaced by a new transfer from C:\Users\lbate\AppData\Local\Temp\pyblish_tmp_9hhlfndf\Main.1001_thumb.jpg
2024-01-24 11:33:40:  0: STDOUT: !!! ERR: 2024-01-24 11:33:40,083 >>> { CLI-publish }: [ Failed IntegrateAsset: Transfer to destination is already in queue: C:\Users\lbate\AppData\Local\Temp\op_transcoding_cax4llrd\Main.1001.jpg -> C:\projects\Ayon_Tests\maya\props\crate\publish\render\renderGenericMain_beauty\v001\ATS_crate_renderGenericMain_beauty_v001.jpg. It's not allowed to be replaced by a new transfer from C:\Users\lbate\AppData\Local\Temp\pyblish_tmp_9hhlfndf\Main.1001_thumb.jpg -- ('C:\\Users\\lbate\\AppData\\Local\\Ynput\\AYON\\addons\\openpype_3.18.5-nightly.2\\openpype\\plugins\\publish\\integrate.py', 98, 'add', 'raise DuplicateDestinationError(') ] 
2024-01-24 11:33:40:  0: STDOUT: --- your system is set to use custom CA certificate bundle.

[LiborBatek]

In the last testing step with review tag added, it ends pretty much the same with failure of the DL publish job with the exact same error msg. so check that above too.

[tokejepsen]

Did test the first case using Tray Publisher

@LiborBatek I've replicated here and it works. I can see what you are using Production bundle, could you let me know how you build and are setup, so I can replicate here?

[LiborBatek]

Did test the first case using Tray Publisher

@LiborBatek I've replicated here and it works. I can see what you are using Production bundle, could you let me know how you build and are setup, so I can replicate here?

You can build from within your OP Repo by

OpenPype\server_addon> python .\create_ayon_addons.py

then just locate the folder \server_addon\packages\ and upload those needed packages on Ayon server and put those into new Addon Bundle, copy settings from previously used Addon bundle and set the new one as Production so it will be used when run Ayon Launcher in normal mode (aka production).

I rather do it this way...to be sure it works really. Not much using developer mode...

[antirotor]

Tested all combinations - same image, two different images, jpgs, pngs, representation and little thumbnail and all works! Thanks.

Note

Tested using dev-mode

[LiborBatek]

@tokejepsen ok I re did the test again and double checked my addons being correctly set and in production and finally also works for me. sorry for confusion (probably didnt deleted local ayon data folder content before!)

Went smoothly in Tray Publisher and with thumbnails integrated on Ayon too

and on Ayon: