Chore: Use AYON entities

[iLLiCiTiT]

Changelog Description

This PR is removing ayon_core.client and replaces it's usage with ayon_api calls. That changed structure of all entity types from OpenPype compatible to AYON.

Additional info

I did what I could to change every line of code which are working with entities to work properly, but I've probably missed something so please report anything you hit. And I didn't test most of the changes.

Entity structure changes sometimes required change of logic and function arguments. Some of those changes happened on the spot, so if you hit a change that is questionable then please comment. I'll mention few questionable changes myself. In some cases I did more changes than just change of query and used keys because it helper me with other changes, or I couldn't resist to refactor few lines.

Major changes

• Project, asset, subset, version, representation and workfile documents are now project, folder, product, version, representation and workfile entities.
• Removed ayon_core.client. Its functions were replaced with function from ayon_api.
• Implemented initialize_ayon_connection in ayon_core.lib.ayon_connection to initialize ayon connection. Initialization of the connection happened automatically when a call in ayon_core.client happened. It is automatically called when cli.main is called and on host install or ayon plugin install, so all important places should be covered. This affects local root overrides so it is important to happen as soon as possible in any process that might work with Anatomy.
• Added usage of task entity. Before this PR task information was on asset document, but AYON has task entity separated from folder entity so many places did have to change. Almost all places did expect asset document and task name and it was enough, now is usually needed only task entity, which affected arguments of functions (e.g. get_product_name).
• Renamed multiple funtions which contained asset, e.g. get_current_asset_name > get_current_folder_path, get_current_project_task > get_current_project_folder.
• Context in load plugins does not have "asset" and "subset" keys, but "folder" and "product".

Notes

I was very struggling with changes in unreal host. There is a lot of "asset" naming that is not related to OpenPype asset, and also there was a lot of unusuall way of working with representations and what is stored to container data. If it is working as expected now then it's fine, details can be changed when unreal will be in separated addon.

Questions

• Function get_product_name now expects task entity, but from api point of view it might be better to expect task_name and task_type as that's what is really needed for the function. Changed the function to expect task name and task type instead of task entity.
• Function initialize_ayon_connection is really important and I'm not sure if should stay in ayon_core.lib, I couldn't find more suitable place and name.

Testing notes:

Please consider that some of the bugs may not come from this PR, as was found out many times in previous PRs.

Keep in mind that this PR will completelly break compatibility with almost any other addon, so ftrack, shotgrid, kitsu, slack etc. won't work.

Test every host, every addon, every possible action that can be done should be tested. Create, Publish, Load, Manage in scene inventory. Open old scene, with previously created instances and loaded containers, all of that should be working as nothing changed (the scenes should be from AYON mode).

All tools should show correct data and should work as expected. Console should not log any errors.

• Publisher allows to create all kind of instances, change their context and product name, change attributes. Crashed plugins list does not contain anything suspicious (usually there are deadline plugins). Publishing works. Integration creates entities and copies files. Thumbnails are integrated etc.
• Workfiles tool should open, save as both workfiles and published workfiles. Show notes about the workfile etc.
• Loader tool should load.
• Scene inventory can change version, update to latest, or change folder/product/representation.

Hosts

• 3dsMax
• Aftereffect
• Blender
• Fusion
• Hiero
• Houdini
• Maya
• Nuke
• Photoshop
• Resolve - almost done
• SubstancePainter
• TVpaint
• Unreal

Other

• Push to project works

[moonyuet]

Works in Max now.

[LiborBatek]

when inspecting the report there is obviously missing Review product - also not just visually in the UI but its not published at all

Is that related to this PR, or it happens in develop too?

It also does happen in the develop so not related to this PR

[BigRoy]

when inspecting the report there is obviously missing Review product - also not just visually in the UI but its not published at all

Is that related to this PR, or it happens in develop too?

It also does happen in the develop so not related to this PR

Yup, likely has to do with the review instance 'attaching' itself to the model main instance in this case so likely "removing" itself and merging the instance with the model main instance or alike.

Likely that does not happen if the review instance is not attached.

[MustafaJafar]

Cool, I no longer see previous errors.
New Error 🎉, AYON publish job fails on farm.

2024-03-12 16:49:05:  0: STDOUT: ERROR:pyblish.plugin:Traceback (most recent call last):
2024-03-12 16:49:05:  0: STDOUT:   File "C:\Users\Mustafa Taher\AppData\Local\Ynput\AYON\dependency_packages\ayon_2403111503_windows.zip\dependencies\pyblish\plugin.py", line 527, in __explicit_process
2024-03-12 16:49:05:  0: STDOUT:     runner(*args)
2024-03-12 16:49:05:  0: STDOUT:   File "E:\Ynput\ayon-core\client\ayon_core\plugins\publish\integrate_hero_version.py", line 126, in process
2024-03-12 16:49:05:  0: STDOUT:   File "E:\Ynput\ayon-core\client\ayon_core\plugins\publish\integrate_hero_version.py", line 242, in integrate_instance
2024-03-12 16:49:05:  0: STDOUT: KeyError: 'taskId'
2024-03-12 16:49:05:  0: STDOUT: Traceback (most recent call last):
2024-03-12 16:49:05:  0: STDOUT:   File "C:\Users\Mustafa Taher\AppData\Local\Ynput\AYON\dependency_packages\ayon_2403111503_windows.zip\dependencies\pyblish\plugin.py", line 527, in __explicit_process
2024-03-12 16:49:05:  0: STDOUT:     runner(*args)
2024-03-12 16:49:05:  0: STDOUT:   File "<string>", line 126, in process
2024-03-12 16:49:05:  0: STDOUT:   File "<string>", line 242, in integrate_instance
2024-03-12 16:49:05:  0: STDOUT: KeyError: 'taskId'
2024-03-12 16:49:05:  0: STDOUT: !!! ERR: 2024-03-12 16:49:05,056 >>> { CLI-publish }: [ Failed IntegrateHeroVersion: 'taskId' -- ('E:\\Ynput\\ayon-core\\client\\ayon_core\\plugins\\publish\\integrate_hero_version.py', 242, 'integrate_instance', '') ] 

[LiborBatek]

Didnt find any issues in Maya atm.... so far good too.
Edited: Fusion been tested too and no issues so far...good too.

[MustafaJafar]

Cool, I no longer see previous errors.
New Error 🎉, when clicking Switch to versioned

Traceback (most recent call last):
  File "E:\Ynput\ayon-core\client\ayon_core\tools\sceneinventory\view.py", line 234, in <lambda>
    lambda: _on_switch_to_versioned(items)
  File "E:\Ynput\ayon-core\client\ayon_core\tools\sceneinventory\view.py", line 187, in _on_switch_to_versioned
    standard_versions = ayon_api.get_versions(
  File "E:\Ynput\ayon-launcher\.venv\lib\site-packages\ayon_api\_api.py", line 855, in get_versions
    return con.get_versions(*args, **kwargs)
TypeError: get_versions() missing 1 required positional argument: 'project_name'

[simonebarbieri]

I am getting this error during the thumbnail integration in Blender

Tested again and can confirm that has been fixed!

[MustafaJafar]

Tested in Houdini, So far so good.

• Create
• Local Publish
• Farm Publish
• Load
• Manage

Regards workfiles: I hit an issue when switch context using workfiles tools.. And, I was able to reproduce the same behavior in develop.

[iLLiCiTiT]

I hit an issue when switch context using workfiles tools.. And, I was able to reproduce the same behavior in develop.

I'll dm you, but won't be fixed in this PR if happens in develop too. Please check Houdini checkbox if you finished with houdini testing, and everything seems to be working 🙂 .

[jakubjezek001]

Nuke is now working also with Farm rendering and publishing

[jakubjezek001]

In Resolve following error occurs if I try to start Creator from Ayon menu:

Traceback (most recent call last):
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\hosts\resolve\api\menu.py", line 138, in on_create_clicked
    host_tools.show_creator()
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\tools\utils\host_tools.py", line 383, in show_creator
    _SingletonPoint.show_tool_by_name("creator", parent)
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\tools\utils\host_tools.py", line 349, in show_tool_by_name
    cls.helper.show_tool_by_name(tool_name, parent, *args, **kwargs)
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\tools\utils\host_tools.py", line 309, in show_tool_by_name
    self.show_creator(parent, *args, **kwargs)
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\tools\utils\host_tools.py", line 113, in show_creator
    creator_tool.refresh()
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\tools\creator\window.py", line 384, in refresh
    get_current_project_settings()
KeyError: 'global'

It seems to be unrelated from this PR subject.

Update:
Found the bug and fixed it in following commit.

[jakubjezek001]

In Resolve during loading this had been blocking the loading. Again, I am not sure if it is related to this PRs changes.

Traceback (most recent call last):
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\tools\loader\models\actions.py", line 740, in _load_representations_by_loader
    load_with_repre_context(
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\pipeline\load\utils.py", line 323, in load_with_repre_context
    return loader.load(repre_context, name, namespace, options)
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\hosts\resolve\plugins\load\load_clip.py", line 45, in load
    timeline_item = plugin.ClipLoader(
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\hosts\resolve\api\plugin.py", line 415, in load
    if "slate" in self.data["versionData"]["families"]:
KeyError: 'versionData'

EDIT:
So I have found the cause of the issue and fixed it. Next commit.

[jakubjezek001]

Resolve is working fine now. I was testing publishing and loading and workfiles and all was fine in the end. But during inventory management there was an issue and this is something I could not figure out. It is happening during update to different version of loaded container. It seems that attrib is some key which suppose to be on version, since I was checking it and it occurs to be even in nuke, maya, vusion etc. But here it is missing.

Traceback (most recent call last):
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\tools\sceneinventory\view.py", line 279, in <lambda>
    lambda: self._show_version_dialog(items))
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\tools\sceneinventory\view.py", line 695, in _show_version_dialog
    self._update_containers(items, version)
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\tools\sceneinventory\view.py", line 822, in _update_containers
    update_container(item, item_version)
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\pipeline\load\utils.py", line 532, in update_container
    return Loader().update(container, context)
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\hosts\resolve\plugins\load\load_clip.py", line 79, in update
    loader = plugin.ClipLoader(self, context)
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\hosts\resolve\api\plugin.py", line 323, in __init__
    assert self._populate_data(), str(
  File "C:\CODE\__PYPE\ayon-core\client\ayon_core\hosts\resolve\api\plugin.py", line 371, in _populate_data
    self.data["versionAttributes"] = self.context["version"]["attrib"]
KeyError: 'attrib'

[iLLiCiTiT]

It seems that attrib is some key which suppose to be on version, since I was checking it and it occurs to be even in nuke, maya, vusion etc. But here it is missing.

Should be fixed, please revalidate.