Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blender: Dynamic workflow using linked libraries and overrides #3463

Conversation

kaamaurice
Copy link
Contributor

@kaamaurice kaamaurice commented Jul 1, 2022

Brief description

  • Use collections and overrides for the instances.

Description

Resolve #2890

@ynbot
Copy link
Contributor

ynbot commented Jul 1, 2022

Task linked: OP-2598 Demo generator

@mkolar mkolar added the type: feature Larger, user affecting changes and completely new things label Jul 4, 2022
@kaamaurice kaamaurice force-pushed the Enhancement/OP-2598_Blender-dynamic-workflow-using-linked-libraries branch from 56acc57 to 99c2d0b Compare July 7, 2022 09:01
Copy link
Member

@simonebarbieri simonebarbieri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello. Thank you for your work! Here are the main points we discussed on Thursday that need some changes:

  • A model loader behaves differently depending on the task it is launched from. This shouldn't happen. There should be two different loaders that execute the two alternatives.
  • After loading a layout, it is not possible to update the single rigs in it.
  • After loading a layout, a single animation instance is created. There should be an instance for each rig instead.

@Tilix4
Copy link
Collaborator

Tilix4 commented Jul 26, 2022

I'm wondering if we shouldn't take advantage of families for the layout/anim issue we have. Maybe the layout family should be updatable in the Animation task and it'll define how things are placed. It should be dissociated from the Layout task which is only meant to create the firsts versions of the layout family instance. anim family instances must always contain only actions from Blender, and are most of the time created in the Animation task but could be updated in the Rendering one for example.

Staging information are stored into the layout family and can be updated also in the Animation task.
To avoid confusion between the task and the family, maybe the layout family should be renamed to staging.

@kaamaurice kaamaurice force-pushed the Enhancement/OP-2598_Blender-dynamic-workflow-using-linked-libraries branch 2 times, most recently from 5c969a3 to 99bf5e2 Compare August 1, 2022 16:09
@kaamaurice kaamaurice force-pushed the Enhancement/OP-2598_Blender-dynamic-workflow-using-linked-libraries branch 3 times, most recently from 5778826 to 358a6ff Compare August 10, 2022 15:01
@kaamaurice kaamaurice force-pushed the Enhancement/OP-2598_Blender-dynamic-workflow-using-linked-libraries branch from dfd0f91 to 1d6aa2d Compare August 15, 2022 09:41
@kaamaurice kaamaurice force-pushed the Enhancement/OP-2598_Blender-dynamic-workflow-using-linked-libraries branch from 4bece85 to a8b0812 Compare August 25, 2022 10:11
@kaamaurice kaamaurice force-pushed the Enhancement/OP-2598_Blender-dynamic-workflow-using-linked-libraries branch from a8b0812 to 6e603fd Compare October 4, 2022 13:12
@@ -612,8 +612,14 @@ def register(self, instance, repres):
continue
repre_context[key] = template_data[key]

# Initialize ID, trying to get it from an earlier stage
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conversation related to being able predefine representation id https://github.com/pypeclub/OpenPype/discussions/3902#discussioncomment-3794454

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is not needed anymore right?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm currently developing the correct behaviour. It'll be put in new PR on this branch in some days/weeks. I'll notify you at this moment and it'll remove this change.

@iLLiCiTiT
Copy link
Member

PR adds OpenColorIO-Configs submodule to deprecated place.

@kaamaurice kaamaurice force-pushed the Enhancement/OP-2598_Blender-dynamic-workflow-using-linked-libraries branch from a6d13e7 to d97752b Compare October 24, 2022 15:01
@github-actions github-actions bot added this to the next-patch milestone Oct 28, 2022
@kaamaurice kaamaurice force-pushed the Enhancement/OP-2598_Blender-dynamic-workflow-using-linked-libraries branch from 13413a7 to cf46621 Compare November 3, 2022 11:14
@jakubjezek001 jakubjezek001 removed this from the next-patch milestone Nov 10, 2022
@kaamaurice kaamaurice force-pushed the Enhancement/OP-2598_Blender-dynamic-workflow-using-linked-libraries branch from 0dc0804 to 6ddb041 Compare November 15, 2022 15:03
Copy link
Member

@iLLiCiTiT iLLiCiTiT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR cannot be merged because of changes in integrator plugin which breaks publish workflow conversation here. We can't afford to add ability to define representation id before integrator plugin due to many reasons. Also the branch need update from develop branch.

We would like to have that changed so we could merge the PR.

@Tilix4
Copy link
Collaborator

Tilix4 commented Dec 13, 2022

The PR cannot be merged because of changes in integrator plugin which breaks publish workflow conversation here. We can't afford to add ability to define representation id before integrator plugin due to many reasons. Also the branch need update from develop branch.

We would like to have that changed so we could merge the PR.

@iLLiCiTiT I'm working on the seamless integration #3902 which includes the changes you require and a completely universal way to handle datablocks. It deeply refactors the plugin also. I'm not far from having it finished, therefore we'll be able to merge the current branch once my work have been merged into it!

@iLLiCiTiT
Copy link
Member

I'm not far from having it finished

Happy to hear :)

)

# Filter datablocks by loader types
if loader := containers_loaders.get(entity):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing whitespace after ':'
multiple statements on one line (colon)
whitespace before ':'


from . import pipeline

log = Logger.get_logger(__name__)

def add_datablocks_to_container(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expected 2 blank lines, found 1

wm = context.window_manager

# Try to match the container referencing the current collection
if matched_container := next(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing whitespace after ':'
multiple statements on one line (colon)
whitespace before ':'

wm = context.window_manager

# Try to match the container referencing the current collection
if matched_container := next(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing whitespace after ':'
multiple statements on one line (colon)
whitespace before ':'

if active_instance.name == new_instance_name:
self.report(
{"ERROR"},
f"Duplicate instance cannot have same name as original: {new_instance_name}",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (93 > 79 characters)

if not self.use_selection:
row = layout.row(align=True)

creator_plugin = context.scene["openpype_creators"][self.creator_name]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (82 > 79 characters)


gather_into_collection: bpy.props.BoolProperty(
name="Gather into collection",
description="Gather outliner entities when added to instance under single collection",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (94 > 79 characters)

# Items are defaults from current creator plugin
items=lambda self, context: [
(default, default, "")
for default in context.scene["openpype_creators"][self.creator_name][
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (81 > 79 characters)



class StructDescriptor:
"""Generic Descriptor to store and restor properties from blender struct."""
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (80 > 79 characters)

(Container, List of loaded datablocks)
"""
container = self._get_scene_container(container_metadata)
assert container, f"The asset is not loaded: {container_metadata.get('objectName')}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (92 > 79 characters)

)

# Filter datablocks by loader types
if loader := containers_loaders.get(entity):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing whitespace after ':'
multiple statements on one line (colon)
whitespace before ':'


from . import pipeline

log = Logger.get_logger(__name__)

def add_datablocks_to_container(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expected 2 blank lines, found 1

wm = context.window_manager

# Try to match the container referencing the current collection
if matched_container := next(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing whitespace after ':'
multiple statements on one line (colon)
whitespace before ':'

wm = context.window_manager

# Try to match the container referencing the current collection
if matched_container := next(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing whitespace after ':'
multiple statements on one line (colon)
whitespace before ':'

if active_instance.name == new_instance_name:
self.report(
{"ERROR"},
f"Duplicate instance cannot have same name as original: {new_instance_name}",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (93 > 79 characters)

if not self.use_selection:
row = layout.row(align=True)

creator_plugin = context.scene["openpype_creators"][self.creator_name]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (82 > 79 characters)


gather_into_collection: bpy.props.BoolProperty(
name="Gather into collection",
description="Gather outliner entities when added to instance under single collection",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (94 > 79 characters)

# Items are defaults from current creator plugin
items=lambda self, context: [
(default, default, "")
for default in context.scene["openpype_creators"][self.creator_name][
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (81 > 79 characters)



class StructDescriptor:
"""Generic Descriptor to store and restor properties from blender struct."""
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (80 > 79 characters)

(Container, List of loaded datablocks)
"""
container = self._get_scene_container(container_metadata)
assert container, f"The asset is not loaded: {container_metadata.get('objectName')}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (92 > 79 characters)

…_nodegroups

Bugfix: Switch asset duplicates nodegroups.
Args:
datablocks (List[bpy.types.ID], optional): Datablocks to process and append to instance. Defaults to None.
gather_into_collection (bool):
Process outliner gathering of elements under a single collection.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (81 > 79 characters)

appends the datablocks to it.

Args:
datablocks (List[bpy.types.ID], optional): Datablocks to process and append to instance. Defaults to None.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (118 > 79 characters)


@exec_process
def process(
self, datablocks: List[bpy.types.ID] = None, gather_into_collection = True
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (82 > 79 characters)
unexpected spaces around keyword / parameter equals

bpy.types.Scene.openpype_instance_active_index = bpy.props.IntProperty(
name="OpenPype Instance Active Index", options={"HIDDEN"}
)
# Cannot store the containers in the scene, it overloads the RAM when rendering
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (83 > 79 characters)


col.separator()

col.menu("SCENE_MT_openpype_instances_context_menu", icon='DOWNARROW_HLT', text="")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (91 > 79 characters)

entity (Union[bpy.types.Collection, bpy.types.Object]): Outliner entity to get children from.

Returns:
Set[Union[bpy.types.Collection, bpy.types.Object]]: All outliner children.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (82 > 79 characters)

For an Object, only objects parented to the given one.

Args:
entity (Union[bpy.types.Collection, bpy.types.Object]): Outliner entity to get children from.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (101 > 79 characters)

from openpype.hosts.blender.api.properties import OpenpypeInstance
from openpype.hosts.blender.api.utils import get_all_outliner_children

def _get_camera_from_datablocks(datablocks: List[bpy.types.ID])->Union[bpy.types.Object, None]:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expected 2 blank lines, found 1
line too long (95 > 79 characters)
missing whitespace around operator

@@ -0,0 +1,13 @@
"""Create a setdress asset."""
import bpy
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'bpy' imported but unused


avalon_info = obj[AVALON_PROPERTY]
avalon_info.update({"container_name": group_name})
plugin.link_to_collection(objects, asset_group)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

undefined name 'asset_group'

'action': action,
'create_animation': True if family == 'rig' else False,
'animation_asset': asset
"parent": asset_group,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

undefined name 'asset_group'


import bpy
from openpype.hosts.blender.api.properties import OpenpypeContainer
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'openpype.hosts.blender.api.properties.OpenpypeContainer' imported but unused

@@ -1,417 +1,137 @@
"""Load a rig asset in Blender."""

from pathlib import Path
from pprint import pformat
from typing import Dict, List, Optional
from typing import List, Tuple
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'typing.List' imported but unused
'typing.Tuple' imported but unused

"""Get images used by the datablocks.

Args:
datablocks (Set[bpy.types.ID], optional): Datablocks to get images from. Defaults to None.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (102 > 79 characters)


try:
bpy.ops.file.make_paths_relative()
except BaseException as e:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

local variable 'e' is assigned to but never used

* Fix: Switch transfer custom properties

* format line too long
label = "Extract workfile"
hosts = ["blender"]
families = ["workfile"]

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blank line contains whitespace

result = future.result().decode()

# Iterate through matched errors
if errors_stack := list(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing whitespace after ':'
multiple statements on one line (colon)
whitespace before ':'

invalid = self.get_invalid(instance)
if invalid:
raise RuntimeError(
f"Objects found with special characters in their name: {invalid}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (81 > 79 characters)

invalid_chars.remove(".")
for obj in instance:
# any is faster than regex to get char in obj.name and return bool
if obj is not None and any(char in invalid_chars for char in obj.name):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line too long (83 > 79 characters)

invalid = []

# Set of special characters except "_" and "." which is used in naming
invalid_chars = set(string.punctuation.replace("_",""))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing whitespace after ','

@@ -0,0 +1,40 @@
from typing import List

import bpy
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'bpy' imported but unused

str(Path(datablock.filepath).resolve()),
start=str(Path(bpy.data.filepath).parent.resolve()),
)
except BaseException as e:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

local variable 'e' is assigned to but never used

import bpy

from openpype.hosts.blender.api.utils import get_datablocks_with_filepath
from openpype.client.entities import get_representations
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'openpype.client.entities.get_representations' imported but unused

required=True,
)
args, unknown = parser.parse_known_args(
sys.argv[sys.argv.index("--") + 1 :]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whitespace before ':'

* Refactor: Loader API for blend files

* load and replace datablocks base funcs moved to utils.py

* restore files for non blend loaders

* rename datapath to datacol

* fix class inheriting

* fix rebase conflict

* clean code

* cleaning changes

* rename to load_library_as_container, without '_' as not private function

* Update openpype/hosts/blender/api/utils.py

Co-authored-by: Sharkitty <81646000+Sharkitty@users.noreply.github.com>

---------

Co-authored-by: Kaa Maurice <kaalium@gmail.com>
Co-authored-by: Sharkitty <81646000+Sharkitty@users.noreply.github.com>
if hasattr(old_datablock, "pose") and old_datablock.pose:
for bone in old_datablock.pose.bones:
if new_datablock.pose:
if new_bone := new_datablock.pose.bones.get(bone.name):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing whitespace after ':'
multiple statements on one line (colon)
whitespace before ':'

old_datablocks, key=lambda d: 1 if d.library else 0
):
# Match new datablock by name
if new_datablock := next(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing whitespace after ':'
multiple statements on one line (colon)
whitespace before ':'

for outliner_datablock in get_root_datablocks(
old_datablocks, BL_OUTLINER_TYPES
):
if parent_collection := get_parent_collection(outliner_datablock):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing whitespace after ':'
multiple statements on one line (colon)
whitespace before ':'

@mkolar
Copy link
Member

mkolar commented Jan 25, 2024

@kaamaurice we're closing this PR as it is now stale. If there is any functionality from this you'd like to contribute again, feel free to re-open this again, targeting AYON

@mkolar mkolar closed this Jan 25, 2024
@ynbot ynbot added this to the next-patch milestone Jan 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community contribution host: Blender module: Sitesync size/XXL Denotes a PR changes 2500+ lines, ignoring general files type: documentation type: enhancement Enhancements to existing functionality type: feature Larger, user affecting changes and completely new things
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Blender: 3.1.0 Support