Chore: Wrapper for click proposal

[iLLiCiTiT]

Changelog Description

This is a proposal how to resolve issues with click python module. Issue #5921 reported that in Houdini 20+ is our click clashing with click in houdini, where is expected higher version. We can't update our version to support older pythons (NOTE older Python 3).

Possible solutions

1. Use this wrapper (in this PR) that mimics click approach how to define commands and their options. This gives a little bit more options to enhance cli callbacks.
2. Move cli definition out of the addon main file and import it only when needed.

def cli(self, click_group):
    from .cli import cli_main
    click_group.add_command(cli_main)

This PR solution

Created wrapper to mimic click behavior, with small enhancement to be able to use chain function calls which simplify registering methods as cli functions, that might be helpfull in cli commands that need to create ModulesManage. Used the wrapper in ftrack addon and it works...

Now the question is if we should use option n1, option n2 or we should look for option n3.

Note: This PR does not resolve the issue fully, we would still need to remove click from dependencies and move it next to pyside.

[cuID:OP-7416]

[BigRoy]

Hmm - I wouldn't push this direction. We're now basically maintaining our own vendorized copy of click via this - which is something we should rather not touch if not absolutely necessary.

Honestly if this is the hack we'd allow I'd say that maybe we should put a click.py replacement namespace package on path that just registers the package as if it was the original package and then does:

click.py:

try:
    import click
except ImportError as exc:
    # Import our own vendorized version, e.g. adding it on path
    import_our_click()

I recall there being a feature where you could do that so that that file acts as if it was whatever it imported, it can act like click you imported exactly. That way, if click is available from the DCC - we use that, otherwise load our own.

I'd consider that less maintenance likely.
Still a hack, but easier to maintain + no other code changes needed?

[iLLiCiTiT]

Hmm - I wouldn't push this direction. We're now basically maintaining our own vendorized copy of click via this - which is something we should rather not touch if not absolutely necessary.

The wrapper is not even 5% of click logic. I don't think this wrapper needs more maintainance, only what could need maintainance in future is convert_to_click function.

I recall there being a feature where you could do that so that that file acts as if it was whatever it imported, it can act like click you imported exactly. That way, if click is available from the DCC - we use that, otherwise load our own.

To be honest, that is more complicated than this wrapper. Also more complicated to understand what is used when. I would rather explicitly name it differently and use always only the one, or use proposed option 2.

Still a hack, but easier to maintain + no other code changes needed?

I'll bet my life, it would be more maintainance than this wrapper.

[BigRoy]

If I need to write CLI code for an addon and I see click I assume it's the click API - if it then doesn't have half its features I'd just be confused.

I feel like if we're differentiating we might as well just put this into openpype.lib as openpype.lib.cli_utils or whatever - the lib file itself could still say it's based on click but then at least there's not the expectancy that it's click itself.

If anyone then wants to instead expose API using click for a custom addon then indeed it's their mess to solve the lib dependencies, etc.

So if what you're saying is just using a thin command line interface that we just put in openpype.lib I think that's fine as long as we make it super clear (e.g. differentiating name) that it's not click.

[BigRoy]

Oh wait, no - this does still import click and use it; but then, how does it solve the dependency? :) How do we make sure that cli still has access to the click that it imports when used?

Are we just saying that now we're not passing on the click dependency to host installs? Like we're not passing on the python package itself?

Is that just still missing from this PR?

[iLLiCiTiT]

Oh wait, no - this does still import click and use it; but then, how does it solve the dependency? :) How do we make sure that cli still has access to the click that it imports when used?

The difference is that it does not need click to define cli commands, groups, options etc. it needs click only when you want to convert it for click. Look at change in ftrack addon, specifically to cli method on the addon, the click is imported only when to_click_object is called. And cli method of an addon is used only in openpype.cli logic. So technically click is not needed to import the addon itself (we don't break DCCs that don't have click available).

With that, we can remove click from pyproject toml dependencies to runtime dependencies, and problem solved...

[BigRoy]

With that, we can remove click from pyproject toml dependencies to runtime dependencies, and problem solved...

Yes, I was missing this part - thanks. Then I suppose the approach is fine with me. Would still love if we can improve on the dependency hell in a better way; and even better, drop all Py2 support too.

[iLLiCiTiT]

and even better, drop all Py2 support too.

We have to keep Python 2 in OpenPype (not in AYON 🙏 ).

[MustafaJafar]

Not sure, if my comment related to this PR or not but here are my 2 cents.

I'm not able to launch Houdini 20 without #5932
so, I merged both this PR and #5932
and Here's what I see in Houdini 20 console on startup.

In essence, the problem with these:

  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python310\lib\site-packages\flask\cli.py", line 15, in <module>
    from click.core import ParameterSource
ImportError: cannot import name 'ParameterSource' from 'click.core' (E:\Ynput\OpenPype\.venv\lib\site-packages\click\core.py)


  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\Image.py", line 103, in <module>
    from . import _imaging as core
ImportError: cannot import name '_imaging' from 'PIL' (E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\__init__.py)

Full log:

Could not initialize the help server:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\houdinihelp\api.py", line 126, in initialize
    bookish_app = get_houdini_app(use_houdini_path=bool(hou))
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\houdinihelp\server.py", line 25, in get_houdini_app
    from bookish import flaskapp, flasksupport
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\bookish\flaskapp.py", line 37, in <module>
    import flask
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python310\lib\site-packages\flask\__init__.py", line 2, in <module>
    from .app import Flask as Flask
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python310\lib\site-packages\flask\app.py", line 34, in <module>
    from . import cli
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python310\lib\site-packages\flask\cli.py", line 15, in <module>
    from click.core import ParameterSource
ImportError: cannot import name 'ParameterSource' from 'click.core' (E:\Ynput\OpenPype\.venv\lib\site-packages\click\core.py)

Nothing to change, Houdini vars are already up to date.
= = = Resources registration = = =
ERROR: C:/PROGRA~1/SIDEEF~1/HOUDIN~1.506/packages/apex/viewer_states/apexanimate.py
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\viewerstate\utils.py", line 967, in register_pystate
    mod = importlib.import_module(module_name)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python310\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/packages/apex/viewer_states\apexanimate.py", line 15, in <module>
    from apex.ui.selectionmanager import model as ssm
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/packages/apex/python3.10libs\apex\ui\selectionmanager\model.py", line 9, in <module>
    from poselib.itemlibrary import IdentityProxyModel as plIdentityProxyModel
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\poselib\itemlibrary.py", line 16, in <module>
    from . import plglobals
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\poselib\plglobals.py", line 23, in <module>
    from PIL import Image
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\Image.py", line 103, in <module>
    from . import _imaging as core
ImportError: cannot import name '_imaging' from 'PIL' (E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\__init__.py)

ERROR: C:/PROGRA~1/SIDEEF~1/HOUDIN~1.506/packages/apex/viewer_states/apexposestate.py
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\viewerstate\utils.py", line 967, in register_pystate
    mod = importlib.import_module(module_name)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.506\python310\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/packages/apex/viewer_states\apexposestate.py", line 7, in <module>
    import apex.ui.selectionmanager.model as ssm
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/packages/apex/python3.10libs\apex\ui\selectionmanager\model.py", line 9, in <module>
    from poselib.itemlibrary import IdentityProxyModel as plIdentityProxyModel
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\poselib\itemlibrary.py", line 16, in <module>
    from . import plglobals
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.506/houdini/python3.10libs\poselib\plglobals.py", line 23, in <module>
    from PIL import Image
  File "C:\Program Files\Side Effects Software\Houdini 20.0.506\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\Image.py", line 103, in <module>
    from . import _imaging as core
ImportError: cannot import name '_imaging' from 'PIL' (E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\__init__.py)

[iLLiCiTiT]

This PR is not fixing the issue, it is first step to be able to fix it...

[BigRoy]

Actually - it's worse. As Mustafa's logs show there is also a conflict with the PIL library from OpenPype.

[iLLiCiTiT]

Actually - it's worse. As Mustafa's logs show there is also a conflict with the PIL library from OpenPype.

One at a time, please add it to the issue, not related to this PR.

[BigRoy]

Actually - it's worse. As Mustafa's logs show there is also a conflict with the PIL library from OpenPype.

Side note: Just heard from @fabiaserra that the PIL lib is only used for the TVPaint and Photoshop integration. For their case they'd just removed it as a required dependency to OpenPype and it fixed the PIL thing for them.

---

@fabiaserra could you check out this proposal and provide your own feedback? Then maybe @iLLiCiTiT can finish the PR so it actually fixes the issue, because last he said:

This PR is not fixing the issue, it is first step to be able to fix it...

I suppose the bug is still there and would need more input to continue.

[iLLiCiTiT]

Just a note, we're AYON focused now, and in AYON is Pillow already not added to PYTHONPATH, so it is resolved for us. Change of dependencies would require to bump minor version of OpenPype.

This PR needs final decisions: Where to put the wrapper, and how to name it.

• Current location is in root of openpype. I think it can be moved to openpype.lib or openpype.modules.
• Current name is click_wrap. Here I didn't find better name.

[fabiaserra]

Just a note, we're AYON focused now, and in AYON is Pillow already not added to PYTHONPATH, so it is resolved for us. Change of dependencies would require to bump minor version of OpenPype.

This PR needs final decisions: Where to put the wrapper, and how to name it.

• Current location is in root of openpype. I think it can be moved to openpype.lib or openpype.modules.
• Current name is click_wrap. Here I didn't find better name.

Isn't it here https://github.com/ynput/OpenPype/blob/develop/pyproject.toml#L50 ? that will be picked up by the dependency package for Ayon as well? I removed it from there and confirmed that I no longer get the Pillow error running latest OP minor (haven't tested in Ayon because I was lazy to add H20 and update the dependency package but I'd assume it's the same). I now only get the click error reported here.

As for my preferred solution, I think here's where rez would provide the proper solution to have independent environments per DCC but given the lack of it, I'd try prio the python packages from the DCC when there's conflicts like @BigRoy was suggesting than creating extra wrappers and more technical debt because this is not only going to happen with click but numerous other packages... In fact I had numpy installed in OP as a dependency and that was another conflict that I was getting so I now need to come up with some hack to roll that only for Hiero. Is there a solution in Ayon to set packages only for specific DCCs?

[MustafaJafar]

In fact I had numpy installed in OP as a dependency and that was another conflict that I was getting so I now need to come up with some hack to roll that only for Hiero. Is there a solution in Ayon to set packages only for specific DCCs?

Is relying on environment variables enough?

[MustafaJafar]

I think the target of this PR should be openpype.

---

I opened Houdini 20 from develop branch and faced no issues in Ayon.
In openpype I faced the same error mentioned in this PR #5928 (comment).

---

then I checked out this PR and didn't face any errors in Ayon.
but, in openpype I faced the following error.

Could not initialize the help server:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\houdinihelp\api.py", line 126, in initialize
    bookish_app = get_houdini_app(use_houdini_path=bool(hou))
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\houdinihelp\server.py", line 25, in get_houdini_app
    from bookish import flaskapp, flasksupport
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\bookish\flaskapp.py", line 37, in <module>
    import flask
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.502\python310\lib\site-packages\flask\__init__.py", line 2, in <module>
    from .app import Flask as Flask
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.502\python310\lib\site-packages\flask\app.py", line 34, in <module>
    from . import cli
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.502\python310\lib\site-packages\flask\cli.py", line 15, in <module>
    from click.core import ParameterSource
ImportError: cannot import name 'ParameterSource' from 'click.core' (E:\Ynput\OpenPype\.venv\lib\site-packages\click\core.py)

Nothing to change, Houdini vars are already up to date.
= = = Resources registration = = =
ERROR: C:/PROGRA~1/SIDEEF~1/HOUDIN~1.502/packages/apex/viewer_states/apexanimate.py
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\viewerstate\utils.py", line 967, in register_pystate
    mod = importlib.import_module(module_name)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.502\python310\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/packages/apex/viewer_states\apexanimate.py", line 15, in <module>
    from apex.ui.selectionmanager import model as ssm
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/packages/apex/python3.10libs\apex\ui\selectionmanager\model.py", line 9, in <module>
    from poselib.itemlibrary import IdentityProxyModel as plIdentityProxyModel
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\poselib\itemlibrary.py", line 16, in <module>
    from . import plglobals
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\poselib\plglobals.py", line 23, in <module>
    from PIL import Image
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\Image.py", line 103, in <module>
    from . import _imaging as core
ImportError: cannot import name '_imaging' from 'PIL' (E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\__init__.py)

ERROR: C:/PROGRA~1/SIDEEF~1/HOUDIN~1.502/packages/apex/viewer_states/apexposestate.py
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\viewerstate\utils.py", line 967, in register_pystate
    mod = importlib.import_module(module_name)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.502\python310\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/packages/apex/viewer_states\apexposestate.py", line 7, in <module>
    import apex.ui.selectionmanager.model as ssm
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/packages/apex/python3.10libs\apex\ui\selectionmanager\model.py", line 9, in <module>
    from poselib.itemlibrary import IdentityProxyModel as plIdentityProxyModel
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\poselib\itemlibrary.py", line 16, in <module>
    from . import plglobals
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "C:\PROGRA~1/SIDEEF~1/HOUDIN~1.502/houdini/python3.10libs\poselib\plglobals.py", line 23, in <module>
    from PIL import Image
  File "C:\Program Files\Side Effects Software\Houdini 20.0.502\python310\lib\site-packages-forced\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\Image.py", line 103, in <module>
    from . import _imaging as core
ImportError: cannot import name '_imaging' from 'PIL' (E:\Ynput\OpenPype\.venv\lib\site-packages\PIL\__init__.py)

[iLLiCiTiT]

First of all, target of this PR is to fix click import issues inside DCCs.

Isn't it here https://github.com/ynput/OpenPype/blob/develop/pyproject.toml#L50 ? that will be picked up by the dependency package for Ayon as well?

No, it is not. That is only OpenPype build related. OpenPype addon has different pyproject.toml.

I think the target of this PR should be openpype.

The target is AYON and could be used in OpenPype too (with another PR). This PR is to avoid import of click in addons which have cli interface, because click import breaks inside DCCs which have incompatible python, in both AYON and OpenPype.

The PIL is NOT handled by this PR, and won't be, please stop to mention it here as it's not relevant. First crash in your traceback in because of click, that is what this PR is about.

I opened Houdini 20 from develop branch and faced no issues in Ayon.

Did you have enabled an addon that is using click, like ftrack or kitsu, when you've tried?

As for my preferred solution, I think here's where rez would provide the proper solution to have independent environments per DCC but given the lack of it, I'd try prio the python packages from the DCC when there's conflicts like @BigRoy was suggesting than creating extra wrappers and more technical debt because this is not only going to happen with click but numerous other packages...

That's a long run task. This PR is solution for current issue without reimplementing whole applications launch logic and all addons and figuring out how to deploy rez to workstations and many other issues that must be solved.

[fabiaserra]

No, it is not. That is only OpenPype build related. OpenPype addon has different pyproject.toml.

Oh nice, I hadn't seen that other pyproject, good to know

That's a long run task. This PR is solution for current issue without reimplementing whole applications launch logic and all addons and figuring out how to deploy rez to workstations and many other issues that must be solved.

Yeah I understand, that's why I think as a first stop solution it's better if we just hack it by prioritizing the DCC click than creating the wrapper, both should be short-term solutions until we come up with a better plan

[MustafaJafar]

First crash in your traceback in because of click, that is what this PR is about.

Thanks for the explanation.

Did you have enabled an addon that is using click, like ftrack or kitsu

I have Kitsu enabled in both Openpype standalone and Ayon server.

when you've tried?

Today.. now.
I'm running Ayon dev, Openpype version 3.18.3-nightly.1 and Ayon launcher 1.0.0-rc.3

---

I don't know why the click error doesn't show up any more in H20 Ayon Mode.

Here's a screenshot from develop branch

Another Screenshot from this PR

[iLLiCiTiT]

Even if we prepend the Houdini python libs to the PYTHONPATH?

We can't determine the path in reliable way. We have clients which use custom launch scripts instead of houdini executable in application settings.

[fabiaserra]

Even if we prepend the Houdini python libs to the PYTHONPATH?

We can't determine the path in reliable way. We have clients which use custom launch scripts instead of houdini executable in application settings.

is $HHP resolved too late?

[BigRoy]

is $HHP resolved too late?

You'd need to set the PYTHONPATH before the actual houdini launch to ensure anything houdini does, accesses the right entry.
I assume this $HHP is a houdini var and thus would be too late.

Then there might also be a case where $HHP gets set DIRECTLY by Houdini and before the call to click but you'd still need to somehow get Python access directly after Houdini sets that var - nothing is ensuring if you do it e.g. in pythonlibs or whatever startup procedure that those run before Houdini does any call to click for example. (And it's not about an actual call to the functions of click; but when it imports it.)

[fabiaserra]

You'd need to set the PYTHONPATH before the actual houdini launch to ensure anything houdini does, accesses the right entry. I assume this $HHP is a houdini var and thus would be too late.

Then there might also be a case where $HHP gets set DIRECTLY by Houdini and before the call to click but you'd still need to somehow get Python access directly after Houdini sets that var - nothing is ensuring if you do it e.g. in pythonlibs or whatever startup procedure that those run before Houdini does any call to click for example. (And it's not about an actual call to the functions of click; but when it imports it.)

Yeah makes sense. On the interim I was able to hack it in our env on the H20 entry, pretty straightforward:

[fabiaserra]

You'd need to set the PYTHONPATH before the actual houdini launch to ensure anything houdini does, accesses the right entry. I assume this $HHP is a houdini var and thus would be too late.
Then there might also be a case where $HHP gets set DIRECTLY by Houdini and before the call to click but you'd still need to somehow get Python access directly after Houdini sets that var - nothing is ensuring if you do it e.g. in pythonlibs or whatever startup procedure that those run before Houdini does any call to click for example. (And it's not about an actual call to the functions of click; but when it imports it.)

Yeah makes sense. On the interim I was able to hack it in our env on the H20 entry, pretty straightforward:

As long as the bug is documented and the different possible solutions on how to hack it... I think it would be perfectly fine as a temporary solution until we can tackle rez or a better system

[ynbot]

Task linked: OP-7416 Bug: Houdini 20+ errors on launch due to click dependency

[BigRoy]

Yeah makes sense. On the interim I was able to hack it in our env on the H20 entry, pretty straightforward:

For what it's worth - we ended up doing the same thing as it felt most reliable since there were multiple conflicts we had to manage where click was only one of them.

[fabiaserra]

For what it's worth - we ended up doing the same thing as it felt most reliable since there were multiple conflicts we had to manage where click was only one of them.

exactly, adding the wrapper is just a tiny bandaid that doesn't fix the actual root of the problem...

[antirotor]

tested it a little bit,but the code looks fine