Use SG API key for creating session and remove login functionality #58

fabiaserra · 2023-12-01T16:46:24Z

This removes the need for logging in to SG through the Tray and just make use of the API key + sudo_as_login to integrate into SG. We are already doing extra validations of the user if it exists in SG to also need them to type yet another password... which in some studios might have the 2FA authentication and make this even more complicated. Unless we can enable SSO authentication across Ayon and SG I don't see any value for us on enabling this. On our use case I make use of os.getenv("USER") instead of AYON_SG_USER but on this case we might need them to be able to type what's their SG user if for any reason that differs from the Ayon username.

On the other hand, I bring here some of the changes that we have done on the OP shotgrid module to make it work better for our needs:

Avoid creating SG versions/published files unless there's actually some data to update
Populate some extra fields on the SG versions coming from Ayon
Remove version from PublishedFiles so they are grouped together
Change the name of the SG version code to be specific to the subset (although as noted on the code already this should be coming from the settings instead)
Allow 'admin' roles to publish to any SG project

…y module to login

… representation

… together and set basic published file type functionality

…te some other relevant fields on the version

matteoveglia · 2023-12-01T17:34:30Z

Love these changes Fabia, I agree with the removal of an SG password and using the server SG API key for auth.
Part of that, as a stretch goal of course, would be to be able to sync users from SG so we can ensure usernames are the same; my only concern would be that right now as I'd want to make sure we submit versions to SG as the artist

fabiaserra · 2023-12-01T18:34:17Z

Love these changes Fabia, I agree with the removal of an SG password and using the server SG API key for auth. Part of that, as a stretch goal of course, would be to be able to sync users from SG so we can ensure usernames are the same; my only concern would be that right now as I'd want to make sure we submit versions to SG as the artist

yeah ideally with the SSO feature of Ayon https://community.ynput.io/t/sso-active-directory-integration/259/12 this would be consolidated. However, this is not an issue as long as whatever you pass as shotgridUser (right now coming by the AYON_SG_USER but can be anything else) exists in your SG. If it doesn't exist the publish will fail but if it's successfuly the versions will always be submitted as the artist

fabiaserra · 2023-12-01T18:39:11Z

client/ayon_shotgrid/plugins/publish/integrate_shotgrid_publish.py

@@ -141,3 +168,43 @@ def process(self, instance):
                )
            instance.data["shotgridPublishedFile"] = sg_published_file

+    def _find_published_file_type(self, instance, filepath, representation):


this function is a pretty simplistic way to set some types on the PublishedFiles of SG and it relies on setting these on your SG instance:

Yeah this is great too, how would you select what SH publish type (representation) you're publishing too?
Do I need to make sure I name the SG and AYON representations exactly the same?

atm I'm not making use of the Ayon family or representation name, I'm purely basing it off the extension of the file type and you don't have to do anything extra, just that this function detects all the file types you want to track

Summary field must be a dict. Human readable description of an event should go to description. Passing strings to summary causes 419 errors.

Minkiu · 2023-12-08T14:47:00Z

client/ayon_shotgrid/addon.py

-
-        if not sg_username or not sg_password:
-            return None
+        sg_username = os.getenv("AYON_SG_USERNAME")


Where do you normally define this environment variable, and what should happen if it's not set empty?

on my case I'm actually just using $USER as all our artists machines have that defined so we don't need any extra user input to define their login for SG but that's why I left it open to discuss. We could potentially expose an option on the settings where admins can choose any existing env var to use as SG login and/or optionally expose the login menu so each user can set it themselves on the registry

Minkiu · 2023-12-08T16:38:59Z

Hey! Thanks for the contribution :D

I just got around trying this, and I'm getting an error when trying to publish:

Setting AYON_SG_USERNAME=admin, ended up in:

ERROR:pyblish.CollectShotgridSession:Failed to connect to Shotgrid.
Traceback (most recent call last):
  File "/home/minkiu/.local/share/AYON/addons/shotgrid_0.2.10/ayon_shotgrid/plugins/publish/collect_shotgrid_session.py", line 30, in process
    sg_session = shotgrid_module.create_shotgrid_session()
  File "/home/minkiu/.local/share/AYON/addons/shotgrid_0.2.10/ayon_shotgrid/addon.py", line 46, in create_shotgrid_session
    return credentials.create_sg_session(
  File "/home/minkiu/.local/share/AYON/addons/shotgrid_0.2.10/ayon_shotgrid/lib/credentials.py", line 29, in create_sg_session
    session.preferences_read()
  File "/home/minkiu/.local/share/AYON/dependency_packages/ayon_2312081315_linux.zip/dependencies/shotgun_api3/shotgun.py", line 3262, in preferences_read
    return self._call_rpc("preferences_read", {"prefs": prefs})
  File "/home/minkiu/.local/share/AYON/dependency_packages/ayon_2312081315_linux.zip/dependencies/shotgun_api3/shotgun.py", line 3423, in _call_rpc
    self._response_errors(response)
  File "/home/minkiu/.local/share/AYON/dependency_packages/ayon_2312081315_linux.zip/dependencies/shotgun_api3/shotgun.py", line 3722, in _response_errors
    raise AuthenticationFault(sg_response.get("message", "Unknown Authentication Error"))
shotgun_api3.shotgun.AuthenticationFault: Cannot 'sudo' - unknown or retired user: 'admin'
ERROR:pyblish.plugin:Traceback (most recent call last):
  File "/home/minkiu/.local/share/AYON/addons/shotgrid_0.2.10/ayon_shotgrid/plugins/publish/collect_shotgrid_session.py", line 30, in process
    sg_session = shotgrid_module.create_shotgrid_session()
  File "/home/minkiu/.local/share/AYON/addons/shotgrid_0.2.10/ayon_shotgrid/addon.py", line 46, in create_shotgrid_session
    return credentials.create_sg_session(
  File "/home/minkiu/.local/share/AYON/addons/shotgrid_0.2.10/ayon_shotgrid/lib/credentials.py", line 29, in create_sg_session
    session.preferences_read()
  File "/home/minkiu/.local/share/AYON/dependency_packages/ayon_2312081315_linux.zip/dependencies/shotgun_api3/shotgun.py", line 3262, in preferences_read
    return self._call_rpc("preferences_read", {"prefs": prefs})
  File "/home/minkiu/.local/share/AYON/dependency_packages/ayon_2312081315_linux.zip/dependencies/shotgun_api3/shotgun.py", line 3423, in _call_rpc
    self._response_errors(response)
  File "/home/minkiu/.local/share/AYON/dependency_packages/ayon_2312081315_linux.zip/dependencies/shotgun_api3/shotgun.py", line 3722, in _response_errors
    raise AuthenticationFault(sg_response.get("message", "Unknown Authentication Error"))
shotgun_api3.shotgun.AuthenticationFault: Cannot 'sudo' - unknown or retired user: 'admin'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/minkiu/.local/share/AYON/dependency_packages/ayon_2312081315_linux.zip/dependencies/pyblish/plugin.py", line 527, in __explicit_process
    runner(*args)
  File "/home/minkiu/.local/share/AYON/addons/shotgrid_0.2.10/ayon_shotgrid/plugins/publish/collect_shotgrid_session.py", line 34, in process
    raise KnownPublishError(
openpype.pipeline.publish.publish_plugins.KnownPublishError: Could not connect to Shotgrid https://ynput.shotgrid.autodesk.com with user admin.
Traceback (most recent call last):
  File "/home/minkiu/.local/share/AYON/addons/shotgrid_0.2.10/ayon_shotgrid/plugins/publish/collect_shotgrid_session.py", line 30, in process
    sg_session = shotgrid_module.create_shotgrid_session()
  File "/home/minkiu/.local/share/AYON/addons/shotgrid_0.2.10/ayon_shotgrid/addon.py", line 46, in create_shotgrid_session
    return credentials.create_sg_session(
  File "/home/minkiu/.local/share/AYON/addons/shotgrid_0.2.10/ayon_shotgrid/lib/credentials.py", line 29, in create_sg_session
    session.preferences_read()
  File "/home/minkiu/.local/share/AYON/dependency_packages/ayon_2312081315_linux.zip/dependencies/shotgun_api3/shotgun.py", line 3262, in preferences_read
    return self._call_rpc("preferences_read", {"prefs": prefs})
  File "/home/minkiu/.local/share/AYON/dependency_packages/ayon_2312081315_linux.zip/dependencies/shotgun_api3/shotgun.py", line 3423, in _call_rpc
    self._response_errors(response)
  File "/home/minkiu/.local/share/AYON/dependency_packages/ayon_2312081315_linux.zip/dependencies/shotgun_api3/shotgun.py", line 3722, in _response_errors
    raise AuthenticationFault(sg_response.get("message", "Unknown Authentication Error"))
shotgun_api3.shotgun.AuthenticationFault: Cannot 'sudo' - unknown or retired user: 'admin'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/minkiu/.local/share/AYON/dependency_packages/ayon_2312081315_linux.zip/dependencies/pyblish/plugin.py", line 527, in __explicit_process
    runner(*args)
  File "/home/minkiu/.local/share/AYON/addons/shotgrid_0.2.10/ayon_shotgrid/plugins/publish/collect_shotgrid_session.py", line 34, in process
    raise KnownPublishError(
openpype.pipeline.publish.publish_plugins.KnownPublishError: Could not connect to Shotgrid https://ynput.shotgrid.autodesk.com with user admin.

And when setting it to AYON_SG_USERNAME=admin@ynput.io:

ERROR:pyblish.plugin:Traceback (most recent call last):
  File "/home/minkiu/.local/share/AYON/dependency_packages/ayon_2312081315_linux.zip/dependencies/pyblish/plugin.py", line 527, in __explicit_process
    runner(*args)
  File "/home/minkiu/.local/share/AYON/addons/shotgrid_0.2.10/ayon_shotgrid/plugins/publish/validate_shotgrid_user.py", line 33, in process
    admin = sg_user["permission_rule_set"]["name"] == "Admin"
TypeError: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/home/minkiu/.local/share/AYON/dependency_packages/ayon_2312081315_linux.zip/dependencies/pyblish/plugin.py", line 527, in __explicit_process
    runner(*args)
  File "/home/minkiu/.local/share/AYON/addons/shotgrid_0.2.10/ayon_shotgrid/plugins/publish/validate_shotgrid_user.py", line 33, in process
    admin = sg_user["permission_rule_set"]["name"] == "Admin"
TypeError: 'NoneType' object is not subscriptable

Unsure if it's cause it's "Admin", but at the moment we don't have more seats in SG.

client/ayon_shotgrid/plugins/publish/validate_shotgrid_user.py

fabiaserra · 2023-12-11T14:55:10Z

Hey! Thanks for the contribution :D

I just got around trying this, and I'm getting an error when trying to publish:

Setting AYON_SG_USERNAME=admin, ended up in:

And are you sure you have an admin user in your SG server?

Minkiu · 2024-01-02T17:51:06Z

Hey! Thanks for the contribution :D
I just got around trying this, and I'm getting an error when trying to publish:
Setting AYON_SG_USERNAME=admin, ended up in:

And are you sure you have an admin user in your SG server?

Sorry for the late reply, admin is the only user I have, but after checking the login is the same as the email, so that's probably why!

Minkiu · 2024-01-12T16:15:12Z

Thanks for your contribution! Merging.

m-u-r-p-h-y · 2024-03-11T16:53:14Z

Not sure why the login functionality was removed. This is not working and breaking the addon completely . . .

fabiaserra · 2024-03-11T16:56:08Z

Not sure why the login functionality was removed. This is not working and breaking the addon completely . . .

it works for us while the login functionality didn't. You just need to provide a way to fill up the user env var, either automatically through the Ayon login logic or on the general environments. Or if you'd rather bring that login dialog back, it should be just another optional way to provide the user login

m-u-r-p-h-y · 2024-03-11T17:19:09Z

there is no mention of any user env in the Readme file.

good to hear it is working for you, but everyone else needs to dig through the code to figure out how to log in to Shotgrid and why it is not working.

fabiaserra · 2024-03-11T17:52:57Z

there is no mention of any user env in the Readme file.

good to hear it is working for you, but everyone else needs to dig through the code to figure out how to log in to Shotgrid and why it is not working.

really the same counter argument with the prior iteration with the login -> no readme and not working haha I think we just need to put some more hands on this codebase and give it some love for the rest, as it is it's not in a state of plug and play and have fun, it's never been for SG <-> OP/Ayon

jakubjezek001 · 2024-03-29T13:03:55Z

I am considering the feasibility of reinstating the original functionality that was removed in this pull request, within the context of the addon studio settings. My recommendation would be to have both options available, but with the environment variable workflow being the default. However, if anyone wishes to utilize the logging functionality from AY Desktop tray menu, they could easily activate it.

m-u-r-p-h-y · 2024-03-31T12:13:55Z

The question is what is the SG user good for actually, if all interaction is happening using sg_script API key. Why is it different from ftrack integration? How do we deal with different user permissions in Ayon? Does it mean a user with limited permissions in Ayon has unlimited access to SG? etc..

fabiaserra · 2024-04-01T10:01:28Z

The question is what is the SG user good for actually, if all interaction is happening using sg_script API key. Why is it different from ftrack integration? How do we deal with different user permissions in Ayon? Does it mean a user with limited permissions in Ayon has unlimited access to SG? etc..

How is the ftrack integration?

A user with limited permissions in Ayon won't really have unlimited access to SG as the frontend is blocking him from accessing any of the projects he doesn't have access. Of course a technical artist could always figure out the API to grab the session and do queries or publishes to any project with the API key but I wouldn't worry about that, it's really the same paradigm as in Ayon once you know about the API key

Fabia Serra Arrizabalaga added 5 commits December 1, 2023 11:35

client Use API key to create session and remove need for having tra…

44a4ad3

…y module to login

client Only create PublishedFiles if 'shotgridreview' tag exists on…

3ab8b6a

… representation

client Strip version number from PublishedFiles so they are grouped…

4f19aa9

… together and set basic published file type functionality

client Only create SG versions if there's data to update and popula…

0dccce7

…te some other relevant fields on the version

client Allow 'admin' roles to publish on any SG project

6650acf

fabiaserra commented Dec 1, 2023

View reviewed changes

Fabia Serra Arrizabalaga and others added 3 commits December 6, 2023 10:02

services Pass custom project code field to other functions that use it

fc64c00

client Fix typo referencing sg_session

4ef4407

services.processor fix incorrect usage of event.summary field

8cea5ae

Summary field must be a dict. Human readable description of an event should go to description. Passing strings to summary causes 419 errors.

Minkiu reviewed Dec 8, 2023

View reviewed changes

client/ayon_shotgrid/plugins/publish/validate_shotgrid_user.py Show resolved Hide resolved

Merge branch 'develop' into feature/use_api_key_for_session

2acebd3

Minkiu force-pushed the develop branch from 0f92cd7 to f20ceb1 Compare December 13, 2023 16:50

fabiaserra and others added 2 commits January 11, 2024 12:04

Merge branch 'develop' into feature/use_api_key_for_session

2742967

client Update SG user validation to work with Admin permission

9bc1af9

Minkiu approved these changes Jan 12, 2024

View reviewed changes

Fabia Serra Arrizabalaga added 2 commits January 12, 2024 09:55

client Fix typo referencing sg_session

90537a7

client Remove bad merge conflict

9b31909

Minkiu merged commit b074ee3 into ynput:develop Jan 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use SG API key for creating session and remove login functionality #58

Use SG API key for creating session and remove login functionality #58

fabiaserra commented Dec 1, 2023

matteoveglia commented Dec 1, 2023

fabiaserra commented Dec 1, 2023

fabiaserra Dec 1, 2023

matteoveglia Dec 1, 2023

fabiaserra Dec 1, 2023

Minkiu Dec 8, 2023

fabiaserra Dec 11, 2023

Minkiu commented Dec 8, 2023

fabiaserra commented Dec 11, 2023

Minkiu commented Jan 2, 2024

Minkiu commented Jan 12, 2024

m-u-r-p-h-y commented Mar 11, 2024

fabiaserra commented Mar 11, 2024

m-u-r-p-h-y commented Mar 11, 2024

fabiaserra commented Mar 11, 2024

jakubjezek001 commented Mar 29, 2024

m-u-r-p-h-y commented Mar 31, 2024

fabiaserra commented Apr 1, 2024

Use SG API key for creating session and remove login functionality #58

Use SG API key for creating session and remove login functionality #58

Conversation

fabiaserra commented Dec 1, 2023

matteoveglia commented Dec 1, 2023

fabiaserra commented Dec 1, 2023

fabiaserra Dec 1, 2023

Choose a reason for hiding this comment

matteoveglia Dec 1, 2023

Choose a reason for hiding this comment

fabiaserra Dec 1, 2023

Choose a reason for hiding this comment

Minkiu Dec 8, 2023

Choose a reason for hiding this comment

fabiaserra Dec 11, 2023

Choose a reason for hiding this comment

Minkiu commented Dec 8, 2023

fabiaserra commented Dec 11, 2023

Minkiu commented Jan 2, 2024

Minkiu commented Jan 12, 2024

m-u-r-p-h-y commented Mar 11, 2024

fabiaserra commented Mar 11, 2024

m-u-r-p-h-y commented Mar 11, 2024

fabiaserra commented Mar 11, 2024

jakubjezek001 commented Mar 29, 2024

m-u-r-p-h-y commented Mar 31, 2024

fabiaserra commented Apr 1, 2024