Refactor avalon.tools to avoid cross-referencing imports between tools #440

BigRoy · 2019-09-04T12:03:27Z

Note: This is a big PR as it refactors locations of models, views, delegates and functions. It's this big because unfortunately it needs to be.

Problems

Different tools cross-referenced each other with imports from the other tools.
The tree model's item is called Node, which is wrong terminology as Qt refers to QAbstractItemModel where it is an Item
The family and group configs that are "refreshed" in loader.lib are actually caches reused across multiple tools. This is unclear from the variable and function naming. (This "global state" is a problem of its own, but this PR will not try to solve that as it would require even more changes.) It only solves the cross-referencing across tools.
Some variables are mixed case like lowerCamelCase where they should be snake_case as per Add CONVENTION.md #437
An AssetView class exists which purely exists for one-time use and only sets 3 small view tweaks - it's redundant complexity.
The TreeModel defines an attribute called COLUMNS which should be lowercase columns.
There is an unused model ExactMatchesFilterProxyModel which is redundant lines of code.

What's changed?

All shared tool models, views, delegates, widgets and functions have been respectively refactored to avalon.tools.models, avalon.tools.views, avalon.tools.delegates, avalon.tools.widgets and avalon.tools.lib
The Node is refactored to Item, including variables node and alike that I could find are refactored to item.
This is still a global state, however the variables and functions now are clearly referring to it as a cache and simplified to the only way they are used. It's up to a following PR to move the caching to inside a model as opposed to a global state.
Refactored to snake_case.
The AssetView class has been removed and the related lines of code can now directly be found in the AssetWidget.
TreeModel.COLUMNS refactored to TreeModel.Columns
Remove ExactMatchesFilterProxyModel.

This is a work in progress PR and has not been tested yet. However I'm opening this to kickstart the discussion about any faulty choices I'm making. Plus, since it's a lengthy PR it gives others time to go through what is being changed.

I'll clearly state in a comment when I have been able to thoroughly test changes and feel it's in a good state for others to test too.

This PR acts as a replacement to #414 trying to do some similar things yet now along the new Contribution guidelines. Additionally it tries to describe the problems that we faced before and how they have been resolved.

- Remove AssetView class which solely set 3 properties on the TreeView - Refactor `Node` to `Item` as it is for QAbstractItemModels - Refactor some variables to `snake_case` as opposed to `lowerCamelCase` - Add some todo notes (notes to self for cleanup)

- Plus clarify that the group/family config data is a cache.

avalon/tools/widgets.py

avalon/tools/views.py

avalon/tools/models.py

avalon/tools/loader/model.py

avalon/tools/delegates.py

avalon/tools/projectmanager/model.py

davidlatwe

Looks like a good start !

davidlatwe · 2019-09-04T13:57:08Z

avalon/tools/widgets.py

+    preserve_expanded_rows,
+    preserve_selection,
+    iter_model_rows
+)


I believe that we are not suppose to directly import functions and classes now ? :P

Thanks David. Is that based on the e3 example in CONTRIBUTING.md? I interpreted it as not importing as a shorter name and not necessarily removing all functions imports. Especially now that there's a lib at multiple levels (avalon-wide + tools-wide + per tool) seeing lib. wouldn't tell much.

But probably @mottosso can best describe his intentions. I'm fine with both, do note however that if this needs to be adapted that it accounts for more things, e.g. see these lines.

I did now see there's an import there that states:

from ..vendor import qtawesome as awesome

Which I believe should be:

from ..vendor import qtawesome

That irritates me for a long time I think it should be from ..vendor import qtawesome

from ..vendor import qtawesome as awesome

This has now been removed and resolved with this commit.

Is that based on the e3 example in CONTRIBUTING.md?

I was actually referencing from A8 in CONTRIBUTING.md, which aim to preserve the namespace :P

Especially now that there's a lib at multiple levels (avalon-wide + tools-wide + per tool) seeing lib. wouldn't tell much.

Yeah.. maybe this will work ? Say we are in bad luck, used all three levels' lib.py in one module:

from ... import lib as avalon_lib from .. import lib as tool_lib from . import lib

This one's a hard problem.

On the one hand, word and line length obscure code. One the other, namespaces give the reader a hint at what a function or class is coming from.

For some modules, it's obvious.

qtawesome.some_function() # I know you're third-party

For others, not so much.

model = SomeModel() # Third-party, local module, separate root-level module?

Let's try this as a general guideline; preserve the leaf-level module namespace.

from .vendor import qtawesome is Good

from .vendor.qtawesome import icon is Bad

maybe this will work ?

This is another hard problem; on the one hand, we should use lib at various levels, as they are context sensitive. A lib under tools is tools libraries, whereas a lib under maya/ is Maya libraries. And on the other they complicate the above general guideline.

I think this is as elegant of a solution as we'll get, if we are to have at least some guideline for how to deal with it. Preserve the minimum number of namespaces; in this case, we need both tool and lib, if avalon_lib is also present.

Thanks @mottosso - just to be complete, if I understand you correctly the above example to import as tool_lib, etc. when required is considered the correctly approved change. Yes? 👍

BigRoy · 2019-09-04T14:46:39Z

With this last commit I checked out the pipeline features in Maya and ran through:

Switching contexts with ContextManager
Publishing with Pyblish QML
Loading some published assets.
Managing some published assets with scene inventory (removing + updating to specific versions)
Creating some publish instances with the Creator.
Opening the Project Manager, creating an asset and adding tasks.
Work files opening a file and saving a file.
Switching assets with the Switch Asset UI launched from the scene inventory.
Tested out grouping/ungrouping in Loader and whether "Enable Grouping" still worked as expected.

This now all worked so I consider this testable for others.

davidlatwe

Added a few more comments for discussion :)

avalon/tools/delegates.py

avalon/tools/widgets.py

davidlatwe · 2019-09-05T11:42:34Z

Ah, backward compatibility, but do we need to do that for this kind of low level refactoring ?

I have some script/tools that, for example, used projectmanager.model.TreeModel. But since these doesn't count as part of api nor other higher level things, so maybe we won't need to support this ?

BigRoy · 2019-09-05T12:13:46Z

I have some script/tools that, for example, used projectmanager.model.TreeModel. But since these doesn't count as part of api nor other higher level things, so maybe we won't need to support this ?

Exactly, as those have never been exposed to any public facing API these changes are in the realm of "allowed to change". So, those don't need to be backwards compatible and you'll have to refactor in-house code that used it to change along. We had some internal tools referencing things too. ;)

Anyway, anything not exposed in the API is allowed to change without being fully backwards compatible. (e.g. a function name may behave differently, be renamed, moved, etc.). And the recommendation is not to reference those methods, but of course... you can if you really need to at your own risk, like now.

avalon/tools/loader/model.py

mottosso

Ok I've taken a closer look and I think it looks good.

Based on your description, I've deliberately not commented on the actual content or behavior of code (there is some debt in there), and treat this PR as only moving code around. If that's the case, then structurally I think this reads very well.

mottosso · 2019-09-06T07:08:59Z

I suppose that "Approved" should have been "Requesting changed", as I think we should address my comments before merging. I'll let @davidlatwe hit the Merge button for this PR on my behalf. Great work guys.

BigRoy · 2019-09-06T11:02:21Z

This should be ready to go if I understood @mottosso correctly here then there are now no open comments anymore as I refactored the columns to Columns which was the last remark.

I'd say, give it one other test run and feel free to merge @davidlatwe 🚀

Thanks all!

mottosso · 2019-09-07T07:23:09Z

avalon/tools/loader/app.py

-    refresh_family_config,
-    refresh_group_config,
-    get_active_group_config,
+from ..lib import (


No, this should be from .. import lib which you're already doing above, but then not using?

It's being imported three times (!) in three different ways, let's clean this up.

Thanks @mottosso - well spotted! I tried to simplify and clean things up with this commit. Let me know if that suffices.

davidlatwe

Hooray !!
If no other objections, will merge this tomorrow !

davidlatwe · 2019-09-10T06:45:26Z

Ah, I believe we should bump the version to 5.4.0, 👋 @BigRoy :)
I will merge this PR and draft a ~~pre-release~~ release after verison bumped !

BigRoy · 2019-09-10T09:17:57Z

@davidlatwe this hasn't been merged yet. Do you want me to quickly bump version? :)

davidlatwe · 2019-09-10T09:22:38Z

Yes, please do ! It seems that I could not bump it myself into this PR :P

BigRoy added 5 commits September 4, 2019 10:43

Refactor absolute import to relative import

e31d9d9

Fix import consistency, order by most global to closest relative

d07dcf6

Refactor node to item in variables for consistency/clarity

e2b59e7

Refactor family and group config cache from loader.lib to tools.lib

c350d18

- Plus clarify that the group/family config data is a cache.

houndci-bot reviewed Sep 4, 2019

View reviewed changes

Refactor model COLUMNS to columns

93ed86f

houndci-bot reviewed Sep 4, 2019

View reviewed changes

BigRoy added 7 commits September 4, 2019 14:14

Fix refactored imports

5ea6fb7

Fix refactored Qt

df4ec95

Fix missing QtGui import

90e159e

Fix some outdated/wrong docstrings

8bf7bc8

Use list literal

3904439

Move utility function to top (ease debugging import error stack trace)

dc1a3fc

Move RecursiveSortFilterProxyModel to tools.models + fix imports

27449a5

houndci-bot reviewed Sep 4, 2019

View reviewed changes

avalon/tools/projectmanager/model.py Outdated Show resolved Hide resolved

BigRoy added 2 commits September 4, 2019 15:58

Comment example code again as it requires database/qt app to pass tests

4c6343a

Remove unused model

bc3df32

davidlatwe reviewed Sep 4, 2019

View reviewed changes

BigRoy added 2 commits September 4, 2019 16:25

Fix missed refactoring of model.COLUMNS to model.columns

db6f6e6

Fix refactor of NodeRole to ItemRole

a475623

BigRoy requested review from mottosso, mkolar and tokejepsen September 4, 2019 14:46

Remove "import as" for qtawesome imports

9bc098e

BigRoy mentioned this pull request Sep 4, 2019

Remove global state and initialisation from Loader #427

Open

davidlatwe reviewed Sep 4, 2019

View reviewed changes

avalon/tools/delegates.py Show resolved Hide resolved

avalon/tools/widgets.py Show resolved Hide resolved

Unify quote style from single quotes to double quotes (only in tools)

ae70d90

BigRoy force-pushed the fix_tool_imports branch from 4c30973 to ae70d90 Compare September 5, 2019 08:08

tokejepsen approved these changes Sep 5, 2019

View reviewed changes

mottosso reviewed Sep 6, 2019

View reviewed changes

avalon/tools/loader/model.py Outdated Show resolved Hide resolved

mottosso approved these changes Sep 6, 2019

View reviewed changes

Refactor model columns to Columns to separate from variables and methods

8f798ca

mottosso reviewed Sep 7, 2019

View reviewed changes

Cleanup multiple lib imports + reduce importing of specific functions

5de72f3

davidlatwe approved these changes Sep 9, 2019

View reviewed changes

Bump version to 5.4.0

3fa9636

davidlatwe merged commit a496dc8 into getavalon:master Sep 10, 2019

davidlatwe mentioned this pull request Sep 12, 2019

Update model module to adopt getavalon/core#440 davidlatwe/avalon-sftpc#1

Closed

BigRoy deleted the fix_tool_imports branch January 25, 2021 14:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor avalon.tools to avoid cross-referencing imports between tools #440

Refactor avalon.tools to avoid cross-referencing imports between tools #440

BigRoy commented Sep 4, 2019 •

edited by davidlatwe

Loading

davidlatwe left a comment

davidlatwe Sep 4, 2019

BigRoy Sep 4, 2019 •

edited

Loading

iLLiCiTiT Sep 4, 2019

BigRoy Sep 4, 2019 •

edited

Loading

davidlatwe Sep 4, 2019

mottosso Sep 6, 2019

BigRoy Sep 6, 2019

BigRoy commented Sep 4, 2019 •

edited

Loading

davidlatwe left a comment

davidlatwe commented Sep 5, 2019

BigRoy commented Sep 5, 2019

mottosso left a comment

mottosso commented Sep 6, 2019

BigRoy commented Sep 6, 2019

mottosso Sep 7, 2019

mottosso Sep 7, 2019

BigRoy Sep 9, 2019

davidlatwe left a comment •

edited

Loading

davidlatwe commented Sep 10, 2019 •

edited

Loading

BigRoy commented Sep 10, 2019

davidlatwe commented Sep 10, 2019

Refactor avalon.tools to avoid cross-referencing imports between tools #440

Refactor avalon.tools to avoid cross-referencing imports between tools #440

Conversation

BigRoy commented Sep 4, 2019 • edited by davidlatwe Loading

davidlatwe left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BigRoy Sep 4, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BigRoy Sep 4, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BigRoy commented Sep 4, 2019 • edited Loading

davidlatwe left a comment

Choose a reason for hiding this comment

davidlatwe commented Sep 5, 2019

BigRoy commented Sep 5, 2019

mottosso left a comment

Choose a reason for hiding this comment

mottosso commented Sep 6, 2019

BigRoy commented Sep 6, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidlatwe left a comment • edited Loading

Choose a reason for hiding this comment

davidlatwe commented Sep 10, 2019 • edited Loading

BigRoy commented Sep 10, 2019

davidlatwe commented Sep 10, 2019

BigRoy commented Sep 4, 2019 •

edited by davidlatwe

Loading

BigRoy Sep 4, 2019 •

edited

Loading

BigRoy Sep 4, 2019 •

edited

Loading

BigRoy commented Sep 4, 2019 •

edited

Loading

davidlatwe left a comment •

edited

Loading

davidlatwe commented Sep 10, 2019 •

edited

Loading