change: Clarify package vs module imported/available when loading packages

[ayjayt]

Generally, I think that griffe should somehow separate between

a) modules imported
b) modules available because the loaded object is a package
and also
c) .is_package should be True whenever the module is, in fact, a package (it doesn't do this for subpackages right now)

Current Behavior:

Anytime a package (a folder with a __init__.py) is loaded, .modules will list

a) the other modules in that folder
b) all of the other packages in that folder
b) all of the modules imported

You can traverse the package tree by following .modules but the .is_package attribute returns False for subpackages.

package1/
    __init__.py
    package2/
        __init__.py
        ...other.pys

Lets say you .load('package1'), and traverse via .modules to package2, griffe wont recognize it as a package- just a module. However, its .modules will still show its ...other.pys, so in some way we know its a package, because those other modules are not imported.

I should note that a package, once imported in python, is really just a module, except named after the folder and containing the contents of __init__.py, AFAIK. The only time in python I've ever seen the fact that something is a package matter is when using import since it allows us to traverse directories.

[pawamoy]

Hi @ayjayt! Thank you so much for the feedback, that's really appreciated 😁 Let me try to address your various remarks!

Separating imported modules from submodules

Currently, both will be returned by the .modules property, and both appear in the .members dictionary, true.

But imported modules will actually be instances of Alias rather than Module (except when a parent module imports its own submodule, a redundancy that Griffe doesn't support, by design, which is kind of a limitation, I agree).

So it's actually possible to distinguish both. We can also use a module's .imports to know whether one of its members was imported.

Subpackages are packages

That's true! However in Griffe we make the distinction and set .is_subpackage=True instead of .is_package=True for packages that are not the top-level package.

I should note that a package, once imported in python, is really just a module, except named after the folder and containing the contents of init.py, AFAIK. The only time in python I've ever seen the fact that something is a package matter is when using import since it allows us to traverse directories.

Very true! Python doesn't have "package" objects, only "modules". The import system is however affected by it, as you stated. One can import from a.b.c if these are folders, not if c was imported in b or b was imported in a. It's a bit complicated to summarize but that's actually one of the reason I decided not to support the distinction between a submodule and an imported submodule, in terms of objects access. There is some relevant information in this recommendation we make: https://mkdocstrings.github.io/griffe/guide/users/recommendations/python-code/#avoid-member-submodule-name-shadowing.

Ultimately, when navigating APIs (in Python, with Griffe, not as a user) or rendering documentation (in mkdocstrings), knowing whether one can do import a; a.b.c.function() or has to actually import a.b.c; a.b.c.function() does not matter, and supporting this would complicate Griffe a lot.

Checkout #124 which contains additional reasoning.

[pawamoy]

With all that said, your feedback helped me get a few ideas to improve Griffes API docs, to make things more discoverable. Mainly, I should reorder items to group them logically instead of alphabetically, and I will add links between related items (lineno and endlineno, is_package and is_subpackage, etc.). So thanks again!

Also very happy to continue discussing the API ☺️

[ayjayt]

@pawamoy I'd make another suggestion on another topic that, regarding "sponsor for access"- the docs on those could be clearer to know what I'd be sponsoring. Do you think that making the documentation available would be giving away too much?

Without having more info, it does force the potential sponsor to open a conversation with you, but not sure if its easy to that.

That said, I think, maybe not today, but I'll moving onto griffe-typescript soon. Assuming it supports vanilla javascript (a subset of typescript, I believe).

[pawamoy]

regarding "sponsor for access"- the docs on those could be clearer to know what I'd be sponsoring

Is there something in particular that you found unclear? Are you referring to https://mkdocstrings.github.io/griffe/insiders/, or https://mkdocstrings.github.io/insiders/, or the main page, https://pawamoy.github.io/insiders/? Maybe the fact we have several pages is what is confusing you 😅?

Do you think that making the documentation available would be giving away too much?

Are you suggesting that we show the source of Insiders code in the docs? For example https://mkdocstrings.github.io/griffe/reference/api/docstrings/parsers/#griffe.parse_auto? Even though most users would never bother maintaining patches obtained from the docs (somehow), that would kinda defeat the sponsorware strategy, yes.

Without having more info

Happy to try and update the docs if you can tell me what info is missing!

I'll moving onto griffe-typescript soon. Assuming it supports vanilla javascript (a subset of typescript, I believe).

Do you mean mkdocstrings-typescript, or really griffe-typescript? griffe-typescript is just a prototype, not working at all. mkdocstrings-typescript is still experimental too, but works. Also, I don't think mkdocstrings-typescript supports Javascript, since it uses Typedoc under the hood.

[ayjayt]

Alright, I'll set up a sponsorship and dig around, maybe not today. Thanks @pawamoy !

[pawamoy]

You're welcome, thank you for considering becoming a sponsor ☺️

[pawamoy]

Closing, feel free to comment further!