Use a python-standard approach for plugins architecture

[jgsogo]

In the Python packaging user guide, there is one related to plugins, we should think about it and maybe adopt a standard approach for plugins architecture and discovering.

Creating and discovering plugins:

1. Using naming convention.
2. Using namespace packages.
3. Using package metadata.

Please everybody (ping @conan-io/barbarians), read and give your opinion.

[jgsogo]

Thinking about this, I will propose a renaming:

• plugins should be renamed as hooks, as it is more related to what they do.
• we may introduce real plugins (using one of the previous architectures) to extend generators, SCM repository managers,...

[memsharded]

I don't oppose to using python conventions, as long as it is not necessary that such modules are installed with python/pip, but can be bare files put somewhere in the system or conan cache folder.

[jgsogo]

Currently we have hooks, not plugins... and hooks should be installed using Conan commands, can be single files,... I agree with that, I won't change it.

But, what about a...?

pip install conan-mercurial

and then

from conans import ConanFile

class MyLib(ConanFile):
    scm = {
        'type': 'mercurial',
        'url': 'auto',
        'revision': 'auto',
    }

    ...

or... pip install conan-megabuild and...?

from conans import ConanFile, build_helper

class MyLib(ConanFile):
    def build(self):
        generator = build_helper.MegaBuild(self)
        ....
    ...

[uilianries]

I agree with hooks. plugins sounds like separated python packages

[solvingj]

I agree that what has been discussed so far are hooks. Based on these examples and use-cases, I have suggested using existing conan distribution machinery, because I didn't see anything that warranted the invocation of PIP. Honestly, even the custom "scm type" proposed by @jgsogo should be possible to achieve through something like a python_requires.

However, in support of @jgsogo 's point, if Conan were to support the more traditional type of "Plugin" (i.e. something which fundamentally alters a core behavior of Conan) then this would very likely require PIP to distribute. Jenkins Plugins for example, can alter the UI of the job navigation screens, the build page, and even add new "blocks" to the Jenkinsfile format. It's extremely powerful (and difficult to support I imagine).

Looking back, we now have these new hooks and python_requires which can bring a variety of new shared behavior into the existing Conan execution process. We've also have generators, profiles, and custom settings since the beginning. Conan is already loaded with well-structured extensibility points, which provide the right abstractions as a framework for this workload.

A key characteristic of Conan today, which seems really important, is that a conanfile.py is basically "self-contained". As long as it can find all it's .._requirements in Conan repositories, and the features used in it are supported by the version of Conan installed on the machine running it, it can be "executed".

With Plugins, I could see a future where organizations are writing conanfile.py 's which cannot even be executed without their own custom plugins installed via pip install conan-customizations, which they create internally, and then struggle to support over time. At that point, they will effectively be running a Fork of Conan, and it will be bad for everyone who has to deal with that scenario.

[jgsogo]

See #3867: Rename Plugins as Hooks

[jgsogo]

Need to read about the different plugins approaches, and think very carefully about this comment from @solvingj:

With Plugins, I could see a future where organizations are writing conanfile.py 's which cannot even be executed without their own custom plugins installed via pip install conan-customizations, which they create internally, and then struggle to support over time. At that point, they will effectively be running a Fork of Conan, and it will be bad for everyone who has to deal with that scenario.