Add support for installing/managing addons via Git URL

[RobProductions]

Describe the project you are working on

Workflow improvements for the Godot editor

Describe the problem or limitation you are having in your project

Note: I used addons and plugins interchangeably here but what I mean is an extension or library that is meant to be self-contained and managed externally.

I'd like to create packages which aid in the development of Godot projects, but currently the process for importing addons isn't as clean as other engines/platforms due to the lack of dependencies and version management if you forgo the Asset Library. As I've been thinking about what it would take to actually implement #142 (dependency management for addons) I realized we would first need a mechanism to actually retrieve the addons that are listed as dependencies, which may come from Github/some other online source.

Additionally, the official tutorial for installing a plugin has the user go to the Github repo, download the code, and add it manually to your files via the file browser, which isn't quite as user friendly compared to something like npm or Git submodules which allows you to add packages via a single command line.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Something that I really miss from Unity is the ability add a package via URL by inputting it into the engine itself:

Which then allows the engine to internally handle installation and updates for you. I think something like this would be a perfect addition to Godot as it would address 3 different concerns:

1. Improves user experience for adding Git-based plugins since you wouldn't need to leave Godot to download most addons
2. Helps pave the way for Add dependency handling and enable users to organize and publish addons and addon-collections as packages #142 by adding a means (internally for the editor) to download required dependencies that come from online sources
3. Pairs really nicely with Install an add-on manifest to allow downloading add-ons when opening a project #7925 because it would allow the proposed addon list in the config file to simply store the URL that was used to retrieve the package so that it can be downloaded again when the user opens the project for the first time on a new machine. Also, it would be great if this list could be used to track versions for each of the URL-installed addons so that future enhancements like giving the user a button to re-grab the latest version will be possible.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

First, the editor could receive a new suite of functionality for downloading and importing an asset via Git URL. I'm not sure to what extent it already does this for the Asset Library but regardless it would also be nice to expose this as API to the user as well, so that they can do stuff like download optional dependency addons from an addon.

Then, I'm envisioning a button like this in the Plugin window:

The exact icon/wording of the button can be changed but this is a starting point. At first this button would simply prompt the user for a URL, and then it would go through the usual "add plugin" process that you'd go through with the Asset Library. This would be a good proof of concept that the functionality is possible without breaking anything that already exists; essentially it wouldn't keep track of anything you download, it's just another option for installation.

Finally, (or perhaps in place of the above step if its worthwhile) #7925 could be implemented using this functionality and with that we have 3 options:

1. Either keep the Load from URL button exactly the same and have it simply push the URL you use into the settings file to be independently added to the "installed plugins" list for visibility, or
2. The Load from URL button becomes a "Manage my URL addons" button which shows a separate list of addons which come from the settings file with the original installed list left untouched, or
3. The Load from URL button works the same, and all plugins shown in the "installed plugins" list are now derived from the settings file which can store/track the difference between embedded plugins or URL plugins (including Asset Lib). When a user adds a plugin via the file browser we would need some sort of button or automatic process to rescan the local embedded plugins for this to work.

For simplicity on the user end, I believe option 3 makes the most sense but will require a larger rewrite of the existing plugin management system. Hopefully some discussion on this can clear up the best possible direction if this sounds like a good idea. For now though, this proposal is mainly to kickstart the functionality and button that I mocked up.

If this enhancement will not be used often, can it be worked around with a few lines of script?

N/A

Is there a reason why this should be core and not an add-on in the asset library?

Probably possible to be built as an addon, but since it affects UX it would be best for everyone to get it. Also, it's meant to help create a practical step towards some of the other proposals discussing the future of addons, such as #1205

Let me know if this sounds useful or if there's a better way to implement this sort of package management type of functionality that pairs nicely with the other proposals :)

[Calinou]

• Related to Add support for importing projects from URLs in the project manager #6996.

Note that the AssetLib tab in the editor already has an Import button which accepts a ZIP archive:

If we add a similar button, I would make it act the same way. Reusing code here may be challenging though, as the AssetLib tab isn't a popup like the project settings.

Downloading from Git URLs requires git to be present in PATH, unless we just download a ZIP archive but this won't contain revision history (which may be a good thing if you're trying to avoid adding submodules to your repository).

The ZIP download URL can be inferred from Git repository URLs on GitHub, GitLab, BitBucket and more. We can have a default scheme that appends /zipball/master to the specified URL, with the .git suffix trimmed if present. This should allow supporting self-hosted GitLab and Gitea instances regardless of their domain name.

3. Pairs really nicely with Install an add-on manifest to allow downloading add-ons when opening a project #7925 because it would allow the proposed addon list in the config file to simply store the URL that was used to retrieve the package so that it can be downloaded again when the user opens the project for the first time on a new machine. Also, it would be great if this list could be used to track versions for each of the URL-installed addons so that future enhancements like giving the user a button to re-grab the latest version will be possible.

I'd rather not see two different, conflicting ways to install add-ons that require add-ons to be packaged differently. The asset library is here for a reason – decentralized add-ons should be packaged in the same fashion, so it's easy for add-ons to transition from one to another.

[RobProductions]

Note that the AssetLib tab in the editor already has an Import button which accepts a ZIP archive:

Ah that's true; I think I confused the plugin list with plain old addons because that's where my use cases align most. To that end, I find it a little confusing that there's no way to track versions of addons or see a full list of them akin to npm packages, so that's why I thought maybe utilizing the existing plugins list is best for implementing #7925 . But maybe there's a better way that can help improve clarity for users too, more on that below...

Downloading from Git URLs requires git to be present in PATH, unless we just download a ZIP archive but this won't contain revision history (which may be a good thing if you're trying to avoid adding submodules to your repository).

The ZIP download URL can be inferred from Git repository URLs on GitHub, GitLab, BitBucket and more. We can have a default scheme that appends /zipball/master to the specified URL, with the .git suffix trimmed if present. This should allow supporting self-hosted GitLab and Gitea instances regardless of their domain name.

Good idea! Though one of the reasons why Unity's URL loader is nice is because you can simply append #MyTagNumber or #MyBranch at the end of a Git URL like so: my-repo.git#1.3 in order to get a specific version downloaded. With your .zip approach the automatic appending would be a little more complicated because you'd need to do something like my-repo/archive/refs/tags/1.3.zip or maybe my-repo/zipball/1.3 works too? I think it does, just tested it. So I guess that's still probably preferable to requiring installed Git on the user's device... just something to consider.

I'd rather not see two different, conflicting ways to install add-ons that require add-ons to be packaged differently. The asset library is here for a reason – decentralized add-ons should be packaged in the same fashion, so it's easy for add-ons to transition from one to another.

I agree, but my suggestion was more along the lines of this being an additional way to track the sources of an addon allowing Godot to redownload it at a later time. The packages wouldn't have to be packaged differently; they would be structured the same as the Asset Lib addons and this would just give users an additional way to acquire those same types of addons but from a URL as opposed to Asset Lib/direct import. I was thinking the proposed list in #7925 would also keep track of addons acquired through the Asset Lib and addons that you've simply dragged into the res:// folder, as described in my option 3:

all plugins shown in the "installed plugins" list are now derived from the settings file which can store/track the difference between embedded plugins or URL plugins (including Asset Lib).

But just replace the word "plugins" with "addons" :-) On that note, if we were to start tracking addons and addon sources, could they just be anywhere in the project? Or should there be a specific "addon" folder that Godot can use to manage that stuff more cleanly? Another issue is the lack of configuration for plain old addons, or are we supposed to use plugins for those types of managed libraries even if they don't affect the editor in any way? In which case, now we're just back to 2 separate types of packages (Asset Lib and plugins) and then this whole tracking versions/sources thing makes more sense for plugins only.

Hopefully you can see now why it's been confusing to visualize the best direction for this stuff :') Are there any preferences on this from the community or core maintainers?