GitLab support and non-public GitHub/GitLab instances

[hhromic]

It would be really nice if eget could support GitLab repositories in addition to GitHub.
In addition, it would be nice if support for private GitHub/GitLab instances can be used as well.

My proposal for these features is two fold. First, refactor the code so it uses repository provider modules that contain types which satisfy a generic Provider interface with differnet functions needed by eget. The codebase would then consist of:

• A Provider interface with well-defined functions for all operations needed by eget from each provider.
  • For example, to list releases, to find and download assets, etc.
• A GitHubProvider concrete type that satisfies Provider containing all the existing code for the GitHub API.
• A GitLabProvider concrete type that satisfies Provider containing new code for the GitLab API.
• Any future provider implementation, for example for Gitea as requested in Gitea support #61.

Second, define a syntax for the target argument in eget so a provider and private domain can be specified. For this, the syntax proposed in #61 looks like a good start: [domain/]user/repo. However there are some considerations to address:

• Not all providers are limited to user/repo. GitLab for example supports nested groups such as org/group/repo.
  • For this, I would propose to simply generalize the syntax to [domain/]path.
  • If no domain is specified, default to public GitHub to preserve current behavior.
• Just by allowing domain there is the ambiguity of which provider to use for the requests.
  • For public instances, each Provider can implement a PublicDomains() function and do a simple check.
  • For private instances, there are some options to specify the provider to use.
    • eget could implement an optional --provider flag (and/or environment variable).
    • The provider can be incorporated in the target syntax: [provider:[domain/]]path
    • If no provider is specified, default to using domain to detect a public instance.

If you are willing to accept a contribution for all the above, I'm happy to work on it. I would just need your thoughts on the points above so the implementation goes the way you prefer.

[zyedidia]

I'd be happy to have support for GitLab or other providers. The code is currently organized in a pretty modular way -- it should be possible to add this without too much change. In particular, currently there are various types of Finders, which return all the URLs for assets. A Finder is constructed from the getFinder function, which is given the target string and uses that to determine what Finder to use and constructs it.

I think adding GitLab support would mostly just consist of creating a new GitlabAssetFinder that can find assets given a repository, and possibly other information (similar to the current GithubAssetFinder). Then the getFinder function would have to be updated to use the GitlabAssetFinder if it determines that the domain is a GitLab domain. It could also take into account a custom provide (I'm not sure if a flag or the provider:... syntax is better currently).

I think this is a good starting point. If this is easy to implement, then we can think about generalizing getFinder so that each Finder can provide information about whether it matches a domain or not, moving the logic in getFinder to each individual Finder and hopefully simplifying things.

Let me know what you think.

[hhromic]

Ah interesting, I have not dive very mch yet into the codebase, but what you describe sounds good to start with indeed.
Let me try a POC for a set of GitLabXYZ() functions and a simple dispatcher in getFinder() to see how it goes. I will report back here with the results. This is also a good way to get more familiar with your code for me.

I will start with public instance support (similar to existing GitHub support) and then we can discuss target syntax changes or another mean to select a provider for custom private instances.

I think, maybe for later, it would be a good idea to separate github, gitlab, anything-else code into their own module directories/files so the code is easier to maintain if more providers are added later.

[hhromic]

@zyedidia I created an initial POC implementation based on your comments.
See the changes here: hhromic/eget@master...feat-gitlab

In general, I tried to keep the code similar to the existing codebase, with as few changes as possible.
With this experience I got much more familiar with the source code of Eget.

• Like for GitHub, I implemented support for providing GitLab access tokens.
• Like for GitHub, I implemented both, an assets finder and a source tarball finder.
• Like for GitHub, I implemented support for the --pre-release option.
  • However, it's not 100% clear to me how it really works. I have not tested this very much.
• Given that GitLab can directly find releases by tag, I did not need to implement FindMatch().

While it was possible to just implement Gitlab-related functions and plumbing as suggested, the code turned out quite ugly because Eget is strongly designed/implemented around Github as the sole provider. I would really suggest to do a code refactoring to abstract GitHub into a provider interface with clearly separated implementations. I would be happy to do this work if you want.

Let me know what do you think!

EDIT: Here is a demonstration that this implementation works to download Gitlab's official CLI tool:

$ cat eget.toml
[global]
gitlab_token = "@~/secrets/gitlab-token"
target = "~/.local/bin"
upgrade_only = true

["gitlab.com/gitlab-org/cli"]
asset_filters = [".tar.gz"]
file = "glab"

$ ./eget gitlab.com/gitlab-org/cli
https://gitlab.com/gitlab-org/cli/-/releases/v1.33.0/downloads/glab_1.33.0_Linux_x86_64.tar.gz
Downloading 100% [=========================================================================================================================] (7.7/7.7 MB, 9.754 MB/s)
Extracted `bin/glab` to `/home/hhromic/.local/bin/glab`