Gitlab generic packages

[sirlatrom]

Prefer GitLab Package if found for release version

This allows downloads from private repositories where goreleaser/goreleaser#2002 and goreleaser/goreleaser#2005 are used to upload generic packages for a release.

On private repositories, the uploaded project files made by goreleaser without additional generic uploads are not accessible through the API but only through a web browser session.

[sirlatrom]

@marcosnils What do you think about this approach?

[marcosnils]

Will check today Sune 🙏 sent from mobile Em qui, 4 de mar de 2021 04:53, Sune Keller <notifications@github.com> escreveu:

…

@marcosnils <https://github.com/marcosnils> What do you think about this approach? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#45 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMBLWTTEDCGHLI2TLPDXFTTB44ANANCNFSM4YIKVZYQ> .

[marcosnils]

the uploaded project files made by goreleaser without additional generic uploads

Is this a goreleaser thing? Or still related to the original gitlab issue?

I think the overall PR is ok. However, for public repos with generic packages, there's no way to force bin to download Assets if required, right?

Shall we consider both packages and assets as candidates for public repos?

[sirlatrom]

Merge at will?

[marcosnils]

Hey @sirlatrom I'm doing some preliminary tests and I'm getting some weird results. I'm running this command./bin --debug install gitlab.com/graphviz/graphviz I get the following outputs:

Multiple matches found, please select one:

 [1] stable_ubuntu_19.10_debug_x86_64_libgraphviz4-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [2] stable_ubuntu_19.10_debug_x86_64_graphviz-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [3] stable_ubuntu_20.04_debug_x86_64_libgraphviz4-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [4] stable_ubuntu_20.04_debug_x86_64_graphviz-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [5] stable_ubuntu_18.04_debug_x86_64_libgraphviz4-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [6] stable_ubuntu_18.04_debug_x86_64_graphviz-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [7] stable_ubuntu_20.10_debug_x86_64_libgraphviz4-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [8] stable_ubuntu_20.10_debug_x86_64_graphviz-dbgsym_2.46.1-1_amd64.ddeb (asset link)

This is strange, just stable_ubuntu packages and only from the assets link, no generic packages at all.

Another thing that I was thinking that seems the current gitlab provider doesn't support is fectching generic packages from a single release, i.e: bin install gitlab.com/graphviz/graphviz/-/packages/1325470 and it still outputs the same as the first command.

[sirlatrom]

Hey @sirlatrom I'm doing some preliminary tests and I'm getting some weird results. I'm running this command./bin --debug install gitlab.com/graphviz/graphviz I get the following outputs:

Multiple matches found, please select one:

 [1] stable_ubuntu_19.10_debug_x86_64_libgraphviz4-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [2] stable_ubuntu_19.10_debug_x86_64_graphviz-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [3] stable_ubuntu_20.04_debug_x86_64_libgraphviz4-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [4] stable_ubuntu_20.04_debug_x86_64_graphviz-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [5] stable_ubuntu_18.04_debug_x86_64_libgraphviz4-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [6] stable_ubuntu_18.04_debug_x86_64_graphviz-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [7] stable_ubuntu_20.10_debug_x86_64_libgraphviz4-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [8] stable_ubuntu_20.10_debug_x86_64_graphviz-dbgsym_2.46.1-1_amd64.ddeb (asset link)

This is strange, just stable_ubuntu packages and only from the assets link, no generic packages at all.

@marcosnils There are no generic packages because they've chosen not to version their generic packages the same as their tags, e.g. the latest release is v0.0.1110734215291781187721198496291116006578054937833, except from v2.46.1, which looks like the only one. I'll try and change the search order_by to get the largest versions first.

Another thing that I was thinking that seems the current gitlab provider doesn't support is fectching generic packages from a single release, i.e: bin install gitlab.com/graphviz/graphviz/-/packages/1325470 and it still outputs the same as the first command.

That's an interesting idea, and I think that should be possible to support as well.

[marcosnils]

I'll try and change the search order_by to get the largest versions first.

SGTM!

[sirlatrom]

@marcosnils There are no generic packages because they've chosen not to version their generic packages the same as their tags, e.g. the latest release is v0.0.1110734215291781187721198496291116006578054937833, except from v2.46.1, which looks like the only one. I'll try and change the search order_by to get the largest versions first.

Turns out that works, and v2.46.1 comes up first, leading to matched packages.

I'm doing some preliminary tests and I'm getting some weird results. I'm running this command./bin --debug install gitlab.com/graphviz/graphviz I get the following outputs:

Multiple matches found, please select one:

 [1] stable_ubuntu_19.10_debug_x86_64_libgraphviz4-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [2] stable_ubuntu_19.10_debug_x86_64_graphviz-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [3] stable_ubuntu_20.04_debug_x86_64_libgraphviz4-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [4] stable_ubuntu_20.04_debug_x86_64_graphviz-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [5] stable_ubuntu_18.04_debug_x86_64_libgraphviz4-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [6] stable_ubuntu_18.04_debug_x86_64_graphviz-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [7] stable_ubuntu_20.10_debug_x86_64_libgraphviz4-dbgsym_2.46.1-1_amd64.ddeb (asset link)
 [8] stable_ubuntu_20.10_debug_x86_64_graphviz-dbgsym_2.46.1-1_amd64.ddeb (asset link)

This, it turns out after debugging, is because filetype.GetType("ddeb") matches the types.Unknown case - otherwise there would be 0 results to choose between, which is really the correct answer, as there are no single file static binaries in the package file list or asset link list.

[marcosnils]

hey @sirlatrom I was checking the last PR against this repository https://gitlab.com/tezos/tezos/-/releases which has both releases with the same version number and still I'm not seeing the generic package assets in the results.

Multiple matches found, please select one:

 [1] x86_64-linux-tezos-binaries.tar.gz (asset link)
 [2] tezos-endorser-alpha (x86_64 Linux) (asset link)
 [3] tezos-baker-alpha (x86_64 Linux) (asset link)
 [4] tezos-accuser-alpha (x86_64 Linux) (asset link)
 [5] tezos-endorser-008-PtEdo2Zk (x86_64 Linux) (asset link)
 [6] tezos-baker-008-PtEdo2Zk (x86_64 Linux) (asset link)
 [7] tezos-accuser-008-PtEdo2Zk (x86_64 Linux) (asset link)
 [8] tezos-endorser-007-PsDELPH1 (x86_64 Linux) (asset link)
 [9] tezos-baker-007-PsDELPH1 (x86_64 Linux) (asset link)
 [10] tezos-accuser-007-PsDELPH1 (x86_64 Linux) (asset link)
 [11] tezos-endorser-006-PsCARTHA (x86_64 Linux) (asset link)
 [12] tezos-baker-006-PsCARTHA (x86_64 Linux) (asset link)
 [13] tezos-accuser-006-PsCARTHA (x86_64 Linux) (asset link)
 [14] tezos-codec (x86_64 Linux) (asset link)
 [15] tezos-signer (x86_64 Linux) (asset link)
 [16] tezos-node (x86_64 Linux) (asset link)
 [17] tezos-client (x86_64 Linux) (asset link)
 [18] tezos-admin-client (x86_64 Linux) (asset link)

[sirlatrom]

hey @sirlatrom I was checking the last PR against this repository https://gitlab.com/tezos/tezos/-/releases which has both releases with the same version number and still I'm not seeing the generic package assets in the results.

Multiple matches found, please select one:

 [1] x86_64-linux-tezos-binaries.tar.gz (asset link)
...
 [18] tezos-admin-client (x86_64 Linux) (asset link)

@marcosnils First of all, kudos for finding a GitLab project that actually has both releases and generic packages! I've been searching through public projects for a while without finding one 😄

It looks like there are more fields that are not set when getting a public project through the API without a token, including the "packages_enabled" field, which then defaults to false. In 234e422 that is handled as a special case.

Through 8670d0c I found out that the generic package files in https://gitlab.com/tezos/tezos/-/releases/8.2.0 score lower than the asset links due to the asset links having "Linux" in the name, which adds some extra score points for the OS, which is missing in the generic package file names.

[marcosnils]

Through 8670d0c I found out that the generic package files in https://gitlab.com/tezos/tezos/-/releases/8.2.0 score lower than the asset links due to the asset links having "Linux" in the name, which adds some extra score points for the OS, which is missing in the generic package file names.

Hmm makes sense. I guess we're ok now.

We can merge this PR and then open issue to support the install with a specific package version URL. WDYT?