Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Actions] Globally resolvable action names #24789

Closed
silverwind opened this issue May 18, 2023 · 46 comments · Fixed by #25581
Closed

[Actions] Globally resolvable action names #24789

silverwind opened this issue May 18, 2023 · 46 comments · Fixed by #25581
Labels
topic/gitea-actions related to the actions of Gitea type/proposal The new feature has not been accepted yet but needs to be discussed first.

Comments

@silverwind
Copy link
Member

silverwind commented May 18, 2023

Issue

Currently when using this on gitea.com:

- uses: actions/setup-python@v4

It will resolve to instance-specific URL https://gitea.com/actions/setup-python which I see very problematic because it makes action names ambigous across multiple gitea instances and github itself (where this will work).

Proposal

If the action URL is not absolute, assume a hardcoded https://github.com/ prefix, just like podman and docker assume a docker.io/ prefix on images not having a fully qualified name. This will ensure that action names are unambigously resolvable globally.

@silverwind silverwind added type/proposal The new feature has not been accepted yet but needs to be discussed first. type/feature Completely new functionality. Can only be merged if feature freeze is not active. topic/gitea-actions related to the actions of Gitea and removed type/feature Completely new functionality. Can only be merged if feature freeze is not active. labels May 18, 2023
@delvh
Copy link
Member

delvh commented May 18, 2023

@delvh delvh closed this as completed May 18, 2023
@silverwind silverwind reopened this May 18, 2023
@ChristopherHX
Copy link
Contributor

I actually wish that DEFAULT_ACTIONS_URL (app.ini) is an array instead of a string.

With default value APP_URL (host your own actions via relative syntax), (https://gitea.com,
The last time I checked it was a place with mostly outdated copies of github),https://github.com

@silverwind
Copy link
Member Author

silverwind commented May 18, 2023

The option is good but it must not be an option, it needs to be hardcoded to ensure it stays globally resolveable. So the proposal is to remove this configurability and hardcode it to https://github.com/.

If it can not be removed, at least make the default https://github.com/ so that users can destroy the global resolution on their instances themselves.

@delvh
Copy link
Member

delvh commented May 18, 2023

No.
Give me a reason for reopening.

@delvh delvh closed this as completed May 18, 2023
@delvh
Copy link
Member

delvh commented May 18, 2023

That is not a good idea.

@delvh
Copy link
Member

delvh commented May 18, 2023

I actually wish that DEFAULT_ACTIONS_URL (app.ini) is an array instead of a string.
With default value APP_URL (host your own actions via relative syntax), (https://gitea.com,
The last time I checked it was a place with mostly outdated copies of github),https://github.com

okay, I can understand that.
Should we reopen this issue as that, or open a separate issue for it?

@delvh delvh reopened this May 18, 2023
@ChristopherHX
Copy link
Contributor

Should we reopen this issue as that, or open a separate issue for it?

Seems like I misunderstood the intention of this issue, my proposal should be a separate issue then.

However in both scenarios is actions/setup-python@v4 resolveable by default. So I thought that might be related.

@silverwind
Copy link
Member Author

silverwind commented May 18, 2023

Looking closer at podman, it actually has a config that seems to default to:

unqualified-search-registries = ["docker.io", "quay.io"]

From that, I take it could be possible to define multiple search prefixes and I would suggest a default of ["https://github.com"], with those URLs checked in sequence. I would not recommend adding the current instance at the end by default, as it would break global resolution if the value is anything but the single Github URL.

@ChristopherHX
Copy link
Contributor

ChristopherHX commented May 18, 2023

It will resolve to instance-specific URL https://gitea.com/actions/setup-python which I see very problematic because it makes action names ambigous across multiple gitea instances and github itself (where this will work).

The default is hardcoded to https://gitea.com, it is only the instance-specific url if Gitea Actions is hosted on https://gitea.com. For myself it is still https://gitea.com even if the instance is *.home

@silverwind
Copy link
Member Author

silverwind commented May 18, 2023

Yeah, I understand that it's set gitea.com on all instances. That's better than current instance, but still not globally resolvable.

I would set to github.com in all my instances and I would strongly suggest the default to be adjusted to that. GitHub Actions does not support this prefix syntax, so the only way for workflows to be portable in the github -> gitea direction is with the proposed default.

@ChristopherHX
Copy link
Contributor

ChristopherHX commented May 18, 2023

I would set to github.com in all my instances and I would strongly suggest the default to be adjusted to that.

Yes, I also think github.com is a better default than gitea.com.

I have modded my own actions_runner for gitea actions to look also on github.com to find actions.

@lunny
Copy link
Member

lunny commented May 18, 2023

I don't think github.com is better than gitea.com. Because some actions may need to be changed which cannot be used directly by Gitea. i.e. All actions which depend on GitHub APIs which Gitea APIs doesn't compatible will be run failed. For those actions, we need to fork and adjust them.

Considering 3 different Gitea instances.

1 Public Gitea instances
2 Private network Gitea instances that could connect to Internet.
3 Private network Gitea instances that cannot connect to Internet.

To satisfy all three situations, a configurable item is necessary.
And of course, if we can support multiple sites(less than 3), maybe it's better.
For 1 and 2, it could use localhost, gitea.com, github.com
For 3, it could use localhost

@silverwind
Copy link
Member Author

silverwind commented May 18, 2023

Maybe we can have a vote:

    1. 🎉 Set default to https://github.com
    1. 🚀 Multiple prefixes, ["https://github.com", "https://gitea.com"]
    1. ❤️ Set default to https://gitea.com

My option preference is 1 > 2, while 3 is unacceptable to me.

@lunny
Copy link
Member

lunny commented May 18, 2023

Maybe we can have a vote:

* 1. 🎉 Set default to `https://github.com`

* 2. 🚀 Multiple prefixes, `["https://github.com", "https://gitea.com"]`

* 3. ❤️ Set default to `https://gitea.com`

My option preference is 1 > 2, while 3 is unacceptable to me.

Please read my comment above.

@silverwind
Copy link
Member Author

silverwind commented May 18, 2023

Because some actions may need to be changed which cannot be used directly by Gitea. i.e. All actions which depend on GitHub APIs which Gitea APIs doesn't compatible will be run failed. For those actions, we need to fork and adjust them.

Yes, but fork them under a new name so that actions/foo unambiguosly refers to the action on github while https://gitea.com/actions/foo refers to the one one gitea. This is how it works in docker where image namefoo essentially always refers to docker.io/foo, and not another registry.

If we could convince GitHub to support a prefix, those actions would be even runnable (if compatible) on github.com.

@silverwind
Copy link
Member Author

Private network Gitea instances that cannot connect to Internet.

This is a valid concern, and a https://localhost entry would be need to set there, but it should not be the default.

@lunny
Copy link
Member

lunny commented May 18, 2023

👍 Could I vote ["https://gitea.com", "https://github.com"]? So that we can override some non-compatible actions in github.com. Users' workflows could run both successes without changes.

@silverwind
Copy link
Member Author

silverwind commented May 18, 2023

While not ideal, ["https://gitea.com", "https://github.com"] for the default seems like an acceptable compromise to me which will satisfy most scenarios.

@silverwind
Copy link
Member Author

silverwind commented May 18, 2023

I think we could extend DEFAULT_ACTIONS_URL with a comma syntax as comma is unlikely to appear in an URL, e.g.

DEFAULT_ACTIONS_URL = https://gitea.com, https://github.com

Trailing slash should be stripped while parsing the setting and subsequent URL join should be with /.

@silverwind
Copy link
Member Author

silverwind commented May 19, 2023

I think we should also put out a recommendation to always use FQANs (fully qualified action names), eliminiating any ambiguity on where the action is hosted, e.g.

https://github.com/actions/setup-python@4

instead of the ambiguous

actions/setup-python@4

We should also push github to support the FQAN syntax, as currently it's not possible to use actions hosted anywhere else than github.

@silverwind
Copy link
Member Author

Another idea that comes to mind: docker images names omit the https:// prefix, and I think this is something worth considering too. I think it should be possible to enforce that all actions should be hosted on https only, e.g. no http. This benefits both security and is less to type.

@yardenshoham
Copy link
Member

Yes, that also feels more like docker

NexZhu pushed a commit to daotl/gitea_act_runner that referenced this issue Jun 20, 2023
Follow https://gitea.com/gitea/act/pulls/58
Resolve go-gitea/gitea#24789

Co-authored-by: Jason Song <i@wolfogre.com>
Reviewed-on: https://gitea.com/gitea/act_runner/pulls/200
Reviewed-by: Jason Song <i@wolfogre.com>
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
@silverwind
Copy link
Member Author

silverwind commented Jun 23, 2023

I was toying with the idea of making updates support updating actions to latest versions locally, but given that the DEFAULT_ACTIONS_URL mechanism makes action URLs ambigous depending on instance configuration, it'll break in cases when DEFAULT_ACTIONS_URL is not https://github.com because DEFAULT_ACTIONS_URL is only resolved at the Gitea server, not the user's machine.

So I still strongly recommend we:

  • Set DEFAULT_ACTIONS_URL to https://github.com by default, or even better, eliminate the option it completely so instances can not be configured to resolve action names ambigously
  • Clearly document that non-GitHub-hosted actions need to be specified as FQAN
  • Convince GitHub to support FQANs, making it possible to use Gitea-hosted actions on Github Actions

This will ensure that when/if Gitea actions becomes popular, action names will always be unambigous globally and users can seamlessly share actions between GitHub and Gitea instances. It will require a bit more verbose syntax, but I think it's only for the better:

  • Use actions/checkout@v3 to refer to the github.com action
  • Use https://gitea.com/actions/checkout@v3 to refer to the gitea.com action
  • Use https://codeberg.org/actions/checkout@v3 to refer to the codeberg.org action

@silverwind
Copy link
Member Author

silverwind commented Jun 23, 2023

So I would propose the following course of actions:

  • Default DEFAULT_ACTIONS_URL to https://github.com in the next version
  • Deprecate DEFAULT_ACTIONS_URL in the runner in the next version
  • Remove DEFAULT_ACTIONS_URL in the runner in the following version

The minor inconvenience of adding https:/gitea.com to all action names is meager in comparison to breaking the nascent actions ecosystem as a whole. Without this change, GitHub's runner would not be able to resolve gitea-hosted actions.

@lunny
Copy link
Member

lunny commented Jun 23, 2023

I don't think runner support DEFAULT_ACTIONS_URL?

@wolfogre
Copy link
Member

  • Set DEFAULT_ACTIONS_URL to https://github.com by default, or even better, eliminate the option it completely so instances can not be configured to resolve action names ambigously
  • Clearly document that non-GitHub-hosted actions need to be specified as FQAN
  • Convince GitHub to support FQANs, making it possible to use Gitea-hosted actions on Github Actions

This will ensure that when/if Gitea actions becomes popular, action names will always be unambigous globally and users can seamlessly share actions between GitHub and Gitea instances. It will require a bit more verbose syntax, but I think it's only for the better:

  • Use actions/checkout@v3 to refer to the github.com action
  • Use https://gitea.com/actions/checkout@v3 to refer to the gitea.com action
  • Use https://codeberg.org/actions/checkout@v3 to refer to the codeberg.org action

I agree with @silverwind. We had to use DEFAULT_ACTIONS_URL because the act runner did not support uses: https://gitea.com/xxx at first, but now, the confusion it brings is greater than the convenience it brings.

@techknowlogick
Copy link
Member

This sounds like a reasonable approach to me too

@silverwind
Copy link
Member Author

silverwind commented Jun 26, 2023

I don't think runner support DEFAULT_ACTIONS_URL?

Is it a gitea-only setting and gitea communicates the absolute URL to the runner?

DefaultActionsURL string `ini:"DEFAULT_ACTIONS_URL"`

I think we should get DEFAULT_ACTIONS_URL=https://github.com into v1.21, and likely document for v1.20 that the default will change next version. Then maybe deprecate the setting in v1.22 or later, but no pressure on that one.

@wolfogre
Copy link
Member

wolfogre commented Jun 29, 2023

I find a very strong reason to make some changes for DEFAULT_ACTIONS_URL. It's about security so we will talk about it in private.

@silverwind Discord please.

@silverwind
Copy link
Member Author

Don't have time right now, but will check it later there.

@silverwind
Copy link
Member Author

silverwind commented Jun 29, 2023

One niceity we can allow ommitting the protocol, so github.com prefix is equal to https://github.com etc. It saves a bit of typing and is still unambiguous because after the URL origin, the user/repo format is always to strings separated by /. This is also how docker does it, but I'm of the opinion we should support both with and without protocol.

@wolfogre
Copy link
Member

wolfogre commented Jun 29, 2023

but I'm of the opinion we should support both with and without protocol.

I understand, that's what I thought at the beginning, but unfortunately GitHub Actions already support uses: x/y/z, and z means sub folder, like use: actions/cache/restore@v3.

So it's tricky to distinguish between host/user/repo and user/repo/folder.


Update:

Maybe we could and a leading / like /host/user/repo, but it could mislead users because it's easy to think that the leading '/' means nothing or it's just a typo.

@yardenshoham
Copy link
Member

docker treats it as a host if it has . or :, otherwise it's a user

@wolfogre
Copy link
Member

docker treats it as a host if it has . or :, otherwise it's a user

Not ideal for this, what if https://gitea.com/test.com/test?

@yardenshoham
Copy link
Member

It'll be the action test.com/test from the https://gitea.com instance

@silverwind
Copy link
Member Author

silverwind commented Jun 29, 2023

Unfortunatly, gitea allows usernames with ., making it ambigous again with hostname. I think we should forbid . in at least new usernames.

FTR: Github forbids . in username. Reference. I would even go as far as adopt that regex in gitea.

@wolfogre
Copy link
Member

. is not the point (although it is a "point" 😂). Even though GitHub/Gitea/GitLab/GitXX all forbid . in usernames, users can still use use custom git server which we never heard to host actions, right?

And it is also possible for hostname not to contain ., what if localhost or others hostnames definded in /etc/hosts? And the service names in docker-compose.yaml are available hostnames

@yardenshoham
Copy link
Member

You'd do something like localhost:80

@ChristopherHX
Copy link
Contributor

ChristopherHX commented Jun 29, 2023

users can still use use custom git server which we never heard to host actions, right?

I want to note that only act and it's derviates clone actions via git.

actions/runner downloads tar / zip archives via api after asking the GitHub Instance where it is located (GitHub also applies an allowed repository filter and sends a token to authorized private actions for private repos), which is faster than a full unshallow git clone on a microSD

Also the git only server would have to follow the <owner>/<repo> pattern, I saw some using a completly different url path and wouldn't work either way.

You'd do something like localhost:80

Since port 80 is http by default, would this mean like docker it contact the server via http and switch after a redirect to https? I thought that the outcome was strict https for short notation

And the service names in docker-compose.yaml are available hostnames

Same question, not everyone uses https within docker compose.

Eventually you have to fallback to the http/https notation in these cases.

@wolfogre
Copy link
Member

wolfogre commented Jun 29, 2023

To be clear, I agree with any solution that can shorten the path of actions, as long as:

  • schema://host/user/name still works, so that the user can still make it clear. And it is also possible to support more schemas, such as file://.
  • No ambiguity, and no additional operations are required to disambiguate. For example, host/user/name means default_schema://host/user/name , but if the host does not contain ., you must add the port to the host.
  • No guessing . For example, don't do something like pinging the hostname to check if it is a valid hostname.

Unfortunately, I cannot figure out a solution to omit protocol if the format is x/y/z.

silverwind pushed a commit that referenced this issue Jun 30, 2023
…25581)

Resolve #24789

## ⚠️ BREAKING ⚠️

Before this, `DEFAULT_ACTIONS_URL` cound be set to any custom URLs like
`https://gitea.com` or `http://your-git-server,https://gitea.com`, and
the default value was `https://gitea.com`.

But now, `DEFAULT_ACTIONS_URL` supports only
`github`(`https://github.com`) or `self`(the root url of current Gitea
instance), and the default value is `github`.

If it has configured with a URL, an error log will be displayed and it
will fallback to `github`.

Actually, what we really want to do is always make it
`https://github.com`, however, this may not be acceptable for some
instances of internal use, so there's extra support for `self`, but no
more, even `https://gitea.com`.

Please note that `uses: https://xxx/yyy/zzz` always works and it does
exactly what it is supposed to do.

Although it's breaking, I belive it should be backported to `v1.20` due
to some security issues.

Follow-up on the runner side:

- https://gitea.com/gitea/act_runner/pulls/262
- https://gitea.com/gitea/act/pulls/70
GiteaBot pushed a commit to GiteaBot/gitea that referenced this issue Jun 30, 2023
…o-gitea#25581)

Resolve go-gitea#24789

## ⚠️ BREAKING ⚠️

Before this, `DEFAULT_ACTIONS_URL` cound be set to any custom URLs like
`https://gitea.com` or `http://your-git-server,https://gitea.com`, and
the default value was `https://gitea.com`.

But now, `DEFAULT_ACTIONS_URL` supports only
`github`(`https://github.com`) or `self`(the root url of current Gitea
instance), and the default value is `github`.

If it has configured with a URL, an error log will be displayed and it
will fallback to `github`.

Actually, what we really want to do is always make it
`https://github.com`, however, this may not be acceptable for some
instances of internal use, so there's extra support for `self`, but no
more, even `https://gitea.com`.

Please note that `uses: https://xxx/yyy/zzz` always works and it does
exactly what it is supposed to do.

Although it's breaking, I belive it should be backported to `v1.20` due
to some security issues.

Follow-up on the runner side:

- https://gitea.com/gitea/act_runner/pulls/262
- https://gitea.com/gitea/act/pulls/70
silverwind pushed a commit that referenced this issue Jun 30, 2023
…25581) (#25604)

Backport #25581 by @wolfogre

Resolve #24789

## ⚠️ BREAKING ⚠️

Before this, `DEFAULT_ACTIONS_URL` cound be set to any custom URLs like
`https://gitea.com` or `http://your-git-server,https://gitea.com`, and
the default value was `https://gitea.com`.

But now, `DEFAULT_ACTIONS_URL` supports only
`github`(`https://github.com`) or `self`(the root url of current Gitea
instance), and the default value is `github`.

If it has configured with a URL, an error log will be displayed and it
will fallback to `github`.

Actually, what we really want to do is always make it
`https://github.com`, however, this may not be acceptable for some
instances of internal use, so there's extra support for `self`, but no
more, even `https://gitea.com`.

Please note that `uses: https://xxx/yyy/zzz` always works and it does
exactly what it is supposed to do.

Although it's breaking, I belive it should be backported to `v1.20` due
to some security issues.

Follow-up on the runner side:

- https://gitea.com/gitea/act_runner/pulls/262
- https://gitea.com/gitea/act/pulls/70

Co-authored-by: Jason Song <i@wolfogre.com>
@ghnp5
Copy link

ghnp5 commented Jul 16, 2023

Hey!

I believe I understand why this change was made, and I'm happy to keep the default to "github".

However, I'd like to understand whether this means I should prefer the GitHub Actions instead of Gitea Actions, when referring to uses:.

For example, you mirrored a few actions, such as checkout.
Is it OK to use GitHub's version instead of Gitea's version?

Also, there are several actions in the marketplace that use the Octokit library, which I always believed I should avoid those, because they only work for GitHub and not Gitea, am I correct to say that?

I'm just wondering how "safe" it is to default to GitHub, hoping that I will never use an action from GitHub by accident, that could potentially damage my repos or Gitea instance, due to incompatibilities (when Gitea's mirror version would be more compatible).

Apologies if something of what I said above doesn't make sense :)

Thank you!

@wolfogre
Copy link
Member

For example, you mirrored a few actions, such as checkout. Is it OK to use GitHub's version instead of Gitea's version?

IIRC, yes, the mirror is just a mirror.

Also, there are several actions in the marketplace that use the Octokit library, which I always believed I should avoid those, because they only work for GitHub and not Gitea, am I correct to say that?

Not really, of course the Octokit library is designed for GitHub not Gitea, however, Gitea has a certain degree of compatibility with GitHub, so I would say that most actions can be used on Gitea, even though they are developed based on Octokit. However, I cannot guarantee that all actions will be compatible. We are also working on identifying incompatible actions and making them compatible, but you know, it's a big task.

@ghnp5
Copy link

ghnp5 commented Jul 17, 2023

Thank you very much for your reply @wolfogre

That's really great to know that actions using Octokit are actually safe to use on Gitea!

I was trying to avoid any of those, so far :)

but you know, it's a big task

Absolutely - I understand!

My question above was not at all intending to be a bad criticism - it was just asking for clarification.

The work you guys have done for Gitea Actions is outstanding. Well done, really!

I've just submitted another donation. I absolutely love what you do 🧡

makrsmark pushed a commit to makrsmark/act that referenced this issue Aug 3, 2023
Partially resolve go-gitea/gitea#24789.

`act_runner`  needs to be improved to parse `gitea_default_actions_url` after this PR merged (https://gitea.com/gitea/act_runner/pulls/200)

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-on: https://gitea.com/gitea/act/pulls/58
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 1, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
topic/gitea-actions related to the actions of Gitea type/proposal The new feature has not been accepted yet but needs to be discussed first.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants