WIP: Add authorization header support for Gitea webhooks #20267

justusbunsi · 2022-07-06T10:05:59Z

In some cases, a webhook endpoint requires some kind of authentication. In many cases it is enough to add an Authorization header to the webhook delivery. That's what this PR does: It allows to add such header to Gitea-typed webhooks.

Feature description

The header can be defined on a per-webhook basis and is disabled by default for backwards compatibility with already configured webhooks. For specific cases the header key is adjustable but defaults to Authorization. There are two supported authentication types/schemes: Basic authentication and Token based authentication.

Technical details

Authentication information are stored as encrypted string in the database inside the Meta field. If the administrator switches between the schemes and saves the page, the old settings are cleared. Temporarily switching in the UI won't clear data.
I also ensured that this particular header won't be stored in the protocol for sent webhooks.

Screenshots

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

In case the whole auth header feature is disabled for a configured webhook, there is no need to encrypt/decrypt values. Additionally, obsolete values due to auth type switching get cleared. Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

- Password type - Hint regarding "token" prepend Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

justusbunsi

There are some things I'd like to discuss:

justusbunsi · 2022-07-06T10:12:52Z

options/locale/locale_en-US.ini

+settings.webhook.auth_header.section = Authorization Header
+settings.webhook.auth_header.description = Add authorization header to webhook delivery.
+settings.webhook.auth_header.name = Header name
+settings.webhook.auth_header.type = Header content type
+settings.webhook.auth_header.type_basic = Basic authentication
+settings.webhook.auth_header.type_token = Token authentication
+settings.webhook.auth_header.username = Username
+settings.webhook.auth_header.password = Password
+settings.webhook.auth_header.token = Token
+settings.webhook.auth_header.token_description = During webhook delivery, the given value will be prepended with <code>token</code> followed by a space.


TBH, I am not sure about the naming here. Based on MDN docs¹² the naming would be "Authentication schema" instead of "Header content type" and the "Token authentication" would better fit with "Bearer scheme".

Footnotes

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization ↩

https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#authentication_schemes ↩

justusbunsi · 2022-07-06T10:14:03Z

services/webhook/deliver.go

+			case webhook_model.BASICAUTH:
+				content = fmt.Sprintf("Basic %s", base.BasicAuthEncode(meta.AuthHeader.Username, meta.AuthHeader.Password))
+			case webhook_model.TOKENAUTH:
+				content = fmt.Sprintf("token %s", meta.AuthHeader.Token)


Should it be up to the admin if and what is prepended?

What would be the use-case for that?

Sometimes the token must be provided with token <value> or Bearer <value>. Right now the second one is not possible.

Instead of handling this in the backend, maybe it could be exclusively handled in the frontend:

if the user chooses "basic auth", the frontend computes the base64 and sends Basic <computedbase64>

if the user chooses "bearer token", the frontend sends Bearer <token>

and maybe a 3rd option for the use to exactly choose the header value

This way the backend logic can be simplified a bit, while keeping maximum flexibility.

Yeah, I also thought about frontend code for it. But IIRC, it is preferred to do most logical code in the backend. So I went that way. 🙂

That token part definitely needs to be configurable, it's called the "schema" IIRC. Many APIs take Bearer but in the future, other schema could be introduced.

services/webhook/gitea.go

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

oliverpool · 2022-07-07T08:39:09Z

This could be useful for other webhooks (matrix for instance). Would it make sense to add a new dedicated Headers column?

justusbunsi · 2022-07-07T08:57:49Z

This could be useful for other webhooks (matrix for instance). Would it make sense to add a new dedicated Headers column?

We had a conversation in Discord about this feature a few days ago before I started implementing it and came to the conclusion that it would only be necessary for the generic webhook as all the others already have their authentication data configured. What would make the other webhooks require such headers? Maybe we missed use cases. Do you have an example?

oliverpool · 2022-07-07T09:24:51Z

As far as I understand, the repo is not adding custom webhooks until #19307 is somehow resolved.

Webhook PRs will be closed in favor of #19307

https://matrix.to/#/!wkyjxWeAOBpKMvHbno:matrix.org/$OytHn9Tle5j7BIQO1dRqou9XtxPc1f2dP9I8bDfLspc?via=freeyourgadget.org&via=matrix.org&via=tchncs.de

I would like to add a webhook to builds.sr.ht, which needs a Bearer token. So having this supported for all webhook types would be nice.

Moreover this PR encrypts the token (which I think is a good idea), whereas the current implementation (in matrix for instance) stores it in plaintext.

oliverpool · 2022-07-07T09:31:14Z

stupid question: is there any system that uses a different HeaderName than Authorization?

(allowing custom value for this could lead to security issues, since the request are made locally - X-Forwarded-For & co could do more harm than good)

justusbunsi · 2022-07-07T11:39:28Z

I would like to add a webhook to build.sr.ht, which needs a Bearer token. So having this supported for all webhook types would be nice.

Did a quick Google search for that address. Wouldn't that be a candidate for using the general webhook (Gitea webhook)?

Moreover this PR encrypts the token (which I think is a good idea), whereas the current implementation (in matrix for instance) stores it in plaintext.

😃 Agreed, that's why it's encrypted. If other maintainers also see it as better way to store those data, it might be changed for the other webhook types. But that would be a different PR, IMO. And definitely require a data migration.

justusbunsi · 2022-07-07T11:48:56Z

stupid question: is there any system that uses a different HeaderName than Authorization?

(allowing custom value for this could lead to security issues, since the request are made locally - X-Forwarded-For & co could do more harm than good)

TBH, I don't remember any system right now, but stumbled across one in the past, IIRC. Allow to change the header name is flexible enough to support those edge cases as well but I haven't thought about changing the header name being a potential security concern. That's a fair point. Maybe such edge case is not worth changing the name. Before I hardcoded it, I'd like to have opinions from the other maintainers.

oliverpool · 2022-07-07T13:52:05Z

Did a quick Google search for that address. Wouldn't that be a candidate for using the general webhook (Gitea webhook)?

Sorry, I mistype the URL: https://builds.sr.ht/
To trigger a build, it expects a GraphQL request (https://man.sr.ht/graphql.md) so the request must be adjusted on the gitea side before being sent.

that would be a different PR, IMO. And definitely require a data migration.

I don't know how much effort a data migration represents. But yes, you are right, it would be more like:

add an optional encrypted field AuthorizationHeader to the webhook model
add the UI + backend logic to manage this field

I'd like to have opinions from the other maintainers.

Sure!

(anyway, thank you very much for taking the time to reply to my comments. I want to contribute to gitea, but I am not yet experienced with enough its codebase and its development process)

justusbunsi · 2022-07-07T17:11:09Z

🤔 such a webhook request require the custom webhooks, I understand what you mean. So it is not possible as of now to trigger such request from any configurable webhook type. Am I right?

oliverpool · 2022-07-07T17:49:56Z

Yes, you are right. That's why I am looking at this PR and #19307, in the hope that it will enable my usecase at some point :-)

oliverpool · 2022-07-07T18:17:13Z

Apparently TeamCity also needs an Authorization header: #18667 . So with matrix, sourcehut and gitea-webhook, it means at least 4 actual usecase with Authorization header.

Regarding matrix, apparently the header is not sent on hook retry: #19872.
Your PR doesn't seem to suffer from such an issue (since it takes the token from the webhook_model.Webhook and not from webhook_model.HookTask)

oliverpool · 2022-07-11T06:58:32Z

From my side, there are 3 open questions:

store the full header as one line of text (Bearer 123, Basic Qrewhjk=) or as some kind of structure (bearer+123 / basicauth+username+password)
add this header only for gitea hooks or for all hook types (matrix, TeamCity or Sourcehut could use such a header)
allow the header name to be changed, or hard-code it to Authorization

It would be very valuable to have input from the other maintainers. I posted a message on #gitea-dev:matrix.org

silverwind · 2022-07-19T21:46:41Z

Why not make it a generic key/value map of headers that can be added to any webhook? I think it'd be more valuable and we don't need to care on the intrinsics of each header.

It would mean basic auth needs to be entered in raw base64 form, but I consider that acceptable.

justusbunsi · 2022-07-20T04:59:11Z

Why not make it a generic key/value map of headers that can be added to any webhook? I think it'd be more valuable and we don't need to care on the intrinsics of each header.

It would mean basic auth needs to be entered in raw base64 form, but I consider that acceptable.

Then we have to filter out or block all headers Gitea will add on its one to prevent forged headers, right?

silverwind · 2022-07-20T16:14:56Z

Then we have to filter out or block all headers Gitea will add on its one to prevent forged headers, right?

Yes, gitea-set headers should not be modifyable, e.g. do the go equivalent of a shallow JS merge ({...userHeaders, ...giteaHeaders}).

Edit: maps.Copy seems suitable.

oliverpool · 2022-07-26T15:30:17Z

Why not make it a generic key/value map of headers that can be added to any webhook? I think it'd be more valuable and we don't need to care on the intrinsics of each header.

Do you have any actual usecase for this?

I fear that it would add a lot of complexity for no real usage (with possibilities of security issues, as mentioned in #20267 (comment)).

Users will likely want to customize such headers depending on the hook (and not only static values). Maybe this could be linked to #19307

silverwind · 2022-07-26T18:52:48Z

Can't think of a use case besides authorization where one want arbitrary static HTTP headers, so I'm happy if you want to keep the PR to just Authorization right now.

silverwind · 2022-07-26T18:57:49Z

templates/repo/settings/webhook/gitea.tmpl

+			<h4>{{.locale.Tr "repo.settings.webhook.auth_header.section"}}</h4>
+			<div class="field">
+				<div class="ui checkbox">
+					<input class="hidden" name="auth_header_active" type="checkbox" tabindex="0" {{if .GiteaHook.AuthHeaderEnabled}}checked{{end}}>


Does this tabindex have any purpose? There are input elements before this. I'd recommend not setting it and just let the natural browser-derived tabindex take over.

😀 No, I assume that's just a copy/paste left-over.

silverwind · 2022-07-26T18:58:35Z

web_src/js/features/comp/WebHookEditor.js

+
+  if ($authHeaderSection.length === 0) {
+    return;
+  }


Suggested change

if ($authHeaderSection.length === 0) {

return;

}

if ($authHeaderSection.length === 0) return;

silverwind · 2022-07-26T19:02:31Z

web_src/js/features/comp/WebHookEditor.js

+
+    if (isBasicAuth) {
+      $basicAuthFields.addClass('required');
+      $basicAuthFields.find('input').attr('required', '');


I think $basicAuthFields.find('input').prop('required', true) would be the canonical way to set boolean attributes with jQuery as per this.

silverwind · 2022-07-26T19:03:17Z

web_src/js/features/comp/WebHookEditor.js

+      $headerName.parent().addClass('required');
+      $headerName.parent().show();


Suggested change

$headerName.parent().addClass('required');

$headerName.parent().show();

$headerName.parent().addClass('required').show();

This and similar cases can be chained for performance and conciseness.

justusbunsi · 2022-07-26T20:42:14Z

To sum up:

I am going to change the labels for the options to fit better.
Token based auth will be fully configurable by the admins (which technically allows them to set any header they want, combined with header name 🤷‍♂️). I'll ensure that Gitea internal headers are not overwritten.

I don't see an actual use case right now for complete flexibility, so I would like to keep it as authorization headers for now. All currently supported webhook don't need auth headers, so it would be this webhook type only. Custom webhooks (open PR) are a different use case where authorization headers are important.

Any other improvements can be done in another PR.

Ok for everyone?

oliverpool · 2022-07-27T06:13:00Z

Sounds good!

All currently supported webhook don't need auth headers, so it would be this webhook type only.

The (currently supported) matrix hook has some (ugly) workaround for authorization header support.

I would really like the authorization header support to be implemented for all hooks, since we identified at least 2 other systems which need them (Teamcity, Sourcehut) and probably more will come when the “custom webhook” PR is sorted out. Maybe adding an encrypted database column “authorization_header” would be a clean implementation.

Token based auth will be fully configurable by the admins

I’m not sure I understand. What will be configurable, by who?

the header value must be user-configurable
do you want to let the (instance-)admin configure the header name (with a default of “Authorization”)?
or did you mean “admin” in the sense of “repository-admin” ?

(as mentioned, I don’t see any use case for header name customization, only security issues)

justusbunsi · 2022-07-27T17:30:46Z

I would really like the authorization header support to be implemented for all hooks, since we identified at least 2 other systems which need them (Teamcity, Sourcehut) and probably more will come when the “custom webhook” PR is sorted out. Maybe adding an encrypted database column “authorization_header” would be a clean implementation.

Not sure that it is an actual topic for this PR. Maybe I missed something. I'll have another, closer look at that when modifying this PR.

Token based auth will be fully configurable by the admins

I’m not sure I understand. What will be configurable, by who?

the header value must be user-configurable

do you want to let the (instance-)admin configure the header name (with a default of “Authorization”)?

or did you mean “admin” in the sense of “repository-admin” ?

It's going to be a per-webhook configration. The person configuring the webhook also decides the content. Nothing internally prefixed or suffixed.

(as mentioned, I don’t see any use case for header name customization, only security issues)

I see your point on this. No customizable header key might be safer.

I need some time to continue with it. Don't have much time currently.

oliverpool · 2022-08-23T13:02:37Z

Don't have much time currently.

I had some time lately, so I took the liberty of implementing another approach: #20926.
I would really appreciate your feedback, if you have some time :)

@justusbunsi

_This is a different approach to #20267, I took the liberty of adapting some parts, see below_ ## Context In some cases, a weebhook endpoint requires some kind of authentication. The usual way is by sending a static `Authorization` header, with a given token. For instance: - Matrix expects a `Bearer <token>` (already implemented, by storing the header cleartext in the metadata - which is buggy on retry #19872) - TeamCity #18667 - Gitea instances #20267 - SourceHut https://man.sr.ht/graphql.md#authentication-strategies (this is my actual personal need :) ## Proposed solution Add a dedicated encrypt column to the webhook table (instead of storing it as meta as proposed in #20267), so that it gets available for all present and future hook types (especially the custom ones #19307). This would also solve the buggy matrix retry #19872. As a first step, I would recommend focusing on the backend logic and improve the frontend at a later stage. For now the UI is a simple `Authorization` field (which could be later customized with `Bearer` and `Basic` switches): ![2022-08-23-142911](https://user-images.githubusercontent.com/3864879/186162483-5b721504-eef5-4932-812e-eb96a68494cc.png) The header name is hard-coded, since I couldn't fine any usecase justifying otherwise. ## Questions - What do you think of this approach? @justusbunsi @Gusted @silverwind - ~~How are the migrations generated? Do I have to manually create a new file, or is there a command for that?~~ - ~~I started adding it to the API: should I complete it or should I drop it? (I don't know how much the API is actually used)~~ ## Done as well: - add a migration for the existing matrix webhooks and remove the `Authorization` logic there _Closes #19872_ Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: Gusted <williamzijl7@hotmail.com> Co-authored-by: delvh <dev.lh@web.de>

lunny · 2022-11-04T05:47:34Z

Thank you for your time! @justusbunsi

justusbunsi · 2022-11-07T07:28:39Z

Huge thanks to @oliverpool for picking up this topic and get it over the finish line. 🥇

justusbunsi added 14 commits July 2, 2022 19:13

Implement frontend part

1fdabab

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

Persist auth header configuration

08b4317

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

Add authentication header to webhook delivery

3111036

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

Refactor metadata creation to centralized location

f8d2768

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

Improve token input field

36e0414

- Password type - Hint regarding "token" prepend Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

Apply code formatting

fcfc79f

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

Pass encrypt and decrypt functions as parameters

8e210e6

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

Add early exit on error

88b734e

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

Add GiteaHook unit tests

3f33b2d

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

Merge branch 'go-gitea:main' into webhooks/add-authentication-header

182e192

Apply code formatting

4c13cdc

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

Reword header content types

b69dd15

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

Add section to docs

a4ad516

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

justusbunsi added the type/feature Completely new functionality. Can only be merged if feature freeze is not active. label Jul 6, 2022

justusbunsi commented Jul 6, 2022

View reviewed changes

GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Jul 6, 2022

Apply frontend formatting

96c608c

Signed-off-by: justusbunsi <61625851+justusbunsi@users.noreply.github.com>

Gusted added this to the 1.18.0 milestone Jul 6, 2022

oliverpool mentioned this pull request Jul 7, 2022

Matrix webhook doesn't send access token in Authorization header on replay #19872

Closed

justusbunsi added 3 commits July 9, 2022 21:30

Merge branch 'main' into webhooks/add-authentication-header

34118fe

Merge branch 'main' into webhooks/add-authentication-header

f3e71fd

Merge branch 'main' into webhooks/add-authentication-header

6209f2c

Merge branch 'main' into webhooks/add-authentication-header

2eae69c

silverwind reviewed Jul 26, 2022

View reviewed changes

justusbunsi changed the title ~~Add authorization header support for Gitea webhooks~~ WIP: Add authorization header support for Gitea webhooks Jul 27, 2022

silverwind mentioned this pull request Aug 9, 2022

refactor webhook *NewPost #20729

Merged

oliverpool mentioned this pull request Aug 23, 2022

Add Webhook authorization header #20926

Merged

lunny modified the milestones: 1.18.0, 1.19.0 Oct 17, 2022

justusbunsi closed this Nov 4, 2022

lunny removed this from the 1.19.0 milestone Nov 4, 2022

justusbunsi deleted the webhooks/add-authentication-header branch November 7, 2022 07:28

go-gitea locked and limited conversation to collaborators May 3, 2023

		$headerName.parent().addClass('required');
		$headerName.parent().show();

WIP: Add authorization header support for Gitea webhooks #20267

WIP: Add authorization header support for Gitea webhooks #20267

Conversation

justusbunsi commented Jul 6, 2022 • edited Loading

justusbunsi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Footnotes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

justusbunsi Jul 7, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oliverpool commented Jul 7, 2022

justusbunsi commented Jul 7, 2022

oliverpool commented Jul 7, 2022 • edited Loading

oliverpool commented Jul 7, 2022 • edited Loading

justusbunsi commented Jul 7, 2022

justusbunsi commented Jul 7, 2022

oliverpool commented Jul 7, 2022

justusbunsi commented Jul 7, 2022

oliverpool commented Jul 7, 2022 • edited Loading

oliverpool commented Jul 7, 2022 • edited Loading

oliverpool commented Jul 11, 2022

silverwind commented Jul 19, 2022 • edited Loading

justusbunsi commented Jul 20, 2022

silverwind commented Jul 20, 2022 • edited Loading

oliverpool commented Jul 26, 2022

silverwind commented Jul 26, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

silverwind Jul 26, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

justusbunsi commented Jul 26, 2022 • edited Loading

oliverpool commented Jul 27, 2022

justusbunsi commented Jul 27, 2022

oliverpool commented Aug 23, 2022

lunny commented Nov 4, 2022

justusbunsi commented Nov 7, 2022

justusbunsi commented Jul 6, 2022 •

edited

Loading

justusbunsi Jul 7, 2022 •

edited

Loading

oliverpool commented Jul 7, 2022 •

edited

Loading

oliverpool commented Jul 7, 2022 •

edited

Loading

oliverpool commented Jul 7, 2022 •

edited

Loading

oliverpool commented Jul 7, 2022 •

edited

Loading

silverwind commented Jul 19, 2022 •

edited

Loading

silverwind commented Jul 20, 2022 •

edited

Loading

silverwind Jul 26, 2022 •

edited

Loading

justusbunsi commented Jul 26, 2022 •

edited

Loading