Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto deploy webhook FILTER #107

Closed
hikinine opened this issue May 24, 2024 · 7 comments · Fixed by #108
Closed

Auto deploy webhook FILTER #107

hikinine opened this issue May 24, 2024 · 7 comments · Fixed by #108
Labels
enhancement New feature or request

Comments

@hikinine
Copy link
Contributor

Greetings, Great project!

Summary

On webhook autodeploy, consider only specified image tags to trigger deploy action

Description

I'm using dokploy in production for some personal projects.

In /services/application -> deployments there is the option to use the webhook url to integrate with dockerhub.

I have an github actions that, when starting a release, builds the application and pushes it to dockerhub.
On my dockerhub repo, i register that webhook url and everything works fine except...

My actions pushs actually 3 images (that was my call):

frontend:production
frontend:github_sha
frontend:v2.1.329    (lets say)

Is it feasible to implement a filter option?

Problem

Same image, with 3 different tags (for documentation, history, security purposes) are triggering 3 different deploy actions

Suggested Solution

1 - Perhaps we should automatically consider only the <tag> that is provided in the docker setup provider?
2 - On webhook register, allow user to decide which image tags would listener to.

.

image

@Siumauricio
Copy link
Contributor

I understand the problem, definitely the easiest solution would be to filter by the tag chosen in the docker setup provider, this would make it easier

@Siumauricio Siumauricio added the enhancement New feature or request label May 24, 2024
@hikinine
Copy link
Contributor Author

I understand the problem, definitely the easiest solution would be to filter by the tag chosen in the docker setup provider, this would make it easier

You are right, problem is that dockerhub does not offer any support to filter / condition webhook triggers.
Maybe with github webhook i can achieve that behaviour.

Thanks for the reply!

@Siumauricio
Copy link
Contributor

Siumauricio commented May 24, 2024

Yeah, what i mean is we can change the behavior in the endpoint where we recieve the webhook, and filter by the tag you specified in the docker provider, I think this will have more sense, because if tou specified a specific tag, you expect to be deployed only on that specific tag, in fact this is what we do on GitHub provider and git provider, we filter by the branch

@hikinine
Copy link
Contributor Author

With github integration worked as expected.
Kinda different workflow since now its listening to "push" on specified branch and then building that source code and deploying.

I'm still a little worried about build the application in the same machine that I use for production apps (maybe its just over thinking since my apps are lightweight).

Anyway,
I'm studying the dokploy architecture a little bit. As I understood,

1 - Dokploy listens to http request on /api/deploy/<refreshToken> (which do we pass to providers dockerhub, etc..)
2 - Find application by using provided token
3 - If application deploy sourceType is git / github:
-> Ensure that webhook payload matchs with provided branch (if not, ends with 301 status)
4 - Push deployments to queue.

const sourceType = application.sourceType;
if (sourceType === "github") {
const branchName = extractBranchName(req.headers, req.body);
if (!branchName || branchName !== application.branch) {
res.status(301).json({ message: "Branch Not Match" });
return;
}
} else if (sourceType === "git") {
const branchName = extractBranchName(req.headers, req.body);
if (!branchName || branchName !== application.customGitBranch) {
res.status(301).json({ message: "Branch Not Match" });
return;
}
}

5 - deployments worker start job execution.
6 - Processor execute deployApplication which will execute deploy action based on sourceType
Lets say our sourceType is "docker".

7 - Execute dockerBuild which will docker pull the image with provided setup (including tag).

Looks like dokploy is not deploying the other tags, which is great... I mean, we dont read the webhook payload. It is used only to trigger the deploy (which i think is the right choice).

Its a fake data (titleLog naming event payload)

dokploy took from webhook payload the name from tags (production, staging, etc) but it never deployed every image... It only repeat the same deploy for the initial provided tag image
image

So... there is no problem (only multiple deploy of the same image)

Solution

if (sourceType === 'docker') {
    const currentTagName = getTagFromDockerImage(application.dockerImage) ?? 'latest'


   if (currentTagName) {
      const requestBody =  request.body // i dont know how to access it
     // https://docs.docker.com/docker-hub/webhooks/?_gl=1*1ksgu74*_ga*NjA2MjQ5NTU3LjE3MTY1NTI4ODc.*_ga_XJWPQMJYHQ*MTcxNjU5Mjc5OS4zLjEuMTcxNjU5MzI2NS41Ny4wLjA.
      const payloadTagName = requestBody.push_data.tag
      if (payloadTagName  !== currentTagName) {
           res.status(301).json({ message: "Branch Not Match" });
	   return;
      }
   }

}
if (sourceType === "github") {
			const branchName = extractBranchName(req.headers, req.body);
			if (!branchName || branchName !== application.branch) {
				res.status(301).json({ message: "Branch Not Match" });
				return;
			}
		} else if (sourceType === "git") {
			const branchName = extractBranchName(req.headers, req.body);
			if (!branchName || branchName !== application.customGitBranch) {
				res.status(301).json({ message: "Branch Not Match" });
				return;
			}
		}

@hikinine
Copy link
Contributor Author

Here is the titleLog.
Appears like it is deploying the image described. But its only log purpose (deployed tag is always the provided on initial config)

if (headers["user-agent"]?.includes("Go-http-client")) {
if (body.push_data && body.repository) {
return `Docker image pushed: ${body.repository.repo_name}:${body.push_data.tag} by ${body.push_data.pusher}`;
}
}

@Siumauricio
Copy link
Contributor

Thank you, and good analysis that's how really works dokploy under the hood in deployments terms.

You are right, it will deploy only the tag you selected, but in your case you pushed 3 images at the same time it will create 3 deployments obviously with the same tag, but is not ideal to redeploy 3 times the same tag, We need to apply a filter there to prevent the deployments when you push another tag, the same as we do in github when the branch doesnt match it will return 301, We can do the same, altought is not a big problem for now but would be ideal to do make this little detail.

Would you ve up to make a PR?

@hikinine
Copy link
Contributor Author

hikinine commented May 25, 2024

Would you ve up to make a PR?

I'll be happy to work on it

hikinine added a commit to hikinine/dokploy that referenced this issue May 25, 2024
Siumauricio added a commit that referenced this issue May 25, 2024
…y-filter-rules-for-docker-image

feat: (#107) webhook listener filter docker events based on image tag.
@Siumauricio Siumauricio mentioned this issue May 30, 2024
Siumauricio added a commit that referenced this issue May 30, 2024
* feat: add schema for registry and routes

* feat: add docker registry upload

* feat: add show cluster

* refactor: set the registry url in image in case we have a registry asociated

* feat: add update registry and fix the docker url markup

* chore: remove --advertise-ip on swarm script

* refactor: remove listen address of swarm initialize

* feat: add table to show nodes and add dropdown to add manager & workers

* refactor: improve interface for cluster

* refactor: improve UI

* feat: add experimental swarm settings

* refactor: remove comments

* refactor: prettify json of each setting

* refactor: add interface tooltip

* refactor: delete static form self registry

* refactor: allow to se a empty registry

* fix: remove text area warnings

* feat: add network swarm json

* refactor: update ui

* revert: go back to swarm init config

* refactor: remove initialization on server, only on setup script

* Update LICENSE.MD

* feat: appearance theme support system config

* refactor: remove logs

* fix(README-ru): hyperlink-ed docs url

* feat: (#107) webhook listener filter docker events based on image tag.

Fixes #107

* refactor: simplify comparison docker tags

* refactor: remove return in res status

* refactor: prevent to updates download automatically

* feat: support code editor (#105)

* feat: support code editor

* Update codeblock

* refactor: remove unused class

---------

Co-authored-by: Mauricio Siu <47042324+Siumauricio@users.noreply.github.com>

* fix: select the right image from sourcetype (#109)

* chore: bump minor version

---------

Co-authored-by: hehehai <riverhohai@gmail.com>
Co-authored-by: Bayram Tagiev <bayram.tagiev.a@gmail.com>
Co-authored-by: Paulo Santana <30875229+hikinine@users.noreply.github.com>
@Siumauricio Siumauricio mentioned this issue May 30, 2024
Siumauricio added a commit that referenced this issue May 30, 2024
* feat: add schema for registry and routes

* feat: add docker registry upload

* feat: add show cluster

* refactor: set the registry url in image in case we have a registry asociated

* feat: add update registry and fix the docker url markup

* chore: remove --advertise-ip on swarm script

* refactor: remove listen address of swarm initialize

* feat: add table to show nodes and add dropdown to add manager & workers

* refactor: improve interface for cluster

* refactor: improve UI

* feat: add experimental swarm settings

* refactor: remove comments

* refactor: prettify json of each setting

* refactor: add interface tooltip

* refactor: delete static form self registry

* refactor: allow to se a empty registry

* fix: remove text area warnings

* feat: add network swarm json

* refactor: update ui

* revert: go back to swarm init config

* refactor: remove initialization on server, only on setup script

* Update LICENSE.MD

* feat: appearance theme support system config

* refactor: remove logs

* fix(README-ru): hyperlink-ed docs url

* feat: (#107) webhook listener filter docker events based on image tag.

Fixes #107

* refactor: simplify comparison docker tags

* refactor: remove return in res status

* refactor: prevent to updates download automatically

* feat: support code editor (#105)

* feat: support code editor

* Update codeblock

* refactor: remove unused class

---------

Co-authored-by: Mauricio Siu <47042324+Siumauricio@users.noreply.github.com>

* fix: select the right image from sourcetype (#109)

* chore: bump minor version

* fix: add redirect to https by default (#113)

---------

Co-authored-by: hehehai <riverhohai@gmail.com>
Co-authored-by: Bayram Tagiev <bayram.tagiev.a@gmail.com>
Co-authored-by: Paulo Santana <30875229+hikinine@users.noreply.github.com>
Siumauricio added a commit that referenced this issue Jun 3, 2024
* feat: add schema for registry and routes

* feat: add docker registry upload

* feat: add show cluster

* refactor: set the registry url in image in case we have a registry asociated

* feat: add update registry and fix the docker url markup

* chore: remove --advertise-ip on swarm script

* refactor: remove listen address of swarm initialize

* feat: add table to show nodes and add dropdown to add manager & workers

* refactor: improve interface for cluster

* refactor: improve UI

* feat: add experimental swarm settings

* refactor: remove comments

* refactor: prettify json of each setting

* refactor: add interface tooltip

* refactor: delete static form self registry

* refactor: allow to se a empty registry

* fix: remove text area warnings

* feat: add network swarm json

* refactor: update ui

* revert: go back to swarm init config

* refactor: remove initialization on server, only on setup script

* Update LICENSE.MD

* feat: appearance theme support system config

* refactor: remove logs

* fix(README-ru): hyperlink-ed docs url

* feat: (#107) webhook listener filter docker events based on image tag.

Fixes #107

* refactor: simplify comparison docker tags

* refactor: remove return in res status

* refactor: prevent to updates download automatically

* feat: support code editor (#105)

* feat: support code editor

* Update codeblock

* refactor: remove unused class

---------

Co-authored-by: Mauricio Siu <47042324+Siumauricio@users.noreply.github.com>

* fix: select the right image from sourcetype (#109)

* chore: bump minor version

* fix: add redirect to https by default (#113)

* Create FUNDING.yml

* Docker compose support (#111)

* feat(WIP): compose implementation

* feat: add volumes, networks, services name hash generate

* feat: add compose config test unique

* feat: add tests for each unique config

* feat: implement lodash for docker compose parsing

* feat: add tests for generating compose file

* refactor: implement logs docker compose

* refactor: composeFile set not empty

* feat: implement providers for compose deployments

* feat: add Files volumes to compose

* feat: add stop compose button

* refactor: change strategie of building compose

* feat: create .env file in composepath

* refactor: simplify git and github function

* chore: update deps

* refactor: update migrations and add badge to recognize compose type

* chore: update lock yaml

* refactor: use code editor

* feat: add monitoring for app types

* refactor: reset stats on change appName

* refactor: add option to clean monitoring folder

* feat: show current command that will run

* feat: add prefix

* fix: add missing types

* refactor: add docker provider and expose by default as false

* refactor: customize error page

* refactor: unified deployments to be a single one

* feat: add vitest to ci/cd

* revert: back to initial version

* refactor: add maxconcurrency vitest

* refactor: add pool forks to vitest

* feat: add pocketbase template

* fix: update path resolution compose

* removed

* feat: add template pocketbase

* feat: add pocketbase template

* feat: add support button

* feat: add plausible template

* feat: add calcom template

* feat: add version to each template

* feat: add code editor to enviroment variables and swarm settings json

* refactor: add loader when download the image

* fix: use base64 to generate keys plausible

* feat: add recognized domain names by enviroment compose

* refactor: show alert to redeploy in each card advanced tab

* refactor: add validation to prevent create compose if not have permissions

* chore: add templates section to contributing

* chore: add example contributing

* chore: add recomendation to show variables

* chore: add video to contributing templates

* chore: bump version

---------

Co-authored-by: hehehai <riverhohai@gmail.com>
Co-authored-by: Bayram Tagiev <bayram.tagiev.a@gmail.com>
Co-authored-by: Paulo Santana <30875229+hikinine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants