Implement some kind of webhook filtering

[shanemcd]

ISSUE TYPE

• Feature Idea

SUMMARY

While trying to make use of our initial webhook implementation, I am struggling with getting GitHub's events to do the Expected Thing™ in AWX.

There is the option to "Send me everything." or "Let me select individual events.", but unfortunately the latter option does not behave intuitively in a(t least one) key way: Checking "Pull requests" will send you events for:

Pull request opened, closed, reopened, edited, assigned, unassigned, review requested, review request removed, labeled, unlabeled, synchronized, ready for review, locked, or unlocked.

Note the absence of "comment" here. In order to receive a webhook when someone comments on a Pull request, you must select "Issue comments". This would mean we launch a job for every comment, whether or not it is on an Issue or Pull request. Ouch.

The more I explore this feature, it seems to me that we need some type of pre-launch filtering to allow AWX (or the user via configuration?) to decide whether or not we launch a job for any given event. It would be great to be able to check "Send me everything" and have AWX be smart enough to launch a job or ignore it, as I don't think the example I've provided here will be the only scenario where we need this type of functionality.

[rmanibus]

In fact with gitlab the scope of a webhook is rather large.

It would be very useful to be able to create conditions based on the request body to launch the job or not.
Moreover it would be very useful to be able to inject extra variables extracted from the request body.

[tomdaley92]

I would like to give an example of a specific use-case that this feature might be able to solve. Just being able to access a webhook's payload from within a job/playbook would be a massively huge improvement.

First of all, I want to show my long-hauled appreciation for Ansible. I absolutely love it. Our team uses Ansible exclusively for almost all of our application configuration and deployments and we try to adhere to Ansible best-practices as much as possible. We have a strict custom git-flow centered around code branches that represent environments and Ansible as our IaaC. And we have even gone so far as to write a custom jenkins library that helps us enforce this custom git-flow. The library can define a mapping of git branch names to ansible inventory names, install external/internal shared-roles, run playbooks... and much more.

To summarize the current state of things:

Github, by itself, does not provide a user with a way of triggering a "specific" webhook when a PUSH event happens on some "specific" branch. All PUSH events on a repository trigger whatever single webhook you have enabled with the PUSH event selected. Github sends a huge payload.. which is practically BEGGING to get parsed/filtered on the receiving end. We all know this, nuff said.

The problem:

We are not able to conditionally launch a job based on the "branch" of the most recent git push event that triggered the webhook POST to AWX. With the additions of workflow templates and webhooks, this is currently the ONLY thing keeping our team from using solely AWX as our CI/CD tool, without the help of anything other tool or middleware.

Our Team's Goal:

Completely move away from Jenkins as the "glue" language between Github and Ansible. We want to use ONLY AWX for our entire CI/CD flow.

Solution:

AWX solves most of this, since it natively has an RBAC, rest api, and jobs/workflows that can be triggered with webhooks, however AWX webhook filtering would open the doors to so many more options/customizations!

[tomdaley92]

Okay shorly after some playing around with it, I just found out that github event payloads are indeed transferred over into the job as extra vars, this was not clear to me before 👍

[kdelee]

Okay shorly after some playing around with it, I just found out that github event payloads are indeed transferred over into the job as extra vars, this was not clear to me before

Sounds like room for docs improvement

[rmanibus]

is there the same kind of behaviour for gitlab ?

[jbradberry]

@rmanibus the full payload is provided in all cases

[benjaminhon]

is there anyway to only run the workflow if the action=opened?
the problem i have not is that using the "enable webhook" method, i receive multiple actions from PRs, and i need to do a when condition in the playbook for action=opened for the PR.

This works, but is not ideal since for 1 PR it runs the workflow 4 times, the tasks are skipped for 3 of the times, and only run when the PR action=opened.

is there anyway to only run the workflow if the action=opened?

[shanemcd]

@benjaminhon - not currently.. an admittedly non-optimal solution right now would be to have a job template that inspects the webhook payload (injected via extra vars) and fails, preventing the workflow from proceeding.

[benjaminhon]

i see that seems the best solution now, thx

[jamielennox]

I was really surprised to not find this, it basically implies we should go github -> some ci -> AWX because it's the only way to control the triggers. For people in future, the really simple version of this i've done in my repo is:

---
- import_playbook: main.yml
  when:
    - tower_webhook_event_type | default('') == 'push'
    - tower_webhook_payload.ref | default('') == 'refs/heads/master'

Our job history is pretty much destroyed because there are a lot of job runs that basically end up with the whole play skipped. It also means i have a seperate job template that executes main.yml directly for human control.

It's not pretty but it does work.

[yeago]

A bit hacky since you have to target a host but I did an intermediate template in the workflow that only launches this

- name: validate webhook
  assert:
    that:
      - "{{ tower_webhook_payload.context | default('') == 'ci/circleci: build' }}"
      - "{{ tower_webhook_payload.state | default('') == 'success' }}"
      - "{{ tower_webhook_payload.branches[0].name | default('') == project_branch }}"  # aka 'master'
  when: validate_webhook | default(false)
  tags: validate_webhook

[andreasscherbaum]

Had to implement a similar workaround with the assert. My use case: we have a branch where only certain people can merge into, after review. AWX/Tower is then supposed to pick up the changes, triggered by a Webhook, and deploy the Playbook.
Certainly not interested in running the Playbook every single time someone updates a PR, because the main branch is not updated. But it doesn't seem to be possible to filter by branch.
The workaround with the assert still leaves failing jobs around every time it is triggered for a PR.

[Elyytscha]

Is it possible to get dynamically the branchname from a pullrequest which triggers an awx job via github webhook, so its possible to use this branch in the project sync in awx?

Because when a PR exists, which should be tested via an awx job, if the output is like it would be expected, it needs to checkout the project with the specific branch from the pullrequest.. not with master or any other hardcoded branch in the project

[yeago]

yes it is included in the payload

…

On Thu, Dec 22, 2022 at 7:31 AM Elias Rami ***@***.***> wrote: Is it possible to get dynamically the branchname from a pullrequest which triggers an awx job via github webhook, so its possible to use this branch in the project sync in awx? Because when a PR exists, which should be tested via an awx job, if the output is like it would be expected, it needs to checkout the project with the specific branch from the pullrequest.. not with master or any other hardcoded branch in the project — Reply to this email directly, view it on GitHub <#5323 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABFLORPGGGP7FXYACHZYYDWORCZPANCNFSM4JNL3PIQ> . You are receiving this because you commented.Message ID: ***@***.***>