Add Support for Webhooks

[andreas-marschke]

Add support for Webhooks in Ansible-Semaphore

This MR adds support for Webhooks to trigger Ansible Templates and run Playbooks based on configuration given to a webhook that can be potentially received by semaphore.

Webhook Configuration

The change hereby presented adds a new item on the sidebar titled Webhooks, navigating here will present the user with a list of Webhooks being recognized by the system.

A webhook links to a template in Semaphore which will be run when the e webhook satisfied its extractor configuration and has succeeded all Matcher criteria.

Webhook Extractors

A webhook consists of 1 or more Webhook Extractors which in turn have 1 or more Matchers and Extracted Values

Webhook Matchers

A Matcher defines a characteristic of the webhook event being sent to the semaphore system where the system will read the request for its body or header data and match on fields presented in the webhook.

Consider the following Webhook posted to semaphore system:

POST /webhook HTTP/1.1
Host: semaphore.example.org
Content-Type: application/x-application-json

{
  "data": {
    "field1": "a",
    "field2": "b"
  }
}

A matcher may now select to parse the body JSON to retrieve for example data.field1
It may now try to :

• test if a field value is equal
• test if a field value is unequal
• test if a field value contains a string

If all the Matchers for an extractor have been met the system will engage the extractor linked Webhook Extract Values

Extracted Value

An extracted value is any field selected from the webhook to turn into a variable on the template fields/values being passed to the playbook being run as part of the template attached to the webhook object.

Considering the previously described Webhook posted to semaphore system:

POST /webhook HTTP/1.1
Host: semaphore.example.org
Content-Type: application/x-application-json

{
  "data": {
    "field1": "a",
    "field2": "b"
  }
}

The extractvalue may now select data.field2 and assign it to VARIABLE_A which would be passed as configuration variable to the ansible-playbook run invoked by the template.

This way the system can augment the received data to provide it to the playbook being run.

Webhook execution

Upon receiving a Webhook that satisfied all matchers and has successfully extracted all the required values into variables, the system will attempt to trigger a run of a template, build or deployment.

[andreas-marschke]

@fiftin It would be awesome if you could help me get this over the line as I'm currently hitting some issues with boltdb and updating the objects. I believe part of this is due to the way the buckets are set up?

[andreas-marschke]

@fiftin It is finally ready for review.

I will work on the merge conflicts sooner rather than later

[fiftin]

Hi @andreas-marschke

Thank you for your PR. I will help with it if I can.

[fiftin]

Working with the PR! Thank you!

[anardil]

Hello,

Do you know when it will be available on the next version?

Thank you.

[andreas-marschke]

@fiftin I updated the PR with some UI polish and backend fixes for the boltdb version of it. I'm trying to figure out what's wrong on the CI, if you can help me figure this out, that would be cool.

Here are some screenshots:

Webhooks Main Page

Extractor View

Extractor Actual Configuration

[alex-grandson]

Sorry for stupid question, but could you please clearify for me one thing. As I understood these changes point to webhook to semaphore system, not for an external thing.
So, my question is do you have in plans to include external webhook functionality? For situations like after ansible play ends notificate some system in general with data about failure/success, hosts, maybe output optionally? For example, I have some cron-job playbook run and I want to "subscribe" for its runs.
I'll be very grateful for the answer
Thanks!

[andreas-marschke]

Notifications/Pushes/Webhooks _from_ your Ansible Playbooks should be solved differently. This changeset is not to send webhooks ourselves. I suggest researching means of doing so at the end of your playbooks instead by using another web POST step in them.

…

On Tue, 9 Jan 2024, 12:09 Alexey Kutsenko, ***@***.***> wrote: Sorry for stupid question, but could you please clearify for me one thing. As I understood these changes point to webhook to semaphore system, not for an external thing. So, my question is do you have in plans to include external webhook functionality? For situations like after ansible play ends notificate some system in general with data about failure/success, hosts, maybe output optionally? I'll be very grateful for the answer Thanks! — Reply to this email directly, view it on GitHub <#1365 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJEWWH2SNADHLFFQ4O6IGTYNUQODAVCNFSM6AAAAAA2YPPDMOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBSHA3TEMJTG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[andreas-marschke]

@fiftin Thank you for the support and help getting this over the line.
When do you plan on having the next release? I'm fine with a Beta so long as I have a container to run.

[arpanghosh8453]

is this feature available in the docker container yet?

[andreas-marschke]

It's available in the `develop` branch tag on docker but has a bug for which a patch is still to be merged. See: !1816 is this feature available in the docker container yet?

…

[tboerger]

On github it's #1816 :D