Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Github Webhook Integration #36

Open
ChristianRiesen opened this Issue · 17 comments

4 participants

@ChristianRiesen

Github has this wonderful feature of webhooks, which is beautifully generic. Now instead of running a satis update every 2 minutes, wouldn't it be much more convenient to run the update only when the webhook is called? Our installation has only a handful of repos in it, mostly libraries that change either multiple times in a day or then not for days or even weeks. So instead of running a cronjob every 5 minutes to ensure the packages.json is fresh, the webhook would be the ideal candidate for the task.

I'm currently working on creating something like this. I don't know yet how exactly I'll be implementing it, but I would love to hear input, so maybe I can publish my efforts here later.

@stof

Well, Satis is a command-line tool. So it does not bother about the way you trigger it. Using a webhook is indeed possible, and potentially more efficient than using a cron job (whcih would often run for nothing). This is just a matter of creating a webhook calling the satis build instead of writing a cron job calling it.

@ChristianRiesen

I just wonder if there could be something generic, so not everyone repeats the same process over and over.

@stof

Well, if you want to make it generic, do it fully generic: build a small script that can be called from the web as a github webhook and which would then run an arbitrary CLI command. And then, you can use it for Satis (configuring the command to php satis.phar build) but also for other stuff :)

@ChristianRiesen

Good idea. Trouble is the calls would be pretty long then, depending on the work satis has to do. Then also things like environment and so on come into play. Gotta think a bit more about it.

@stof

@ChristianRiesen the fact that the webhook implementation is generic or hardcodes the satis call does not change anything about the fact that Satis takes some times. The only difference is that it can be reused for other needs.

@ChristianRiesen

True, I'm just thinking about satis specific needs at the moment. We use it for private repos (which I thank many others do too) and so some authentication happens (ssh keys). So I got to test if this even works when called remotely.

@ChristianRiesen

I wrote this little generic piece here. Put it in satis/web, name it github.php and point github to http://packages.example.org/github.php with a webhook. It also saves the data sent, mainly used it for debugging but decided to leave it in there. Adjust for php binary file and maybe the satis config. It will try to write into a github.log in the same folder in any case. Maybe someone can use this or I will get around to make something more useful later on.

<?php

// Your satis config file, in the root folder of satis
$configfilename = 'satis.json';

// PHP binary path 
$phpbinpath = '/usr/bin/php';

// Only allow calls from here
$githubips = array(
    '207.97.227.253',
    '50.57.128.197',
    '108.171.174.178',
);

if (!in_array($_SERVER['REMOTE_ADDR'], $githubips)) {
    satislog('IP not a github IP!');
    exit('ERROR');  
} 

if (!isset($_POST['payload']) || !strlen($_POST['payload']) > 10) {
    satislog('No payload, or paylod too short');
    exit('ERROR');
}

$payloadfile = dirname(__FILE__) . '/' . 'github.json';
file_put_contents($payloadfile, $_POST['payload']);

$basepath = realpath(dirname(__FILE__) . '/../') . '/';

$command = $phpbinpath . ' ' . $basepath . 'bin/satis build  ' . $basepath . $configfilename . ' ' . $basepath . 'web';
$output = array();
$return_var = 0;

exec($command, $output, $return_var);

if ($return_var != 0) {
    satislog('Error in executing command. Output: ' . implode(' ', $output));
    exit('ERROR');
}

satislog('Everything ok, regenerated satis files');

exit('OK');

function satislog($message) {
    $filepath = dirname(__FILE__) . '/' . 'github.log';
    file_put_contents($filepath, date('Y-m-d H:i:s') . ' - ' . $_SERVER['REMOTE_ADDR'] . ' - ' . $message . PHP_EOL, FILE_APPEND);
}
@stof

exit('ERROR') seems weird to me. You should echo a message and exit with an exit code IMO

@stof

btw, you miss a bunch of escapeshellarg when building your command.

@ChristianRiesen

Thanks for the feedback. It's a quick and dirty variant that does the trick for my current needs. But I wonder what a better way would be and where to put it (how to contribute it into the satis codebase). Putting a script like this plain into web seems rather crude, but maybe appropriate to not load a whole stack of other things not really needed for satis. Also I don't really think I should call it this way (again, Q&D hack) but instead calling the command from the class directly (as described in the console component).

@stof

command classes are not meant to be called directly but to be called by the application. So calling the satis CLI is probably better.

@ChristianRiesen

That's what I had in mind. Get the application, instantiate the class, "find" the command, then send it the inputs and run execute. Well I give it a second whirl later on to make this "a better script (tm)"

@stof

Well, getting the application and calling it directly on it means your webhook has to know how Satis is build internally and requires you to build the console input object programmatically (which is often a pain). So calling the CLI is better IMO.
However, you could use the Process component (which is already installed as Composer depends on it) to do the call as its ProcessBuilder can take care of the escapping for you.

@ChristianRiesen

I understand where you are coming from with this. I will try out both (even just as an experiment) and post them when I have them.

@GromNaN

I had the same issue and did this very simple solution :

<?php 
// web/postcommit.php
touch(__DIR__.'/rebuild');

And a cronjob like this :

* * * * * cd /path/to/satis && test -f web/rebuild && rm web/rebuild && php bin/satis build -q satis.json ./web
@ChristianRiesen

Thanks, but running cron was exactly the thing I want to avoid. Your solution works perfectly fine and I can suggest others to use the same one for their installations. My current solution expands on the script above, and uses sudo, limited to that one script, so it uses the users environment and with that it's ssh keys for accessing the private repos.

@mpdude

This webhook feature would allow us to replace a custom composer repo server (written before Satis was created) with Satis, so I'd like to help to make it an "official" feature :)

What I would like to add is the possibility to post the URL of the (git, hg, ...) repo being updated to that webhook and re-scan only this repository for new branches/versions. (This is an performance improvement we had to add to our own solution, because otherwise updates took several minutes.)

And what if two updates are triggered in parallel? At least when using the (relatively new) "selected packages only"-build we might run into problems because information from existing packages.json files has to be re-used and might lose updates, right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.