Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running dredd itself and hook server in separate docker containers #748

Closed
honzatrtik opened this issue Mar 9, 2017 · 19 comments
Closed

Running dredd itself and hook server in separate docker containers #748

honzatrtik opened this issue Mar 9, 2017 · 19 comments

Comments

@honzatrtik
Copy link

@honzatrtik honzatrtik commented Mar 9, 2017

Hi, first of all, thanks for dredd, it is great tool!

Im trying to start my hook tcp server in separate docker container and then run dredd tool itself in another container and link them together. The motivation is that I need to bootstrap our app in hook file, create some fixtures etc. but dont want to bloat our production image with dredd.
My idea was something like:

docker run -d --network dredd --name hook-server my-app-image php start-server.php hooks.php  --listen "0.0.0.0" --port 61321
docker run --rm --network dredd -v "/blueprint.apib:/blueprint.apib"--name dredd dred-image dredd /blueprint.apib http://api --hooks-worker-handler-port 61321 --hooks-worker-handler-host hook-server

Is something like this supposed to work? Im not sure, if hook server must be started by dredd itself...

Thanks in advance!

@w-vi
Copy link
Member

@w-vi w-vi commented Mar 9, 2017

At the moment the hook server is started by dredd but you can at least in theory pass dredd the custom hook handler command in your case the docker run <your hook server> and dredd will start it. I haven't tried it myself yet but was thinking along these lines. I am happy to help you get this setup running.

@w-vi
Copy link
Member

@w-vi w-vi commented Mar 9, 2017

@honzatrtik
Copy link
Author

@honzatrtik honzatrtik commented Mar 9, 2017

but you can at least in theory pass dredd the custom hook handler command in your case the docker run

Im afraid, this is not possible, because you can not run docker container inside docker container (it is actually possible but not in our case), anyway - maybe I can try to pass some dummy command, which will output string "Starting"... I will try and lat you know.
Thanks for the answer!

@w-vi
Copy link
Member

@w-vi w-vi commented Mar 9, 2017

I see, In that case I'd just put some script which would run and stop on SIGTERM, something along these lines should suffice.

#!/bin/bash
function term() {
    echo 'Caught SIGTERM, exiting.'
    exit 0
}

trap 'term' SIGTERM

while true
do
   sleep 1
done
@w-vi
Copy link
Member

@w-vi w-vi commented Mar 9, 2017

Look at the default values of all the related to hook handlers settings so that you listen on correct port and so on http://dredd.readthedocs.io/en/latest/usage-cli/#-hooks-worker-after-connect-wait

@honzajavorek
Copy link
Contributor

@honzajavorek honzajavorek commented Mar 15, 2017

@honzatrtik What @w-vi wrote you is exactly what we do in tests. We use dummy script (e.g. this) as a value for the --language option, so Dredd thinks it's been able to successfully start/run/terminate a hook handler, and then we run a mock hook handler for testing purposes on hook handler port.

So this should work perfectly. Feed Dredd with dummy script and expose the hook handler on correct port. You will just need to start and stop the hook handler yourself.

I'll close this as solved, but please come back and comment in case it doesn't work, so we can re-open and find a solution 🙂 And thanks for using Dredd and for bringing this up! 👍

@TotempaaltJ
Copy link

@TotempaaltJ TotempaaltJ commented Mar 20, 2017

Hey, it'd be super sweet if there was a native solution for this. We're running into this as well now. Maybe null could be an argument to language? The spawnHandler and terminateHandler calls could just be bypassed in that case.

It seems kind of pointless to have the whole handler host/port configuration if Dredd then wants to set it up itself anyway, right?

I was about to create a different issue for this, but this is probably a more appropriate place.

@honzajavorek
Copy link
Contributor

@honzajavorek honzajavorek commented Mar 20, 2017

@TotempaaltJ Actually, I think a new issue linking this one would be nice! I agree it's a good idea to give a standard option to bypass the handler logic.

@TotempaaltJ
Copy link

@TotempaaltJ TotempaaltJ commented Mar 20, 2017

Alright I created #755 ^

honzajavorek added a commit that referenced this issue Jan 3, 2019
@honzajavorek honzajavorek mentioned this issue Jan 3, 2019
2 of 3 tasks complete
honzajavorek added a commit that referenced this issue Jan 3, 2019
honzajavorek added a commit that referenced this issue Jan 3, 2019
@ApiaryBot
Copy link
Collaborator

@ApiaryBot ApiaryBot commented Jan 8, 2019

🎉 This issue has been resolved in version 5.4.2 🎉

The release is available on:

Your semantic-release bot 📦🚀

@rayzeller
Copy link

@rayzeller rayzeller commented Mar 8, 2019

Does anyone have an example dredd.yml to achieve the above bypass? included my docker-compose.yml and dredd.yml, and sleep.js but I can't seem to get it working.

https://gist.github.com/rayzeller/255fbbf88f9686b3062e84a67dbfa8fa
https://gist.github.com/rayzeller/77d0df2c33be05a064257590081a749e
https://gist.github.com/rayzeller/eaf49b661f5bbb6fa1b723972193ec81

Current error:
When i run dredd-hooks-ruby ./hooks.rb on api, then
docker-compose up dreddhooks, my tests run, but it doesn't hit the hooks. I have a bunch of logging in place that doesn't hit. And it also doesn't throw any error when the server's not running at all.

It seems that dredd is bypassing any connection to the hook server when passing the sleep.js bypass into the yml file. Was this issue possibly introduced in a new version in the past 1.5 years?

@honzajavorek
Copy link
Contributor

@honzajavorek honzajavorek commented Mar 11, 2019

@rayzeller
Copy link

@rayzeller rayzeller commented Mar 11, 2019

@honzajavorek i have those set in my dredd.yml. does dredd not always respect the yml? should i put them in command line?

hooks-worker-handler-host: http://api
hooks-worker-handler-port: 61321
@honzajavorek
Copy link
Contributor

@honzajavorek honzajavorek commented Mar 11, 2019

No, having them in YAML should be enough. Are you able to see anything interesting using --loglevel=debug?

@rayzeller
Copy link

@rayzeller rayzeller commented Mar 11, 2019

AH. i see it now. correct me if i'm wrong, but It looks like if nodejs is the language, then dredd uses the current filesystem for the hooks file. and does not (always?) start a server?

if (!runner.configuration.options.language

anyway, i was able to get hooks to register by changing my sleep.js to the following:

var ignore;

ignore = function() {
  return console.log('ignoring termination');
};

process.on('SIGINT', ignore);


const hooks = require('hooks');

//example hook just to get things going!
hooks.before('auth > Facebook Auth > Callback (Get) > 200 > application/json', (transaction) => {
  transaction.request.headers.Authorization = 'Basic: YWxhZGRpbjpvcGVuc2VzYW1l';
});

setInterval((function() {}), 1000);

Meaning. the hooks server doesn't get called at all.

@honzajavorek
Copy link
Contributor

@honzajavorek honzajavorek commented Mar 11, 2019

If the language is nodejs, there is no hook handler started, because Dredd can handle it within its own process in more efficient way, that's correct. In your original post you mentioned Ruby, so I replied in that context. For Ruby Dredd does try to spawn a separate hook handler process.

If you're about to use the JavaScript hooks, this should be enough to get you going:

const hooks = require('hooks');

//example hook just to get things going!
hooks.before('auth > Facebook Auth > Callback (Get) > 200 > application/json', (transaction) => {
  transaction.request.headers.Authorization = 'Basic: YWxhZGRpbjpvcGVuc2VzYW1l';
});

Save it as hooks.js and use --hookfiles to let Dredd to load the file. The hack with endlessly running process is needed only to fool Dredd in case you're using a different language than JS and you want to run the hook handler process in a separate Docker container.

@rayzeller
Copy link

@rayzeller rayzeller commented Mar 11, 2019

Awesome, thanks. It definitely says that in the docs, but I missed it!

@rayzeller
Copy link

@rayzeller rayzeller commented Mar 11, 2019

For anyone else needing a quick POC for using hooks with js in docker, here's an abbreviated version:

docker-compose.yml

version: '2'

services:
  postgis:
    image: mdillon/postgis:11-alpine
    volumes:
      - postgis:/var/lib/postgresql/data
    env_file:
      - '.env'

  api:
    depends_on:
      - 'postgis'
    build: .
    ports:
      - '3000:3000'
    volumes:
      - '.:/app'
    env_file:
      - '.env'

  dredd:
    depends_on:
      - 'api'
    image: apiaryio/dredd
    volumes:
      - './dredd.yml:/dredd.yml'
      - './hooks.js:/hooks.js'

volumes:
  postgis:

dredd.yml

color: true
dry-run: null
hookfiles: hooks.js
language: nodejs
require: null
server: null
server-wait: 3
init: false
custom: {}
names: false
only: []
reporter: apiary
output: []
header: []
sorted: false
user: null
inline-errors: false
details: false
method: []
loglevel: warning
path: []
hooks-worker-timeout: 5000
hooks-worker-connect-timeout: 1500
hooks-worker-connect-retry: 500
hooks-worker-after-connect-wait: 100
hooks-worker-term-timeout: 5000
hooks-worker-term-retry: 500
hooks-worker-handler-host: http://api
hooks-worker-handler-port: 61321
config: ./dredd.yml
blueprint: http://api:3000/apidocs
endpoint: http://api:3000

hooks.js

const hooks = require('hooks');

hooks.beforeEach((transaction) => {
  console.log("YEET");
  transaction.request.headers['X-User-Id'] = '1';
  transaction.request.headers['X-User-Authentication-Token'] = 'notreal'
});

setInterval((function() {}), 1000);

and then
docker-compose up api
docker-compose up dredd

@rayzeller
Copy link

@rayzeller rayzeller commented Mar 14, 2019

OK, it turns out the issue for me was that the hardcoded 127.0.0.1 in dredd_hooks_ruby prevented me from making a connection. changing it to 0.0.0.0 fixes that issue. will make a PR to address the hardcoding.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

6 participants