A broker system between a public service and a private service
joshje Merge pull request #140 from snyk/feat/broker-support-get-repo
feat: add support for getting a single repo
Latest commit 26cf000 Sep 28, 2018

README.md

Snyk logo

Known Vulnerabilities


snyk/broker

Snyk broker proxies access between snyk.io and source-code management systems (SCMs) such as GitHub Enterprise, GitHub.com and Bitbucket Server.

The broker server and client establish an applicative tunnel, proxying requests from snyk.io to the SCM (fetching manifest files from monitored repositories), and vice versa (webhooks posted by the SCM).

The broker client runs within the user's internal network, keeping sensitive data such as SCM tokens within the network perimeter. The applicative tunnel scans and whitelists only relevant requests, narrowing down the access permissions to the bare minimum required for Snyk to actively monitor a repository.

Usage

The broker client is published as a set of docker images, each configured for a specific SCM. Custom configuration is provided as environment variables.

GitHub.com

To use the the broker client with GitHub.com, run docker pull snyk/broker:github-com. The following environment variables are needed to customize the broker client:

  • BROKER_TOKEN - the snyk broker token, obtained from your snyk org settings view.
  • GITHUB_TOKEN - a personal access token with full repo, read:org and admin:repo_hook scopes.
  • PORT - the local port at which the broker client accepts connections. Default is 7341.
  • BROKER_CLIENT_URL - the full URL of the broker client as it will be accessible by GitHub.com webhooks, such as http://my.broker.client:7341

Command-line arguments

You can run the docker container by providing the relevant configuration:

docker run --restart=always \
           -p 8000:8000 \
           -e BROKER_TOKEN=secret-broker-token \
           -e GITHUB_TOKEN=secret-github-token \
           -e PORT=8000 \
           -e BROKER_CLIENT_URL=http://my.broker.client:8000 \
       snyk/broker:github-com

Derived docker image

Another option is to build your own docker image and override relevant environment variables:

FROM snyk/broker:github-com

ENV BROKER_TOKEN      secret-broker-token
ENV GITHUB_TOKEN      secret-github-token
ENV PORT              8000
ENV BROKER_CLIENT_URL http://my.broker.client:8000

GitHub Enterprise

To use the the broker client with a GitHub Enterprise deployment, run docker pull snyk/broker:github-enterprise tag. The following environment variables are needed to customize the broker client:

  • BROKER_TOKEN - the snyk broker token, obtained from your snyk org settings view.
  • GITHUB_TOKEN - a personal access token with full repo, read:org and admin:repo_hook scopes.
  • GITHUB - the hostname of your GitHub Enterprise deployment, such as your.ghe.domain.com.
  • GITHUB_API - the API endpoint of your GitHub Enterprise deployment. Should be your.ghe.domain.com/api/v3.
  • GITHUB_GRAPHQL - the graphql endpoint of your GitHub Enterprise deployment. Should be your.ghe.domain.com/api.
  • PORT - the local port at which the broker client accepts connections. Default is 7341.
  • BROKER_CLIENT_URL - the full URL of the broker client as it will be accessible by your GitHub Enterprise deployment webhooks, such as http://my.broker.client:7341

Command-line arguments

You can run the docker container by providing the relevant configuration:

docker run --restart=always \
           -p 8000:8000 \
           -e BROKER_TOKEN=secret-broker-token \
           -e GITHUB_TOKEN=secret-github-token \
           -e GITHUB=your.ghe.domain.com \
           -e GITHUB_API=your.ghe.domain.com/api/v3 \
           -e GITHUB_GRAPHQL=your.ghe.domain.com/api \
           -e PORT=8000 \
           -e BROKER_CLIENT_URL=http://my.broker.client:8000 \
       snyk/broker:github-enterprise

Derived docker image

Another option is to build your own docker image and override relevant environment variables:

FROM snyk/broker:github-enterprise

ENV BROKER_TOKEN      secret-broker-token
ENV GITHUB_TOKEN      secret-github-token
ENV GITHUB            your.ghe.domain.com
ENV GITHUB_API        your.ghe.domain.com/api/v3
ENV GITHUB_GRAPHQL    your.ghe.domain.com/api
ENV PORT              8000
ENV BROKER_CLIENT_URL http://my.broker.client:8000

Bitbucket Server

To use the the broker client with a Bitbucket Server deployment, run docker pull snyk/broker:bitbucket-server tag. The following environment variables are needed to customize the broker client:

  • BROKER_TOKEN - the snyk broker token, obtained from your Bitbucket Server integration settings view.
  • BITBUCKET_USERNAME - the Bitbucket Server username.
  • BITBUCKET_PASSWORD - the Bitbucket Server password.
  • BITBUCKET - the hostname of your Bitbucket Server deployment, such as your.bitbucket-server.domain.com.
  • BITBUCKET_API - the API endpoint of your Bitbucket Server deployment. Should be $BITBUCKET/rest/api/1.0.
  • BROKER_CLIENT_URL - the full URL of the broker client as it will be accessible by your Bitbucket Server for webhooks, such as http://my.broker.client:7341
  • PORT - the local port at which the broker client accepts connections. Default is 7341.

Command-line arguments

You can run the docker container by providing the relevant configuration:

docker run --restart=always \
           -p 8000:8000 \
           -e BROKER_TOKEN=secret-broker-token \
           -e BITBUCKET_USERNAME=username \
           -e BITBUCKET_PASSWORD=password \
           -e BITBUCKET=your.bitbucket-server.domain.com \
           -e BITBUCKET_API=your.bitbucket-server.domain.com/rest/api/1.0 \
           -e BROKER_CLIENT_URL=http://my.broker.client:8000 \
           -e PORT=8000 \
       snyk/broker:bitbucket-server

Derived docker image

Another option is to build your own docker image and override relevant environment variables:

FROM snyk/broker:bitbucket-server

ENV BROKER_TOKEN        secret-broker-token
ENV BITBUCKET_USERNAME  username
ENV BITBUCKET_PASSWORD  password
ENV BITBUCKET           your.bitbucket-server.domain.com
ENV BITBUCKET_API       your.bitbucket-server.domain.com/rest/api/1.0
ENV PORT                8000

Gitlab

To use the the broker client with Gitlab.com or an on-prem Gitlab deployment, run docker pull snyk/broker:gitlab tag. The following environment variables are needed to customize the broker client:

  • BROKER_TOKEN - the snyk broker token, obtained from your Gitlab integration settings view.
  • GITLAB_TOKEN - a Gitlab personal access token with api scope
  • GITLAB - the hostname of your Gitlab deployment, such as your.gitlab.domain.com or gitlab.com.
  • PORT - the local port at which the broker client accepts connections. Default is 7341.
  • BROKER_CLIENT_URL - the full URL of the broker client as it will be accessible by GitLab.com webhooks, such as http://my.broker.client:7341

Command-line arguments

You can run the docker container by providing the relevant configuration:

docker run --restart=always \
           -p 8000:8000 \
           -e BROKER_TOKEN=secret-broker-token \
           -e GITLAB_TOKEN=secret-gitlab-token \
           -e GITLAB=your.gitlab.domain.com \
           -e BROKER_CLIENT_URL=http://my.broker.client:8000 \
           -e PORT=8000 \
       snyk/broker:gitlab

Derived docker image

Another option is to build your own docker image and override relevant environment variables:

FROM snyk/broker:gitlab

ENV BROKER_TOKEN        secret-broker-token
ENV GITLAB_TOKEN        secret-gitlab-token
ENV GITLAB              your.gitlab.domain.com
ENV BROKER_CLIENT_URL   http://my.broker.client:8000
ENV PORT                8000

Jira

To use the the broker client with a Jira deployment, run docker pull snyk/broker:jira tag. The following environment variables are needed to customize the broker client:

  • BROKER_TOKEN - the snyk broker token, obtained from your Jira integration settings view.
  • JIRA_USERNAME - the Jira username.
  • JIRA_PASSWORD - the Jira password.
  • JIRA_HOSTNAME - the hostname of your Jira deployment, such as your.jira.domain.com.
  • BROKER_CLIENT_URL - the full URL of the broker client as it will be accessible by your Jira for webhooks, such as http://my.broker.client:7341
  • PORT - the local port at which the broker client accepts connections. Default is 7341.

Command-line arguments

You can run the docker container by providing the relevant configuration:

docker run --restart=always \
           -p 8000:8000 \
           -e BROKER_TOKEN=secret-broker-token \
           -e JIRA_USERNAME=username \
           -e JIRA_PASSWORD=password \
           -e JIRA_HOSTNAME=your.jira.domain.com \
           -e BROKER_CLIENT_URL=http://my.broker.client:8000 \
           -e PORT=8000 \
       snyk/broker:jira

Derived docker image

Another option is to build your own docker image and override relevant environment variables:

FROM snyk/broker:jira

ENV BROKER_TOKEN        secret-broker-token
ENV JIRA_USERNAME       username
ENV JIRA_PASSWORD       password
ENV JIRA_HOSTNAME       your.jira.domain.com
ENV PORT                8000

Monitoring

Healthcheck

The broker exposes an endpoint at /healthcheck, which can be used to monitor the health of the running application. This endpoint responds with status code 200 OK when the internal request is successful, and returns { ok: true } in the response body.

In the case of the broker client, this endpoint also reports on the status of the broker websocket connection. If the websocket connection is not open, this endpoint responds with status code 500 Internal Server Error and { ok: false } in the response body.

To change the location of the healthcheck endpoint, you can specify an alternative path via an environment variable:

ENV BROKER_HEALTHCHECK_PATH /path/to/healthcheck

Systemcheck

The broker client exposes an endpoint at /systemcheck, which can be used to validate the brokered service (SCM or the like) connectivity and credentials. This endpoint causes the broker client to make a request to a preconfigured URL, and report on the success of the request. The supported configuration is:

  • BROKER_CLIENT_VALIDATION_URL - the URL to which the request will be made.
  • BROKER_CLIENT_VALIDATION_AUTHORIZATION_HEADER - [optional] the Authorization header value of the request. Mutually exclusive with BROKER_CLIENT_VALIDATION_BASIC_AUTH.
  • BROKER_CLIENT_VALIDATION_BASIC_AUTH - [optional] the basic auth credentials (username:password) to be base64 encoded and placed in the Authorization header value of the request. Mutually exclusive with BROKER_CLIENT_VALIDATION_AUTHORIZATION_HEADER.
  • BROKER_CLIENT_VALIDATION_METHOD - [optional] the HTTP method of the request (default is GET).
  • BROKER_CLIENT_VALIDATION_TIMEOUT_MS - [optional] the request timeout in milliseconds (default is 5000 ms).

This endpoint responds with status code 200 OK when the internal request is successful, and returns { ok: true } in the response body. If the internal request fails, this endpoint responds with status code 500 Internal Server Error and { ok: false } in the response body.

To change the location of the systemcheck endpoint, you can specify an alternative path via an environment variable:

ENV BROKER_SYSTEMCHECK_PATH /path/to/systemcheck

Advanced Configuration

HTTPS

The broker client runs an HTTP server by default. It can be configured to run an HTTPS server for local connections. This requires an SSL certificate and a private key to be provided to the docker container at runtime.

For example, if your certificate files are found locally at ./private/broker.crt and ./private/broker.key, provide these files to the docker container by mounting the folder and using the HTTPS_CERT and HTTPS_KEY environment variables:

docker run --restart=always \
           -p 8000:8000 \
           -e BROKER_TOKEN=secret-broker-token \
           -e GITHUB_TOKEN=secret-github-token \
           -e PORT=8000 \
           -e HTTPS_CERT=/private/broker.crt \
           -e HTTPS_KEY=/private/broker.key \
           -e BROKER_CLIENT_URL=https://my.broker.client:8000 \
           -v /local/path/to/private:/private \
       snyk/broker:github-com

Note that BROKER_CLIENT_URL now has the HTTPS scheme.

SCM with an internal certificate

By default, the broker client establishes HTTPS connections to the SCM. If your SCM is serving an internal certificate (signed by your own CA), you can provide the CA certificate to the broker client.

For example, if your CA certificate is at ./private/ca.cert.pem, provide it to the docker container by mounting the folder and using the CA_CERT environment variable:

docker run --restart=always \
           -p 8000:8000 \
           -e BROKER_TOKEN=secret-broker-token \
           -e BITBUCKET_USERNAME=username \
           -e BITBUCKET_PASSWORD=password \
           -e BITBUCKET=your.bitbucket-server.domain.com \
           -e BITBUCKET_API=your.bitbucket-server.domain.com/rest/api/1.0 \
           -e PORT=8000 \
           -e CA_CERT=/private/ca.cert.pem \
           -v /local/path/to/private:/private \
       snyk/broker:bitbucket-server

Custom white-listing filter

The default white-listing filter supports the bare minimum to operate on all repositories supported by Snyk. In order to customize the white-listing filter, create the default one locally by installing snyk-broker and running broker init [SCM type]. The created accept.json is the default filter for the chosen SCM. Place the file in a separate folder such as ./private/accept.json, and provide it to the docker container by mounting the folder and using the ACCEPT environment variable:

docker run --restart=always \
           -p 8000:8000 \
           -e BROKER_TOKEN=secret-broker-token \
           -e GITHUB_TOKEN=secret-github-token \
           -e PORT=8000 \
           -e BROKER_CLIENT_URL=https://my.broker.client:8000 \
           -e ACCEPT=/private/accept.json
           -v /local/path/to/private:/private \
       snyk/broker:github-com

Misc