New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[9.0][ADD] runbot_traefik: Traefik in Dockerized Runbot #133

Closed
wants to merge 17 commits into
base: 9.0
from

Conversation

Projects
None yet
4 participants
@lasley
Copy link
Member

lasley commented Jul 25, 2017

This is a WIP of a module allowing for Traefik load balancing in a Dockerized Runbot environment.

Depends:

lasley added some commits Jul 24, 2017

[IMP] runbot_travis2docker: Move run cmd to helper method
* Move the docker run command generation to a helper method to allow for better inherited usage
[ADD] runbot_traefik: Module for Traefik Load Balancing
* Create module for Traefik Load Balancing in a Dockerized environment

@lasley lasley added this to the 9.0 milestone Jul 25, 2017

@lasley lasley changed the title Release/9.0/runbot traefik [9.0][ADD] runbot_traefik: Traefik in Dockerized Runbot Jul 25, 2017

@pedrobaeza

This comment has been minimized.

Copy link
Contributor

pedrobaeza commented Jul 25, 2017

I'm eager to see some "how to" guides about your Runbot deployment 😉

@lasley

This comment has been minimized.

Copy link
Member

lasley commented Jul 25, 2017

Hah I'll let you know when I have a working one! At this point, I have a Docker-ception going on - with the Runbot controllers spawning their own Docker instances.

I plan on making a Rancher catalog, so hopefully there won't be a how-to needed & it'll just be a simple install

@lasley

This comment has been minimized.

Copy link
Member

lasley commented Jul 25, 2017

Just as an FYI this is my current compose file. The image is based off of the Tecnativa base too ;)

version: '2'

volumes:
  odoo-db-data:
    driver: local
  odoo-web-data:
    driver: local

services:

  web:
    image: lasley/runbot:9.0
    restart: unless-stopped
    links:
      - postgresql:db
    volumes:
      - odoo-web-data:/var/lib/odoo
      - /var/run/docker.sock:/var/run/docker.sock
      - /Users/dlasley/Documents/Repos/oca-runbot-addons/runbot_travis2docker:/opt/odoo/auto/addons/runbot_travis2docker
      - /Users/dlasley/Documents/Repos/oca-runbot-addons/runbot_traefik:/opt/odoo/auto/addons/runbot_traefik
    tty: true
    privileged: true
    ports:
      - 10080:8069
    environment:
      PGPASSWORD: 'odoo'
      PGUSER: 'odoo'
      ADMIN_PASSWORD: 'admin'

  postgresql:
    image: postgres:9.6-alpine
    restart: unless-stopped
    environment:
      PGDATA: /var/lib/postgresql/data/pgdata
      POSTGRES_PASSWORD: 'odoo'
      POSTGRES_USER: 'odoo'
    volumes:
      - odoo-db-data:/var/lib/postgresql/data/pgdata

It's worth noting that in a production deploy, the non-existent cron worker instances would be the only one with privilege to the host. There's still obviously gaping security holes - particularly because the privileged workers still have to be exposed to the interwebs, but that's Runbot for ya.

@pedrobaeza

This comment has been minimized.

Copy link
Contributor

pedrobaeza commented Jul 25, 2017

Thanks for the info, Dave. I would love to have Odoo experience nearer for having our own code sprint on infrastructure to complete the puzzle together, but well, meanwhile, we continue through PRs.

@pedrobaeza

This comment has been minimized.

Copy link
Contributor

pedrobaeza commented Jul 25, 2017

cc @Yajo

lasley added some commits Jul 25, 2017

@lasley lasley force-pushed the LasLabs:release/9.0/runbot_traefik branch from aae7464 to 960b313 Jul 26, 2017

@lasley

This comment has been minimized.

Copy link
Member

lasley commented Jul 26, 2017

As an update - it seems this works! There's some issues with Traefik & the HostRegexp that I just couldn't crack. It seems like a bug so I submitted containous/traefik#1897, but I'm willing to bet I'm just stupid and doing something wrong instead.

Below is a copy of my current stack. The Docker image is built from this.

services:
  cron:
    privileged: true
    image: lasley/runbot:9.0
    environment:
      PGPASSWORD: PGPASS
      PGUSER: odoo
      WAIT_NOHOST: install
    volumes:
    - odoo-web-data:/var/lib/odoo
    - /var/run/docker.sock:/var/run/docker.sock
    - runbot-builds:/opt/odoo/custom/src/odoo-extra/runbot/static
    tty: true
    links:
    - postgresql:db
    command:
    - /usr/local/bin/odoo
    - --max-cron-threads=1
    - --workers=1
    - --limit-time-real=600
    - --limit-time-cpu=300
    labels:
      io.rancher.container.pull_image: always
      io.rancher.container.hostname_override: container_name
  postgresql:
    image: postgres:9.6-alpine
    hostname: db
    environment:
      PGDATA: /var/lib/postgresql/data/pgdata
      POSTGRES_PASSWORD: PGPASS
      POSTGRES_USER: odoo
    volumes:
    - odoo-db-data:/var/lib/postgresql/data/pgdata
    labels:
      io.rancher.sidekicks: postgresql-utils
  web:
    image: lasley/runbot:9.0
    environment:
      ADMIN_PASSWORD: ADMINPASS
      PGPASSWORD: PGPASS
      PGUSER: odoo
      PROXY_MODE: 'true'
      WAIT_NOHOST: install
    volumes:
    - odoo-web-data:/var/lib/odoo
    - /var/run/docker.sock:/var/run/docker.sock
    - runbot-builds:/opt/odoo/custom/src/odoo-extra/runbot/static
    tty: true
    links:
    - postgresql:db
    ports:
    - 59880:8069/tcp
    command:
    - /usr/local/bin/odoo
    - --max-cron-threads=0
    - --workers=4
    labels:
      traefik.alias.fqdn: runbot.laslabs.io
      traefik.enable: 'true'
      traefik.port: '8069'
      traefik.domain: runbot.laslabs.io
      io.rancher.container.hostname_override: container_name
      io.rancher.container.pull_image: always
      traefik.frontend.passHostHeader: 'true'
  traefik:
    image: lasley/traefik-runbot
    stdin_open: true
    volumes:
    - /var/run/docker.sock:/var/run/docker.sock
    - traefik-certs:/opt/traefik/certs
    tty: true
    ports:
    - 53313:80/tcp
    - 42065:8080/tcp
    command:
    - --web
    labels:
      traefik.enable: 'true'
      io.rancher.scheduler.affinity:host_label: traefik_lb=true
      traefik.domain.regexp: '{subdomain:.+}.runbot.laslabs.io'
      traefik.port: '80'
      traefik.domain: runbot.laslabs.io
      io.rancher.container.pull_image: always
      traefik.frontend.passHostHeader: 'true'
  longpolling:
    image: lasley/runbot:9.0
    environment:
      PGPASSWORD: PGPASS
      PGUSER: odoo
      PROXY_MODE: 'true'
      WAIT_NOHOST: install
    volumes:
    - odoo-web-data:/var/lib/odoo
    - /var/run/docker.sock:/var/run/docker.sock
    - runbot-builds:/opt/odoo/custom/src/odoo-extra/runbot/static
    tty: true
    links:
    - postgresql:db
    ports:
    - 50921:8072/tcp
    command:
    - /usr/local/bin/odoo
    - --max-cron-threads=0
    - --workers=2
    labels:
      traefik.alias.fqdn: runbot.laslabs.io
      traefik.enable: 'true'
      traefik.path.prefix: /longpolling
      traefik.port: '8072'
      traefik.domain: runbot.laslabs.io
      io.rancher.container.hostname_override: container_name
      io.rancher.container.pull_image: always
      traefik.frontend.passHostHeader: 'true'
  postgresql-utils:
    image: grupocitec/pgutils
    volumes:
    - /tmp:/tmp
    labels:
      io.rancher.container.start_once: 'true'
'-l', 'traefik.enable=true',
'-l', 'traefik.frontend.rule=Host:%s;' % self._get_traefik_domain(),
'-l', 'traefik.frontend.passHostHeader=true',
'-l', 'traefik.port=8069',

This comment has been minimized.

@Yajo

Yajo Jul 26, 2017

Member

Do we use workers? If so, you need to change these rules.

This comment has been minimized.

@lasley

lasley Jul 26, 2017

Member

In the child runbot instances? Nah, pretty sure those are single threaded.

@moylop260 can you confirm on that?

This comment has been minimized.

@lasley

lasley Jul 26, 2017

Member

(This is with travis2docker)

This comment has been minimized.

@lasley

lasley Jul 27, 2017

Member

Confirmed no workers!

build_info = super(RunbotController, self).build_info(build)
if build.repo_id.is_traefik:
build_info.update({
'host': build.repo_id._domain(),

This comment has been minimized.

@lasley

lasley Jul 27, 2017

Member

Note to self: this is used in the template to show the worker, so that will need to be updated in the template
image

This comment has been minimized.

@lasley

lasley Jul 27, 2017

Member

(It's used more in the links, and we don't want to link directly to the worker)

@lasley

This comment has been minimized.

Copy link
Member

lasley commented Jul 27, 2017

Note to self- in demo mode, need to explicitly add an ir.config_param for the runbot domain so that the runbot config doesn't have to be run

moylop260 added a commit that referenced this pull request Aug 24, 2017

[REF] runbot_travis2docker: Adding new enrironment variable WEBLATE_S…
…SH (#134)

* [REF] runbot_travis2docker: Adding new enrironment variable WEBLATE_SSH (#133)

* [FIX] runbot_travis2docker: Fix the if bad condition (#134)

hbrunn added a commit that referenced this pull request Apr 10, 2018

[REF] runbot_travis2docker: Adding new enrironment variable WEBLATE_S…
…SH (#134)

* [REF] runbot_travis2docker: Adding new enrironment variable WEBLATE_SSH (#133)

* [FIX] runbot_travis2docker: Fix the if bad condition (#134)
@hbrunn

This comment has been minimized.

Copy link
Member

hbrunn commented May 31, 2018

I'll close this as it seems the docker people are on 11 by now

@hbrunn hbrunn closed this May 31, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment