Add dedicated environment for buildbot - buildmaster

[adiroiban]

Scope

@glyph wants to have the buildmaster on a dedicated VM.

See http://twistedmatrix.com/pipermail/twisted-python/2016-January/030114.html

Changes

Create the new environment... which needs to be updated once the VM is created.

To do

• create the VM
• document the VM - ip, hosting, who can manage it (destroy/recrecate), who has root access
• update the environment with new hostname/ipaddress/admin-account
• add dump/restore command to get sqlite3 file from an installations.

How to test

Once the new VM is ready check that braid is able to star it.

fab config.buildbot base.bootstrap
fab config.buildbot services.buildbot

for now there are no dump/restore commands... not sure if we want this or we can do a manual copy :)

[tomprince]

There is actually (unused and mostly untested and entirely undocumented) code for doing this without having to create a seperate environment: see https://github.com/twisted-infra/braid/compare/buildbot-migration-poc

[adiroiban]

Why do we want to go for unused, untested and undocumented code?

Why do we need both hosts and roledefs ?

I will give it a try on this branch and see how it goes and if I can document it.

Thanks for the info

[glyph]

Why do we want to go for unused, untested and undocumented code?

The same reason anyone would: because it provides useful functionality.

Why do we need both hosts and roledefs ?

Because a "host" is a place where code runs, but a "roledef" is a description of the type of role that the host serves. For example, we might have a staging environment which has buildmaster, nameserver, trac, and postgres host, and then a production environment which also has a buildmaster, nameserver, trac and postgres host, each of which is different than the production version. I would like to tell Twisted production administrators to just use config.production for production, rather than memorizing what hosts go with which environments.

[tomprince]

Why do we want to go for unused, untested and undocumented code?

I'm not suggesting that it remain untested and undocumented, and I am suggesting that it become used. It is currently undocumented, just like (sadly) most of the rest of braid, when I stopped actively doing anything with it. The underlying fabric primitives are documented. The bit that is undocumented is passing a role to getTasks to run stuff in a role.

It is untested simply because it never ended up being used non-trivially.

[tomprince]

Why do we need both hosts and roledefs ?

This is just an artifact of how fabric views the world. The hosts entry acts like a default role, in that things will be run on that list of hosts, unless a task is set to run against a specific role, in which case that takes precedence.

http://docs.fabfile.org/en/1.10/usage/execution.html#defining-host-lists has lots of words on how things work.

It boils down to the fact that when given a role, getTasks wraps the tasks with @role, which means that there is a per-task role, so that hosts from the env dictionary are ignored.

---

If everything specified roles, then we wouldn't also need hosts set.

[adiroiban]

@glyph my comments regarding undocumented/untested code were made to see if Twisted sysadmins still want to continue to use braid ... we already have some ansible code and I was not sure if new things should be implemented using ansible or we should continue to develop braid.

---

Many thanks @tomprince for the info. I think that an inline comment should help document this.

@tomprince how do you suggest that we will handle this command for multiple hosts ?

$ fab base.bootstrap

Thanks!

[tomprince]

@tomprince how do you suggest that we will handle this command for multiple hosts ?

$ fab base.bootstrap

I don't know. I guess it partly depends on if all the hosts have an identical base setup. If they do, then perhaps that command should have a role of all, although that would make it always run against all hosts.

[hawkowl]

Fixed in 6de617f