Skip to content
Brad Smith edited this page May 1, 2015 · 58 revisions

Deployment

Overview

There are four ways in which the provisioning system might be used:

  • Deploying to a vagrant-based VM
    • Best for development and testing, unless you need a non-x86 architecture
  • Deploying to a cubietruck
    • x2go default for deployments to locations with no existing hardware, or power/space limitations
  • Deploying to a commodity PC
    • Tunapanda default for deployments to locations that already have hardware
  • Refreshing/reloading the config after deployment
    • Applies to all of the above methods
    • bootstrap.sh is run from /etc/rc.local at every boot (but skips most tasks if not net connection is detected)

Deploying to a vagrant VM:

  1. Install vagrant and virtualbox
  2. git clone https://github.com/tunapanda/provision
  3. cd provision
  4. See the Customization section below for ways you can fine-tune your install beyond the default setup.
  5. vagrant up
  6. To access the system, either:
  • vagrant ssh (CLI only, no password required)
  • A new VirtualBox window should appear during deployment. You can log in there with username teacher and password Tunapanda2. If you installed the GUI (see 'customization' below), you will have full desktop access here.
  1. See the Post-deplyment steps, below.

An important note about vagrant deployments: Normally, the deployment process clones the provision git repo into /usr/local/tunapanda/provision. In this case, you've already cloned the repo on your host system, so we can just use that. The Vagrantfile that controls the VM's behavior mounts the directory where you cloned the repo on your host as /usr/local/tunapanda/provision on the VM. This is why VM-based deployments can be very convenient for development. Just modify stuff in the repo and run vagrant provision to apply your changes.

Deploying to a cubietruck

edX requires several modifications to work on ARM. Thanks mainly to the work of Arturo Samanez and Ahmad Draidi, we have a prebuilt image of Ubuntu 14 for ARM (based on Igor Pecovnik's Cubietruck Ubuntu), with edX pre-installed. You will need to begin by using it to create an SD card from which to boot the cubie.

  1. (Optional) Normally, after the initial installation, several rsync processes will be spawned to start downloading site content in the background. You can also pre-download this content if it's easier for you to just do a few downloads overnight before you even deploy your cubietruck. Just download these archives and put them on a USB drive or something else you can use to extract them onto the cubietruck later:
  1. Download the x2go initial setup image
  • wget http://geekdome.net/x2go/cubietruck_trusty_edx/x2go_edx.img.bz2
    • This file is hosted on a server in the US. Outside of the US, you may get better performance downloading it from Google here.
  1. Write the image to your SD card
  • bzcat x2go_edx.img.bz2 | sudo dd bs=4096 of=/dev/sdX
    • Replace X with your device's letter. Run dmesg after inserting the card for hints if you're not sure.
    • This step can take 2+ hours to complete!
  1. Plug your cubietruck's wired ethernet adapter into the network so it can access the Internet for installations later.
  2. Boot your cubietruck from the SD card.
  3. The initial boot may take a while. It should configure its self to use either VGA or HDMI, whichever is plugged in during the initial boot, but you don't even need video because...
  4. When the cubie finishes booting, you should see a wireless access point called x2go.
  5. Connect to the x2go wifi network and ssh to root@10.0.0.1 with the password x2go
  6. Change the root password!
  • passwd
  1. (Optional) Use your cubietruck's SATA drive, if it has one. You have two options:
  • Store just the site data (course content, etc) on the SATA drive. This lets you re-OS and reprovision from scratch if you need to without having to re-download content that probably won't change between provisionings.
    1. Be sure you are OK losing all data currently on the drive!
    2. sudo mkfs.ext4 -L TUNAPANDA_DATA /dev/sda1
    • The provisioning script will cause any device with the label TUNAPANDA_DATA to be auto-mounted under /usr/local/tunapanda/data/.
  • Store everything possible on the SATA drive. This includes the OS and site data, but not the boot files, so the cubie will still need an SD card to boot from. We're still testing, but this may improve performance.
  1. ~/nand-sata-install
  2. Follow the instructions to install to /dev/sda1
  3. Note: Leave the SD card in the cubietruck. The cubie will still need it to boot, even if most data has been moved to the SATA drive.
  4. Reboot, and log in again.
  5. (Optional) Expand the previously-downloaded content (if you downloaded it in the first step).
  6. mkdir -p /usr/local/tunapanda/data/portal_modules
  7. cd /usr/local/tunapanda/data/portal_modules
  8. Expand the archives into this directory
  9. Update the provisioning scripts
  10. cd /usr/local/tunapanda/provision
  11. git pull Important! Don't skip this!
  12. Customize your deployment
  13. cd /usr/local/tunapanda/provision
  14. cp localconfig.yml-sample localconfig.yml
  15. Edit localconfig.yml and at least set the provision__site and provision__instance vars. * If you have limited storage space (e.g. no SATA drive), you may also wish to include wikipedia__enabled: false and/or kalite__enabled: false, since kalite and wikipedia require more space than other services.
  16. Start the main provisioning
  17. scripts/bootstrap.sh
  18. Go to post-deployment steps.

Deploying to a commodity PC

Note: These instructions assume a machine already running Ubuntu 14 (other Debian-like distros will probably also work, but have not been tested).

Really important note: Following these instructions will completely reconfigure the system. Don't do this on a personal machine you don't plan to use as a dedicated server. If you're just curious, use the instructions for deploying on a virtual machine instead!

  1. cd /tmp
  2. wget https://github.com/tunapanda/provision/blob/master/scripts/bootstrap.sh
  3. sudo bash bootstrap.sh # add -x if troubleshooting

At this point some basic configuration been done, but mostly we've just set up the /usr/local/tunapanda/provisioning directory and created some users. To customize your deployment further, see Customizing below.

Post-deployment steps

  • If provisioning failed, run scripts/diag.sh, and email the resulting archive to brad@tunapanda.org.
  • If things don't seem to be working after the provisioning process completes, try rebooting the cubietruck.
    • The provisioning process starts a set of background rsync processes to download files for most of the content modules linked from the main page, but if you reboot they can die and will need to be re-started. You can do this by running /usr/local/tunapanda/data/cron_datasync/syncscripts.d/* as root
  • In order for the content modules to be searchable, the database for the web crawler sphider must be populated. You can do this by running the following as root on the cubie.
    1. wget http://geekdome.net/x2go/cubietruck_trusty_edx/sphider_plus.sql.bz2
    2. bzcat sphider_plus.sql.bz2 | mysql sphider_plus
  • PLEASE REPORT BUGS!. Email Brad Smith with questions and comments.

Customizing

First, a very brief overview of Ansible, the system we use to configure systems during deployment. You will need to understand at least three concepts: roles, playbooks, and variables.

  • A role describes how to deploy a specific service or machine type. Most of the roles we use can be found in playbooks/roles/.
  • A playbook describes which roles to load, among other things. The file matched by playbooks/*.yml are all playbooks. By default, when you run scripts/bootstrap.sh, it uses playbooks/main.yml for deployment instructions.
  • A variable is a setting that can control the behavior of roles and playbooks.

Simple customization

If all you need to do is customize which services are installed, how many users are created, the system's hostname, etc, the easiest way to do this is with a localconfig.yml.

  1. cd /usr/local/tunapanda/provision
  2. cp localconfig.yml-sample localconfig.yml
  3. Open localconfig.yml in a text editor and make whatever customizations you need. The comments in the file should explain everything, but basically there are two things you can define here:
  • The groups: section lets you select one or more "groups" for your deployment. A group is a convenient collection of variables, and sometimes custom playbooks, for a particular deployment scenario.
    • Each group has variables are set playbooks/group_vars/$GROUPNAME.
      • ...so, to get a list of available groups, run ls playbooks/group_vars/.
    • Groups may also have a custom playbook at playbooks/$GROUPNAME.yml. Otherwise, it just loads all roles enabled in the vars file.
    • Note that if you are deploying on a cubietruck (or really anything with an ARM architecture), the provisioning system will automatically use the cubietruck group by default.
  • The vars: section lets you override any setting, including the ones set by your selected groups.
    • The most common vars to overide are provision__site and provision__instance, as described in the comments.
    • You can also enable or disable any role by setting $ROLENAME__enabled (note the two '_' characters) to true or false.
    • To see all of the variables for a role, look in playbooks/roles/$ROLENAME/defaults/main.yml

You can also create a localconfig.yml with all available variables set to their default values like this:

(echo "---" ; echo "groups: [ default ]"; echo "vars:" ; for r in $(ls playbooks/roles/) ; do echo "" ; echo "  ## $r role" ; cat playbooks/roles/$r/defaults/main.yml | grep -v '^---' | grep -v '^[[:space:]]*$' | sed 's,^,  ,'; done) > localconfig.yml

To apply your changes, run sudo scripts/bootstrap.sh

Advanced customization

If the file playbooks/custom.yml exists, bootstrap.sh will use it instead of the default. If you know what you're doing with Ansible, you can use this to create your own custom playbook. Normally, the best profile is automatically chosen based on hardware architecture, etc, but if you want to (and you know what you're doing!), you can fine-tune things with a special profile called custom, and the localconfig.yml file. Note that this file does not exist by default, but you can create one using localconfig.yml.sample as a template, or using the following command, which will create a config file with all available settings set to their default values:

To apply your changes, run sudo scripts/bootstrap.sh

Creating an ISO

Note: this feature hasn't been tested much recently. If you encounter problems, please file an issue with information on how you used it and what went wrong

You can generate a bootable DVD of the provisioned system, which can then be used to demo or install a duplicate of the system elsewhere, by adding build_iso__enabled: true to your localconfig.yml.

Clone this wiki locally