Running Nupic in a Virtual Machine

Matthew Taylor edited this page Sep 5, 2015 · 42 revisions
Clone this wiki locally

DISCLAIMER: The preferred VM installation method is with docker. This document may be wrong or out-of-date.


Numenta provides a Vagrant virtual machine instance you can download and run locally on OS X, Windows, or Linux. It will provide an environment guaranteed to be set up to run NuPIC without any fiddling. You do not need to configure your own VM. A working VM is provided below.

There're also instructions How-to run NuPIC on Amazon EC cloud

Another option is completely preset 1-click VM image

Vagrant method

Instructions

These instructions are detailed, but they are not very hard to accomplish. Don't be intimidated by the number of steps. :)

  1. Download and install Virtual Box. 4.3.0 is known to work with Vagrant 1.3.5 on OS X 10.9 Mavericks as of 2013-10-23.
  2. Download and install Vagrant one of the following ways (on your host OS):
    • download and install from the Vagrant website
    • if you run Vagrant in Windows, you can run vagrant via MINGW32 or Cygwin. Alternately, there's a .bat file that works directly from Windows, but you'll want to be careful when handling line endings if checking out NuPIC from Windows and then running it from the VM
    • Do not sudo gem install vagrant on *nix or Mac OS X: the rubygems repositories have an old version of vagrant which can't deal with the nupic box file, so you'll do best to download the binary from the above link.
  3. Make a directory to run Vagrant and cd into that directory
    • The default .bashrc configuration assumes that the NuPIC repo is located on the VM at "/vagrant". By default, your host machine's Vagrant directory will be shared to the VM as "/vagrant". So it's most convenient if you use a NuPIC repo as your Vagrant directory. (but these paths can be modified if you want)
      • If you're on Unix, simply run: git clone https://github.com/numenta/nupic.git
      • If you're on Windows, you need to avoid git's auto-conversion to DOSfiles, since you'll be running these files inside of a Linux VM. If you're cool with modifying your global settings, run:
        git config --global core.autocrlf false
        git clone https://github.com/numenta/nupic.git
        or if you want to preserve your global settings, run:
        md nupic
        cd nupic
        git init
        git config core.autocrlf false
        git remote add origin https://github.com/numenta/nupic.git
        git fetch
        git checkout master
  4. vagrant init : This will create Vagrantfile in the directory.
  5. Edit Vagrantfile:
  • change line 10: config.vm.box = "base" to config.vm.box = "nupic"
  • Uncomment the setting config.vm.box_url (line 14) and set it to the following value. This is the URL of the actual virtual machine we have hosted on Amazon S3:
    • config.vm.box_url = "https://s3-us-west-2.amazonaws.com/public.numenta.org/vagrant/nupic-current.box"
  1. Edit any other VM settings in Vagrantfile (you might want to change the CPUs and RAM allocation, or port forwarding).
  2. vagrant up
  • downloads the VM from Amazon S3 (it's 635MB, so sit tight)
  • brings up the VM locally
  • may take a long time, depending on your internet connection
  1. The default settings in Vagrantfile will make any files inside the current directory accessible from within the vm in the /vagrant directory
  2. Expose NuPIC codebase to the vagrant instance. If you used your repo as the Vagrant directory, then you've already completed this step. Otherwise, move / copy your repo into the Vagrant folder, or follow instructions above to check out from Github
  3. Connect to the VM: vagrant ssh. On Windows, follow the instructions here for using Putty and PuttyGen, though you may have an SSH client available if Git is in your %PATH%

After connecting, you will be in the /home/vagrant directory on the virtual machine. Change directories to /vagrant directory, which should contain the NuPIC codebase. Follow the directions in the README.md from here, and everything should run swimmingly.

Tips

  • suspend your VM with vagrant suspend
  • destroy your VM with vagrant destroy

Both of the following snippets require that you be using a version 2 Vagrantfile. You can tell that you have a V2 configuration file if the following line is at the top of your Vagrantfile.

Vagrant.configure("2") do |config|

More Memory

config.vm.provider :virtualbox do |vb|
  # Use VBoxManage to customize the VM. For example to change memory:
  vb.customize ["modifyvm", :id, "--memory", "4096"]
  vb.customize ["modifyvm", :id, "--cpus", "4"]
end

Dynamically assign Memory and CPUs to the VM.

To be able to dynamically set CPU and RAM by setting environment variables, add this snippet to your Vagrantfile:

config.vm.provider :virtualbox do |vb|
  # Don't boot with headless mode
  # vb.gui = true

  if ENV.has_key?('VAGRANT_RAM')
    vb.customize ["modifyvm", :id, "--memory", ENV['VAGRANT_RAM']]
  end
  # Set CPUS
  if ENV.has_key?('VAGRANT_CPUS')
    vb.customize ["modifyvm", :id, "--cpus", ENV['VAGRANT_CPUS']]
  end
end

Please note that this does no sanity checking, it hands your settings off to VirtualBox and lets it deal with them.

Preset VM image (Outdated)


NOTE: This is a severely outdated VM image. Don't waste your time, like I did. If you are just getting started, you should build a clean VM with Ubuntu 14.04, and follow the instructions here: https://github.com/numenta/nupic/wiki/Installing-NuPIC-on-Ubuntu


This is really a 1-click and run method, but the download is 6GB.

Works for Virtualbox, VM Player or any other virtualization solution (Win, Linux, OSX). It's a prepared Ubuntu (13.04) with all the NuPIC repositories (and some more), development tools and Netbeans IDE (tuned for C++/Python development), also contains ROS packages.

Download: https://mega.co.nz/#!vkBH0CZA!Kr5LgbBBh-d3LQnyFHA5evXiE25dFcG8nlqLn6Vf61c

Details

NuPIC development platform on Ubuntu 13.04 64bit,

Features

  • installed NuPIC (core HTM CLA)
  • Cerebro (CLA visualisation and debugging)
  • swarming (parameter tuning for Nupic)
  • Netbeans IDE with C++/Python support
  • all dependencies for devel (python 2.6,2.7, gcc 4.7, git)
  • XChat IRC (set to #nupic channel)
  • cppcheck
  • ROS (robotic OS)

all tested and working out-of-the box

CHANGE:

as a user, you can tune these settings:

  • change you name & email in git (for commits under your name)
  • change nick on IRC (so we're not all 'nupic' ;) )
  • in VM settings, you can allocate more RAM/CPUs (but this is a sane default)
  • in VM settings, set up shared folders

..happy hacking! ;)


Changelog:

== 3/11/2014 ==

  • updated NuPIC to the new repo structure - nupic.core (C++ core) and nupic (python base)
  • updated system and ROS packages
  • Netbeans updated to 7.4 (installed locally, not from a PPA repository, updates happen through netbeans IDE)
  • started the README
  • added some (not complete) 3rd-party repos

== 9/17/2013 ==

  • initial release of the VM

FAQ:

Q: Why Ubuntu 13.04 (when 13.10 is the most recent stable)

A: Because of ROS packages are for 13.04 only so far.

DIRECTORY STRUCTURE:

...what you see here :)

~/nupic-soure -- NuPIC repo (python, most code)
~/nupic.core  -- C++ core of NuPIC, requirement for nupic
~/nupic.cerebro -- visualization tool for CLA
~/nupic-linux64 -- python dependencies for NuPIC, not required as all are preinstalled here for you
~/nupic-3rdparty -- unofficial repos with (useful) projects
--  breznak-nupic  -- some of my not mainline branches (ROS, utility,..)
--  breznak-alife  -- research of autonomous agents with CLA
--  chetan-linguist -- text learning/generation app, NLP
--  nupic-fluent   --  NLP platform with CEPT!
--  hackathon1:
	- ravaa/predipic -- IR sensor predictions
	- karchie	 -- prediction charting
	- tobedetermined/streamingweather -- weather predictions
	- spurdy/cpu	-- predicting CPU usage
	- snikolov/song -- song learning & predictions
--  hackathon2