This document is written in the Markdown format and contains some inline HTML. This document is also available online at https://github.com/karesansui/karesansui/blob/master/INSTALL.md.
This document is based on CentOS 6.x (x86_64). However, following the similar steps described within the document allows you to install Karesansui on any other Linux distribution.
- Taizo Ito <taizo at karesansui-project.info>
Thanks to the volunteer efforts of contributors.
- Mihai Moldovan <ionic at ionic.de> - mod_fcgid configuration hints.
Karesansui only works on systems that have the virtualization system installed. So you need to enable KVM (Kernel-based Virtual Machine).
Just do usual installation, but be careful of the points below:
- (Required) On software setup, check "Virtual Host" and "Customize Now"
- (Required) On software group setup, check "Virtualization" - "Virtualization","Virtualization Client","Virtualization Platform" ,"Virtualization Tools"
If you want to enable KVM after installing OS, you need to execute the following commands:
A hash (#) prompt indicates you are logged-in as the root user; a dollar ($) prompt indicates you are logged-in as the specific general account.
###For CentOS 6:
# yum groupinstall "Virtualization" "Virtualization Client" "Virtualization Platform" "Virtualization Tools"
# /sbin/modprobe -b kvm-intel (or /sbin/modprobe -b kvm-amd)
# /sbin/modprobe -b vhost_net
Please make sure that the kernel modules for KVM are loaded.
# /sbin/lsmod | grep kvm
kvm_intel 50380 0
kvm 305081 1 kvm_intel
*If you use an AMD processor, kvm_amd must be loaded instead of kvm_intel.
After installing Linux, the first Ethernet interface is typically identified as eth0 and allows only outbound communication from the KVM guests. To share the interface with KVM guests and enable full network access, including communication to and from an external host, you need to set up a Linux bridge in Linux system.
Note: If you are accessing the server via SSH or Telnet instead of console, you MAY be disconnected when you restart the network service after modifying network settings. You should configure the settings via the local console.
###Procedure for CentOS 6:
####1. Create the network script defining a Linux bridge associated with the network card.
The script file path is /etc/sysconfig/network-scripts/ifcfg-br0, where br0 is the name of the bridge.
# cp /etc/sysconfig/network-scripts/ifcfg-{eth0,br0}
####2. Edit the script file for br0 (/etc/sysconfig/network-scripts/ifcfg-br0)
If your network card is configured with a static IP address, your original network script file should look similar to the following example:
DEVICE=eth0
HWADDR=<the ethernet hardware address for this device>
ONBOOT=yes
IPADDR=<the IP address>
BOOTPROTO=static
NETMASK=<the netmask>
TYPE=Ethernet
You need to edit ifcfg-br0 as shown in the following example.
DEVICE=br0 # <- Changed
#HWADDR=<the ethernet hardware address for this device> # <- Commented out
ONBOOT=yes
IPADDR=<the IP address>
BOOTPROTO=static
NETMASK=<the netmask>
TYPE=Bridge # <- Changed
####3. Edit the script file for eth0 (/etc/sysconfig/network-scripts/ifcfg-eth0)
Now you need to configure your network script for eth0. You will already have a script for eth0, but you’ll need to modify it by adding one line as BRIDGE=br0 so that it looks similar to the following script.
DEVICE=eth0
HWADDR=<the ethernet hardware address for this device>
ONBOOT=yes
#IPADDR=<the IP address> # <- Commented out
#BOOTPROTO=none # <- Commented out
#NETMASK=<the netmask> # <- Commented out
TYPE=Ethernet
BRIDGE=br0 # <- Added
####4. Restart network services
In order for all the network script modifications to take effect, you need to restart your network services.
# /etc/init.d/network restart
####5. Check the status of current interfaces
# /sbin/ifconfig -a
###Alternate Procedure:
####1. Activate promiscuous mode on eth0.
# /sbin/ifconfig eth0 0.0.0.0 promisc up
####2. Create the bridge interface.
# /usr/sbin/brctl addbr br0
####3. Add interfaces to the bridge.
# /usr/sbin/brctl addif br0 eth0
####4. Put up the bridge.
# ifconfig br0 up
####5. Configure the virtual bridge interface to take part in your network.
# ifconfig br0 172.23.233.1 netmask 255.255.255.0
# route add default gw 172.23.233.254
To install and set up Karesansui, you need to install it's dependent software/libs first. You can install most of them by using the software updater provided by each distribution, but some software need to be built on your machine.
###For CentOS 6:
#####Fetching packages from CentOS-Base repository.
# yum install PyXML python-mako python-sqlalchemy python-simplejson rrdtool rrdtool-python
#####Fetching packages from EPEL(Extra Packages for Enterprise Linux) repository.
# wget ftp://ftp.iij.ad.jp/pub/linux/fedora/epel/6/x86_64/epel-release-6-7.noarch.rpm
# rpm -Uvh epel-release-6-7.noarch.rpm
# yum install python-flup python-sqlite2
# yum install collectd collectd-ping collectd-rrdtool collectd-virt
#####Building packages that are not provided by the official or third party repositories.
######Step 1. Setting up RPM build environment.
Create a separate account for building RPMs and set up the environment for it:
# yum install rpm-build
# useradd rpmbuild
# su - rpmbuild
$ mkdir -p ~/pkgs/{BUILD,RPMS/{i{3,4,5,6}86,x86_64,noarch},SOURCES,SPECS,SRPMS}
$ echo '%_topdir %(echo $HOME)/pkgs' > ~/.rpmmacros
######Step 2. Fetching Karesansui source code from github repository.
# yum install git python-setuptools
# su - rpmbuild
$ git clone git://github.com/karesansui/karesansui.git
Now you have karesansui source code under ~rpmbuild/karesansui with our sample spec files to build RPM packages.
######Step 3. Building python-webpy package.
# su - rpmbuild
$ cd ~/pkgs/SOURCES/
$ wget http://webpy.org/static/web.py-0.37.tar.gz
$ rpmbuild -ba ~/karesansui/sample/specs/python-webpy/python-webpy.spec
######Step 4. Building tightvnc-java package.
# su - rpmbuild
$ cd ~/pkgs/SOURCES/
$ wget http://downloads.sourceforge.net/sourceforge/vnc-tight/tightvnc-1.3.10_javabin.tar.gz
$ wget http://downloads.sourceforge.net/sourceforge/vnc-tight/tightvnc-1.3.10_javasrc.tar.gz
$ rpmbuild -ba ~/karesansui/sample/specs/tightvnc-java/tightvnc-java.spec
######Step 5. Installing the newly built packages.
Now you can install the newly built packages.
$ cd ~/pkgs/RPMS/noarch
# rpm -Uvh python-webpy-*.el6.noarch.rpm tightvnc-java-*.el6.noarch.rpm
Pysilhouette is a python-based background job manager, intended to co-work with various python-based web applications such as Karesansui. It makes it available to get job status to programmers, which was difficult in http-based stateless/interactive session before. It is also developed by Karesansui Project Team.
###Procedure for CentOS 6:
####1. Fetching pysilhouette source code from github repository.
# su - rpmbuild
$ git clone git://github.com/karesansui/pysilhouette.git
####2a. (Method 1) Building pysilhouette package and installing it.
$ cd ~/pysilhouette
$ python setup.py sdist
$ rpmbuild -ta dist/pysilhouette-*.tar.gz
# rpm -Uvh ~rpmbuild/pkgs/RPMS/noarch/pysilhouette-*.noarch.rpm
####2b. (Method 2) Installing pysilhouette with Python's distutils.
$ cd ~/pysilhouette
$ python setup.py build
# python setup.py install --root=/ --record=INSTALLED_FILES
#### Create pysilhouette account ####
# /usr/sbin/useradd -c "Pysilhouette" -s /bin/false -r pysilhouette
#### Create the application's system directories ####
# mkdir /etc/pysilhouette
# mkdir /var/log/pysilhouette
# mkdir /var/lib/pysilhouette
#### Copy several programs, configuration files and SysV init script ####
# cp -f ~rpmbuild/pysilhouette/sample/rc.d/init.d/* /etc/rc.d/init.d/
# cp -f ~rpmbuild/pysilhouette/sample/sysconfig/silhouetted /etc/sysconfig/silhouetted
# cp -f ~rpmbuild/pysilhouette/sample/log.conf.example /etc/pysilhouette/log.conf
# cp -f ~rpmbuild/pysilhouette/sample/silhouette.conf.example /etc/pysilhouette/silhouette.conf
# cp -f ~rpmbuild/pysilhouette/sample/whitelist.conf.example /etc/pysilhouette/whitelist.conf
# ln -s `python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"`/pysilhouette/silhouette.py /usr/bin
# cp -f ~rpmbuild/pysilhouette/tools/psil-cleandb /usr/sbin
# cp -f ~rpmbuild/pysilhouette/tools/psil-set /usr/sbin
# chmod 0744 /usr/sbin/psil-*
#### Modify the following files if necessary.
## vi /etc/rc.d/init.d/silhouetted
## vi /etc/sysconfig/silhouetted
###Procedure for CentOS 6:
####1. Fetching karesansui source code from github repository.
# su - rpmbuild
$ git clone git://github.com/karesansui/karesansui.git # No need if already downloaded.
####2a. (Method 1) Building karesansui package and installing it.
$ cd ~/karesansui
$ python setup.py sdist
$ rpmbuild -ta dist/karesansui-*.tar.gz
# rpm -Uvh ~rpmbuild/pkgs/RPMS/noarch/karesansui-{,{bin,data,gadget,lib,plus}-}3.*.noarch.rpm
####2b. (Method 2) Installing karesansui with Python's distutils.
$ cd ~/karesansui
$ python setup.py build
# python setup.py install --record=INSTALLED_FILES --install-scripts=/usr/share/karesansui/bin
#### Create kss account ####
# /usr/sbin/useradd -c "Karesansui Project" -s /bin/false -r -m -d /var/lib/karesansui kss
# gpasswd -a qemu kss
#### Create the application's system directories ####
# mkdir /etc/karesansui/virt
# mkdir /var/log/karesansui
# mkdir -p /var/lib/karesansui/{tmp,cache}
#### Change attributes of the application's directories/files ####
# chgrp -R kss /etc/karesansui
# chmod g+rwx /etc/karesansui/virt
# chmod o-rwx /etc/karesansui/virt
# chmod -R g+rw /etc/karesansui
# chmod -R o-rwx /etc/karesansui
# chgrp -R kss /var/log/karesansui
# chmod -R 0700 /var/log/karesansui
# chgrp -R kss /var/lib/karesansui
# chmod -R 0770 /var/lib/karesansui
# find /usr/share/karesansui/ -type d -exec chgrp -R kss \{\} \;
# find /usr/share/karesansui/ -type d -exec chmod g+rwx \{\} \;
#### Copy several programs, configuration files and SysV init script ####
# cp -f ~rpmbuild/karesansui/sample/application.conf.example /etc/karesansui/application.conf
# cp -f ~rpmbuild/karesansui/sample/log.conf.example /etc/karesansui/log.conf
# cp -f ~rpmbuild/karesansui/sample/service.xml.example /etc/karesansui/service.xml
# cp -f ~rpmbuild/karesansui/sample/logview.xml.example /etc/karesansui/logview.xml
# cp -f ~rpmbuild/karesansui/sample/cron_cleantmp.example /etc/cron.d/karesansui_cleantmp
# cp -f ~rpmbuild/karesansui/sample/whitelist.conf.example /etc/pysilhouette/whitelist.conf
You may need to modify the following configuration files.
| File name | Description |
|---|---|
| /etc/pysilhouette/silhouette.conf | Configuration file for the silhouetted daemon. |
| /etc/pysilhouette/log.conf | Configuration file for logging. |
| /etc/pysilhouette/whitelist.conf | Contains a whitelist of commands that can be executed by the pysilhouette as a background job. |
| /etc/sysconfig/silhouetted | Used to set some other bootup parameters. |
Add pysilhouette as a service and enable it for auto start at bootup.
# /sbin/chkconfig --add silhouetted
# /sbin/chkconfig silhouetted on
You may need to modify the following configuration files.
| File name | Description |
|---|---|
| /etc/karesansui/application.conf | Configuration file for karesansui(*You need to set 'application.uniqkey' parameter. Its value is supposed to be generated by the uuidgen program.) |
| /etc/karesansui/log.conf | Configuration file for logging. |
| /etc/karesansui/service.xml | Used to define services related to karesansui. |
| /etc/karesansui/service.xml | Used to define log files related to karesansui. |
| /etc/cron.d/karesansui_cleantmp | Configuration file for the cron daemon. Clean/delete unnecessary temporary files at a specific interval. |
With a script bundled with the source code, you can create a database for karesansui and insert information of the administrator into the database.
# python ~rpmbuild/karesansui/tools/initialize_database.py -m <Administrator's E-mail Address> -p <Administrator's Password> -l en_US
If you use a SQLite database, you must change the attributes of the database file.
# chgrp -R kss /var/lib/karesansui/
# chmod -R g+rw /var/lib/karesansui/
# chmod -R o-rwx /var/lib/karesansui/
You can create a database for pysilhouette by executing the following command:
# python -c "import karesansui; from pysilhouette.prep import readconf; karesansui.sheconf = readconf('/etc/pysilhouette/silhouette.conf'); import karesansui.db._2pysilhouette; karesansui.db._2pysilhouette.get_metadata().create_all()"
If you use a SQLite database, you must change the attributes of the database file.
# chgrp -R kss /var/lib/pysilhouette/
# chmod -R g+rw /var/lib/pysilhouette/
# chmod -R o-rwx /var/lib/pysilhouette/
# /etc/init.d/silhouetted start
####1. Edit libvirt configuration file
You may need to modify the following configuration files.
/etc/libvirt/libvirtd.conf
- listen_tcp = 1
- tcp_port = "16509"
- unix_sock_group = "kss"
- auth_tcp = "none"
/etc/sysconfig/libvirtd
- LIBVIRTD_ARGS="--listen"
####2. Create directories for libvirt
# mkdir -p /var/lib/libvirt/{disk,domains,snapshot}
# chgrp -R kss /var/lib/libvirt
# chmod -R 0770 /var/lib/libvirt
# chgrp -R kss /etc/libvirt
# find /etc/libvirt -type d -exec chmod g+rwx \{\} \;
####3. Create a TLS certificates for libvirt
First, set up a Certificate Authority (CA).
$ certtool --generate-privkey > cakey.pem
$ vi ca.info
cn = Name of your organization
ca
cert_signing_key
$ certtool --generate-self-signed --load-privkey cakey.pem --template ca.info --outfile cacert.pem
Issue server certificates.
$ certtool --generate-privkey > serverkey.pem
$ vi server.info
organization = Name of your organization
cn = Your FQDN
tls_www_server
encryption_key
signing_key
$ certtool --generate-certificate --load-privkey serverkey.pem --load-ca-certificate cacert.pem --load-ca-privkey cakey.pem --template server.info --outfile servercert.pem
Install the certificates.
# mkdir -p /etc/pki/libvirt/private/
# cp -i cacert.pem /etc/pki/CA/
# cp -i servercert.pem /etc/pki/libvirt/
# cp -i serverkey.pem /etc/pki/libvirt/private/
####4. Enable libvirtd service
Restart service and enable it for auto start at bootup.
# /sbin/chkconfig libvirtd on
# /etc/init.d/libvirtd restart
####5. Check for connectivity to libvirtd
Please make sure that virsh client can connect to the QEMU monitor with libvirt.
# virsh -c qemu+tcp://localhost:16509/system list
If the connection attempt succeed, it will display message as below:
Id Name State
----------------------------------
####6. Create the default storage pool for libvirt
# KARESANSUI_CONF=/etc/karesansui/application.conf python -c "from karesansui.prep import built_in; built_in()"
# /usr/share/karesansui/bin/create_storage_pool.py --name=default --target_path=/var/lib/libvirt/domains --mode=0770 --owner=0 --group=`id -g kss` --type=dir
# virsh pool-refresh default
# rm -fr /var/log/karesansui/*
You have now finished setting up Karesansui itself.
Confirm that you can access the Karesansui URL from your browser through the app's python-webpy built-in web server.
First, run the karesansui program using the built-in server.
# su -s /bin/bash kss -c "KARESANSUI_CONF=/etc/karesansui/application.conf SEARCH_PATH= /usr/share/karesansui/bin/karesansui.fcgi"
If the setting up is successful, it always seems to display the following:
http://0.0.0.0:8080/
Please access the following URL from your browser after starting the server. The username and password values for HTTP basic authentication must match the ones specified during database creation 'Creating database for karesansui'.
http://[your-server-name]:8080/karesansui/v3/
If the console is displayed correctly, the installation appears to be completed successfully.
###Procedure for CentOS 6:
####1. Installing packages.
Install several packages from EPEL(Extra Packages for Enterprise Linux) repository.
# wget ftp://ftp.iij.ad.jp/pub/linux/fedora/epel/6/x86_64/epel-release-6-7.noarch.rpm
# rpm -Uvh epel-release-6-7.noarch.rpm
# yum install lighttpd lighttpd-fastcgi spawn-fcgi
####2. Configuring group members.
Add lighttpd user to kss group and kss user to lighttpd group.
# gpasswd -a lighttpd kss
# gpasswd -a kss lighttpd
####3. Configuring lighttpd settings.
Append the line below to /etc/lighttpd/lighttpd.conf.
include "conf.d/karesansui.conf"
Edit /etc/lighttpd/modules.conf to enable the following modules.
mod_alias
mod_rewrite
mod_fastcgi
Copy the sample configuration file bundled with the source code to the location of lighttpd config directory, and edit it if necessary.
# cp ~rpmbuild/karesansui/sample/lighttpd/karesansui.conf /etc/lighttpd/conf.d/
####4. Setting up a simple SSL configuration with lighttpd
# mkdir -p /etc/lighttpd/ssl
# openssl req -new -x509 -keyout /etc/lighttpd/ssl/server.pem -out /etc/lighttpd/ssl/server.pem -days 3650 -nodes
# chmod 400 /etc/lighttpd/ssl/server.pem
####5. Start web service
If you have already tried to run Karesansui with other web server, you need to remove existing files with the following command:
# rm -fr /var/log/karesansui/*log
Restart service and enable it for auto start at bootup.
# /sbin/chkconfig lighttpd on
# /etc/init.d/lighttpd restart
If SELinux is set to "Enforcing", lighttpd may not work properly. Set it to "Permissive" mode and try to restart the service again.
# /usr/sbin/setenforce 0
# /etc/init.d/lighttpd restart
Moreover, if you need to use "Permissive" mode at the next reboot, you have to modify /etc/selinux/config file as well.
SELINUX=permissive
####6. Checking for connectivity to Karesansui management console
Please access the following URL from your browser after starting the server. The username and password values for HTTP basic authentication must match the ones specified during database creation 'Creating database for karesansui'.
https://[your-server-name]/karesansui/v3/
###Procedure for CentOS 6:
####1. Installing packages.
Install several packages from RepoForge repository.
# wget http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm
# rpm -Uvh rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm
# yum install httpd mod_fastcgi
If mod_fcgid cannot be found in the third-party repository, so you would have to compile it on your own.
####2. Configuring group members.
Add apache user to kss group and kss user to apache group.
# gpasswd -a apache kss
# gpasswd -a kss apache
####3. Configuring apache settings.
Copy the sample configuration file bundled with the source code to the location of httpd config directory, and edit it if necessary.
# cp ~rpmbuild/karesansui/sample/apache/fastcgi.conf /etc/httpd/conf.d/
# cp ~rpmbuild/karesansui/sample/apache/fcgid.conf /etc/httpd/conf.d/
####4. Start web service
If you have already tried to run Karesansui with other web server, you need to remove existing files with the following command:
# rm -fr /var/log/karesansui/*log
Restart service and enable it for auto start at bootup.
# /sbin/chkconfig httpd on
# /etc/init.d/httpd restart
# This section is needed for mod_fastcgi only.
# chmod 777 /tmp/dynamic
# chown apache:apache /tmp/dynamic
# /etc/init.d/httpd restart
If SELinux is set to "Enforcing", apache may not work properly. Set it to "Permissive" mode and try to restart the service again.
# /usr/sbin/setenforce 0
# /etc/init.d/httpd restart
Moreover, if you need to use "Permissive" mode at the next reboot, you have to modify /etc/selinux/config file as well.
SELINUX=permissive
####6. Checking for connectivity to Karesansui management console
Please access the following URL from your browser after starting the server. The username and password values for HTTP basic authentication must match the ones specified during database creation 'Creating database for karesansui'.
https://[your-server-name]/karesansui/v3/
###Procedure for CentOS 6:
####1. Installing packages.
Install several packages from EPEL(Extra Packages for Enterprise Linux) repository.
# wget ftp://ftp.iij.ad.jp/pub/linux/fedora/epel/6/x86_64/epel-release-6-7.noarch.rpm
# rpm -Uvh epel-release-6-7.noarch.rpm
# yum install nginx spawn-fcgi
####2. Configuring group members.
Add nginx user to kss group and kss user to nginx group.
# gpasswd -a nginx kss
# gpasswd -a kss nginx
####3. Configuring nginx settings.
Copy the sample configuration file bundled with the source code to the location of nginx config directory, and edit it if necessary.
# cp ~rpmbuild/karesansui/sample/nginx/karesansui.conf /etc/nginx/conf.d/
####4. Start web service
If you have already tried to run Karesansui with other web server, you need to remove existing files with the following command:
# rm -fr /var/log/karesansui/*log
First, run the karesansui program using the built-in server.
# su -s /bin/bash kss -c "KARESANSUI_CONF=/etc/karesansui/application.conf SEARCH_PATH= /usr/share/karesansui/bin/karesansui.fcgi 127.0.0.1:8080"
Restart service and enable it for auto start at bootup.
# /sbin/chkconfig nginx on
# /etc/init.d/nginx restart
####6. Checking for connectivity to Karesansui management console
Please access the following URL from your browser after starting the server. The username and password values for HTTP basic authentication must match the ones specified during database creation 'Creating database for karesansui'.
http://[your-server-name]/karesansui/v3/
collectd is a daemon which collects system information periodically and provides means to store performance data. In Karesansui, It is used to display the statistics graphs.
###Procedure for CentOS 6:
You may need to change the settings for the collectd plugins. Please follow the steps described below.
####1. Edit collectd configuration file. (/etc/collectd.conf)
Global settings
Hostname your.host.name
FQDNLookup true
BaseDir "/var/lib/collectd"
PIDFile "/var/run/collectd.pid"
PluginDir "/usr/lib64/collectd/"
TypesDB "/usr/share/collectd/types.db"
Interval 10
Timeout 2
ReadThreads 5
LoadPlugin section
LoadPlugin cpu
LoadPlugin df
LoadPlugin disk
LoadPlugin interface
LoadPlugin libvirt
LoadPlugin load
LoadPlugin memory
LoadPlugin network
LoadPlugin rrdtool
LoadPlugin uptime
LoadPlugin users
Plugin configuration
<Plugin df>
ReportReserved false
ReportByDevice true
ReportInodes false
IgnoreSelected false
</Plugin>
<Plugin disk>
Disk "/^[hs]d[a-f][0-9]?$/"
IgnoreSelected false
</Plugin>
<Plugin interface>
IgnoreSelected false
</Plugin>
<Plugin libvirt>
HostnameFormat name
Connection "qemu+tcp://127.0.0.1:16509/system?no_verify=1"
IgnoreSelected false
RefreshInterval 60
</Plugin>
<Plugin rrdtool>
DataDir "/var/lib/collectd"
CacheTimeout 120
CacheFlush 900
</Plugin>
####2. Restart collectd service.
In order for all the modifications to take effect, you need to restart collectd service.
# /sbin/chkconfig collectd on
# /etc/init.d/collectd restart
Is there something wrong, outdated or unclear in this document? Please let us know so we can make it better. If you can contribute anything, we would greatly appreciate it!